We have a project which has hundreds of methods. We’re about to migrate them to use new async API and it will took us sometime to do that.
Do you know some tools/ways which can speed up this process?
Thank you.
3 Likes
At the very least, I am hoping for a linter e.g. eslint rule.
2 Likes
@minhna This is a problem we all share, and I’d dearly like a solution (other than “just suck it up and spend some weeks recoding”). The worst case is that you have to visit every server method and upgrade it, which will take a lot of time, and introduces huge risk to the project.
If a method only does one thing (like update the database) it’s not too bad, make the method async, and do an await.
If a method iterates over an array or collection, there is likely to be .map() or .forEach() calls, neither of which are await compatible, and have to be replaced with for of loops. This makes a direct codemod ineffective. Perhaps a codemod could draw attention to the need for change?
In an ideal world we could work out a way to fix this automatically. At the other end of the scale, brute force coding is an option. I’d like to think that collectively we could build an automated solution that takes us 50% of the way,
Does anyone want to put some effort into this collectively?
Hopeful
That should be a linter
1 Like
I’m writing a jscodeshift transform script which can help migrating current codebase to use new async api. Hopefully can be published soon.
Can jscodeshift take care the function call chain that will need to be come async with the changes?
Yes, something like this:
Links.find({}).map((i) => {})
become
await Links.find({}).mapAsync((i) => {})
How about the calling functions from other files which imported the updated function to async?
export function Sample() {
return Links.find().map();
}
Can it update the entire AST of calling functions?
It’s out of my scope for now. I think this script can help you speed up the migration progress, you will need to review all changes and run your tests against the changes.
In an automated script, I find that very dangerous. Maybe, it can be done per file to ensure that only functions in that file is updated.
For a project with a large codebase that might require multiple sprints to complete the migration to async, the steps I am envisioning is something like:
- Open a file
- Check what must be changed to async
- Update all related files that imported any affected function from this file
- Commit
What I have for now:
Transform this file content
const someFunction = () => {
debug('run some function')
const a = Codes.findOne({})
Codes.update({ a: 1 }, {})
Meteor.users.findOne()
}
function otherFunction() {
debug('run other function')
Codes.findOne()
Codes.insert({})
Codes.upsert({})
Codes.update({})
Codes.remove({})
Codes.createIndex({})
Codes.dropIndex({})
Codes.dropCollection({})
// cursors
Codes.find({}).map((i) => {})
Codes.find({}).count()
const a = Codes.find({}).fetch()
Codes.find({}).forEach((i) => {})
// cursors variable
const cursor = Codes.find({})
cursor.map((i) => {})
}
const functionShouldStayTheSame = () => {
return ['a', 'b'].map((i) => `hi ${i}`)
}
Meteor.methods({
someFunction,
otherFunction,
functionShouldStayTheSame,
alsoThis: functionShouldStayTheSame,
'literal.method': function ({ p1, p2 }) {
return Links.find().fetch()
},
'literal.method.arrow': ({ p1, p2 }) => {
const myVar = Links.find()
return myVar.map((i) => i)
},
})
To this:
const someFunction = async () => {
debug('run some function')
const a = await Codes.findOneAsync({})
await Codes.updateAsync({ a: 1 }, {})
await Meteor.users.findOneAsync()
}
async function otherFunction() {
debug('run other function')
await Codes.findOneAsync()
await Codes.insertAsync({})
await Codes.upsertAsync({})
await Codes.updateAsync({})
await Codes.removeAsync({})
await Codes.createIndexAsync({})
await Codes.dropIndexAsync({})
await Codes.dropCollectionAsync({})
// cursors
await Codes.find({}).mapAsync((i) => {})
await Codes.find({}).countAsync()
const a = await Codes.find({}).fetchAsync()
await Codes.find({}).forEachAsync((i) => {})
// cursors variable
const cursor = Codes.find({})
await cursor.mapAsync((i) => {})
}
const functionShouldStayTheSame = () => {
return ['a', 'b'].map((i) => `hi ${i}`)
}
Meteor.methods({
someFunction,
otherFunction,
functionShouldStayTheSame,
alsoThis: functionShouldStayTheSame,
'literal.method': async function({ p1, p2 }) {
return await Links.find().fetchAsync();
},
'literal.method.arrow': async ({ p1, p2 }) => {
const myVar = Links.find()
return await myVar.mapAsync((i) => i);
},
})
1 Like
To my understanding, both codemod and jscodeshift work on ast level but not on a whole project worth of files which is required if you want to ensure that 100% of call chains are converted to async/await.
I wonder if Webstorm can help. I haven’t run it for a while
It might be difficult to handle all cases if some functions are not handled the regular way.
For example, in most of our async function, we used this simple await-to-js package to easily handle results and errors: await-to-js - npm
Difficult to handle these custom cases
I think the per file transformation will do
I think the only gotcha for the transform is if there are objects with similar methods in the code that are not collections.
You’re right. I made a check to make sure it’s a collection by checking it’s import parth. But it’s very specific for our codebase only.
For example we always define our collection in a schema.js file, so we check if the definition comes from that kind of source.
I didn’t got time to check all of the examples above, but here are my two cents: making all of the calls use their async counterparts (find → findAsync) and adding await in front is enough, as it’s a syntax error to do await without an async context, so you’ll be able to track them easily. Sure, there will be a lot of false positives (e.g., [].find(...)), but I think it’s already “good enough” to start.
2 Likes
I just checked our custom eslint rule also related to Collections and indeed, it is very specific to our code
if (
!success &&
bodyNode.type === 'ImportDeclaration' &&
bodyNode.specifiers?.length
) {
bodyNode.specifiers.forEach(specifier => {
if (
specifier.type === 'ImportSpecifier' &&
specifier.local?.type === 'Identifier' &&
specifier.local.name === collectionName
) {
if (
bodyNode.source?.type === 'Literal' &&
bodyNode.source.value.includes('/collections/')
) {
importedCollection = true;
}
}
});
}
I’ve never used jscodemod but this seems like a great idea and would save a lot of time. Could it be configured to take an array of collection names ? That way we could all just pass our collections and have them modified?
Yes, this is great, being able to run it on a single files means we can do the migration gradually, and it will save a lot of tedious time, even if it only does 50% of the work, that’s a huge saving already. You can run it, and then look at the diff using git difftool to see what it has done, and check through the changes.
Excellent work @minhna
1 Like