Migration to 2.8 Async version

I have started the migration of a large application. I have a few doubts concerning some primitives that maybe some of you have already dealt with:

  • forEachAsync,mapAsync: the callback remained synchronous, in case an asynchronous function needs to be called within it (e.g. an additional call to mongo) , it should become asynchronous.
  • observe: in the synchronous version, the method added for records already in the db was called before terminating the invocation, in the asynchronous version should it be considered asynchronous?
  • Blaze: will using a find in response from a helper still work? Should all functions that have become asynchronous in minimongo (e.g. findOneAsync) be converted to reactiveVar when used in a helper?
1 Like

Hey @pbeato how are you doing?

For that blaze question we do have an issue here on blaze repo and I think @storyteller & @jkuester may have a better understanding on this

forEachAsync,mapAsync question: I think it would be okay, because as you are waiting on the promise to be solved it should make sense.
You can make you forEachAsync and mapAsync into for: as @radekmie said in his pr

await (const document of collection.find(query, options)) /* ... */

for the observe question I could not understand properly. Can you provide some pseudo/real code so that I can think on top of it?
like you are saying something like this?
there is this first pr as well with more examples of asyncMongo code

I’ve been wondering the same thing regarding observe and observeChanges. In the docs, it says

Before observe returns, added (or addedAt ) will be called zero or more times to deliver the initial results of the query.

This is frequently used to differentiate between docs that already exist, and new docs. How does this work with the async api?

An example of how it currently is used:

let initialAdd = true;
const handle = cursor.observe({
  added(id) {
    if (initialAdd) {
      // Doc that already existed in mongo or minimongo
    } else {
     // Is new doc
   }
  }
});

initialAdd = false;

well thanks @grubba, I hope for you too.

the solution proposed by @radekmie surely works, it was to avoid rewriting dozens of forEach/mapEach. Otherwise I will write a jscodeshift codemod to do it :-(.

Regarding observe I quote the @zodern in the previous post.

It would not hurt, at this time of migration, to create a set of shared recipes for common problems.

1 Like

The goal, for now, was to make the async API available, though the code is still synchronous. Making forEachAsync and mapAsync work with async functions is a sane requirement. However, MongoDB driver itself, requires forEach callback to be synchronous and uses the returned value a way to stop the iteration (source). Therefore, at least for now, I wouldn’t change it at all.

As for the observe and observeChanges methods, I don’t have the answer yet. From the top of my head, making them return not only stop but also something like isInitial flag or even initialAddsSent promise seems feasible, although I’m not sure if an entirely good approach.

As for the Blaze, I guess the problem is similar to the one in React (adds fetchAsync to useFindServer by eduwr · Pull Request #360 · meteor/react-packages · GitHub). I don’t know how to tackle that either, but you can find some ideas in the PR.

I return to the subject for other considerations.

The Meteor.user() call most likely becomes asynchronous, right?

The invocation of server-side methods (Meteor.call…), now synchronous due to fiber, I assume will become asynchronous, right? Now, we use a promise version of call, but only client-side.

Do any of you have an idea of the approach that will be used in the kernel in replacing fibre. If we use async/await logic most of the functions will become asynchronous, the impact is almost like writing a new application.

Thanks

Yes exactly. You will need to assume that every single serverside method X in meteor core and every package will be replaced by an XAsync method that you must switch to and adapt your own code accordingly. That is the price we must pay for replacing fibers with async/await.

As for client side code and if it will be possible to keep code isomorphic the jury is still out.
There are some cases already where it seems hard and client side code will need to keep invoking the existing sync versions like collection.findOne

Just think of the Tracker, cornerstone of all Meteor reactivity, but still synchronous and based on global variables. The question I have is: Is it worthwhile to start porting to the asynchronous client-side version of Mongo with still all these blind spots?

@radekmie Throw new Meteor.Error(‘‘not-authorized’’) is no longer thrown in an async method.

example:

new ValidatedMethod({
  name: 'getOrgPeopleAndRoles',
  validate: new SimpleSchema({
    _id: String
  }).validator(),
  async run ({ _id }) {
  if (!this.userId) { throw new Meteor.Error('not-authorized') }
 // ....

The error will not throw but the method will be interrupted. Throwing the error in an async context requires a catch apparently.

Example:

new ValidatedMethod({
  name: 'getOrgPeopleAndRoles',
  validate: new SimpleSchema({
    _id: String
  }).validator(),
  async run ({ _id }) {
  
  const checkUser = async () => {
     if (!this.userId)  { throw new Meteor.Error('not-authorized') }
  }
 
  checkUser()
     .then(() => // method content here )
     .catch(e => console.log(e)) // but I can only console log it, it does not reflect on the client like it used to.

In the catch is where I can log the error on the server console but as mentioned it does not show on the client side.

Are there any suggestions for how to throw an error like if (!this.userId) { throw new Meteor.Error('not-authorized') } in 2.8?

This should be part of the recommended code updates in the migration guide I think.

There most likely be something like Meteor.userAsync for the time being.

There are already .callAync and .applyAsync: callAsync - New method to call async functions by denihs · Pull Request #12196 · meteor/meteor · GitHub.

We’re still figuring out the way to make this work.

I tried to understand the problem but your example is incomplete. Please create a reproduction and file an issue on GitHub.

I might have tagged you by mistake on this one, I thought you were part of the Meteor team.
Anyway, the issue I was referring to in my previous message is related to Error handling in async in NODE, in general. The fact that we move to async, changes the way NODE Error works.

More details can be found here. I am just confirming that this is also the case when using async methods in Meteor. Search down for “The perils of async try/catch”: Node.js Error Handling Best Practices: Ship With Confidence

I know that async context handles erorrs differently, but I still don’t understand what’s the problem – if you await it, your error will be “catchable”.

Thank you @radekmie

As discussed in previous threads, maintaining minimongo’s client-side synchronous functions in the future could be a first step towards supporting node 16. I understand that we would have to revise Meteor’s isomorphic logic, but at least we would have the client working as before and we could concentrate on server-side changes. Already the transition to the server-side asynchronous version will be heavy and complex, adding the client side as well could be untenable. Guaranteeing everything by April may not be plausible.

On the application side, we have to make a lot of changes to adapt to the new architecture, this also has a timeframe that will naturally go beyond April 2023, we will have installations in production without Node support, which is worrying.

I would like to discuss these issues for proper planning of our releases.

Hello everybody, we also have an application that uses React + Redux on the client and Meteor on the server side.
In particular, we use Minimongo on the client side to obtain the collection modification events and consequently perform the status update in redux.
For this purpose we have some code written exactly like in the example of @zodern .
I agree with @pbeato on the fact that in a first phase it would be worth keeping the synchronous Minimongo solving the node compatibility problem only on the server side.
Thanks for your help.

I have tried using zone.js within Tracker without success because zone.js does not intercept native promises (async/await, see #11670). Now, does anyone know of an alternative library to do this?

I tried with the legacy bundle and zone.js works but it feels like going back in time.

Never mind, I found a satisfactory way to deal with it by calling throw new Error('not-authorized') instead of throw new Meteor.Error('not-authorized').

With Meteor.Error I can catch the error on the client, print it and it retains the err => err.error message as thrown on/by the server. However, with Meteor.Error the error is not thrown on the server console. I don’t get a

Exception while invoking method '.......'  Error:not-authorized

With throw new Error(), I get the behavior from the “old” sync methods, it throws an exception on the server and I can catch it on the client, however on the client it does not retain the error message and it only shows a generic error: 500 which I am fine with.
I will have to dig some more to see what is changing in the Meteor.Error in the async environment cause it definitely behaves differently.

I have started a MCP discussion related to this, I want to coordinate updating the MCP packages and establish new common patterns on how to deal with any issues that come our way. Now with 2.8 released and more on its way as you upgrade I would very much appreciate everyone’s feedback on this in the discussion:

1 Like