Meteor 3.0 async frontend? (Minimongo, Tracker, etc.)

[rjdavid]

@zodern

This might be more of a communication issue rather than a technical issue. It’s getting better, but there has been a lack of clear, official information on Meteor 3, which might have made this less straightforward for projects. Also, the current docs for preparing for Meteor 3 imply that the client also needs to migrate to async, which is mostly unnecessary and might mislead people in how difficult it would be to upgrade to Meteor 3.

Agree. Clarity on this will bring confidence on a migration path.

What’s worse than a difficult migration is not knowing what that migration looks like. This brings a lot of anxiety on projects.

[illustreets]

@theosp

For extensive projects like mine, the prospect of moving away from Fibers is a formidable one, raising serious questions about resource allocation. The choice between upgrading to Meteor 3 or, regrettably, considering alternatives, is not straightforward.

I know what you mean. It looked scary at first sight. Our product is by most measures also complex - as an indication, we have our own 100+ internal packages, besides everything else. Then, on top of MongoDB, we communicate with several internal services over HTTP. All this was written using the old Meteor “synchronous” approach.

What I hope may be good news for you is that it works quicker than expected, at least on the server side. We are over 50% of the way to rewriting everything using async and await, and it has been a rather painless process. Of course, it depends on the IDE you use, so it can highlight when you are calling async functions without prefixing them. Also, depends on how modular your code is, how many tests. But there are parts of our code base that are a bit challenging in that respect, and those, if anything, benefited from a bit of tidying up.

No way that, in our case, a complete rewrite would have been faster. Conversion, so far, has taken roughly 200 dev hours. Frontend, on the other hand, is a totally different matter. Also, isomorphic code that relies on Mongo will be a thorn on the side, especially in third party packages that now need to be forked. But nothing insurmountable if you ask me.

[filipenevola]

We need to keep in mind that more and more npm packages will rely on new Node.js APIs incompatible with the Fibers approach, so keeping Fibers running for longer will not help Meteor in the long term. The best example of these problems is APM agents (like Datadog).

Meteor 2.x is a good alternative for people who need to run Fibers code longer. Still, ESM will be impossible in a few months as security fixes are now targeting a very distant version of Node.js.

So IMO what we have right now is the best possible scenario considering all the variables:

• Meteor 2.x with ESM support until it’s not possible (already with Async APIs available);
• Meteor 3 focuses on the future (running Node.js 20).

For those uncertain about the future, please ask your questions in an issue on GitHub, and we can submit PRs to have the answers in official docs.

[jam]

@zodern

the current docs for preparing for Meteor 3 imply that the client also needs to migrate to async, which is mostly unnecessary and might mislead people in how difficult it would be to upgrade to Meteor 3

Excellent point. This should be emphasized in the migration docs.

[rjdavid]

@filipenevola

ESM will be impossible in a few months

@filipenevola

Meteor 2.x with ESM support until it’s not possible

To start, I hope Meteor moves away from using its own semantics and follow the broader JS community. For someone unfamiliar, dropping support for ESM in Meteor 3.x would be a crazy move.

[alawi]

I think they mean extended security maintenance. The one promised with the custom Node 14.

[rjdavid]

Thank you for highlighting my point.

In an ideal world, there is no need to do such explanations given that ESM mean something else in the broader JS community

[alawi]

Ah yes, I see your point.

Anyway, I think communication is critical, and I hope the core team learn the historical lessons from MDG. Lots of a great work and planning is going to Meteor 3.0, last thing they want to is blunder the communication.

[marklynch]

@grubba

Hey!
For the front end, nothing will change, and the methods with their *Async counterpart should not be affected when 3.0 hits.
For the server, it was decided to remove the non-*Async methods, 3.0 using these methods will result throw an error

the current docs for preparing for Meteor 3 imply that the client also needs to migrate to async, which is mostly unnecessary and might mislead people in how difficult it would be to upgrade to Meteor 3

This is good news. I’m assuming it comes with the caveat that there’s no shared code touching async APIs in the mix?
So if I use minimongo in some frontend code it can remain unchanged? But if in my frontend I call some shared logic, that code will need to become async aware correct ? But I don’t see how this shared code can work - and less so how the migration path would look if on client its sync and on server it’s async. Apologies if this has been cleared up elsewhere but I’m confused

[alawi]

@marklynch

Apologies if this has been cleared up elsewhere but I’m confused

I share the same confusion. I’m assuming the main issue at the client is when using stubs and shared methods with the server?

[jam]

Yes, that’s the way I understand it.

For isomorphic code, you’ll need to write asynchronous code using the *Async suffix.

If you’re interacting with Minimongo on the client, you can write synchronous code. I think the most common example here would be using .find().fetch().

[marklynch]

So I’m guessing this will lead to a decision in many places where you either

a) Keep your code isomorphic and then have your frontend ‘infected’ with async
b) Make a client and server specific version of the code so you can keep your frontend sync.

[filipenevola]

@Alawi

extended security maintenance

Yes, that is what ESM meant in this thread.

[filipenevola]

@marklynch

So I’m guessing this will lead to a decision in many places where you either

a) Keep your code isomorphic and then have your frontend ‘infected’ with async
b) Make a client and server-specific version of the code so you can keep your frontend sync.

Yes, that is correct.

Many Meteor projects don’t share code; in these projects, the methods always run on the server only, so it’s important to make this distinction.

[marklynch]

Thanks for the confirmation Filipe
Thankfully we only run about 2 or 3 simulations, so our methods are 99% server only. But we do have plenty of logic that’s in common code and called by both sides. I’m guessing the simplest would be to go with b and just avoid sharing code that touches things like minimongo.

[waldgeist]

My biggest concern ist actually not my own code, although it will take a while to migrate it. My concern is the 3rd party packages I am using. A lot of their maintainers have left the ecosystem. I’m not sure if I will be able to patch these myself, and the list is pretty long.

Just as an idea: How about setting up a list of packages that have been forked and patched for 3.0, even if these forks might not be “official community packages” yet?

[illustreets]

I understand your concern. I have been wrestling with this issue as well. But, there are some good news here as well:

1. The Meteor Community Packages are rather well looked after. If you have a patch to contribute I’d suggest you do it there directly so everyone can benefit from it.
2. If it is a package that looks unmaintained, perhaps try and search for a recent fork and contribute to that one if it is not already updated. I was rather pleasantly surprised to find quite a few of these on GitHub.

Well, maybe this is the break that needed to happen. After all this, whatever packages have api.versionsFrom('3.0.0') are the ones to keep moving forward.

As a side note, this may also be a great opportunity to trim down the fat of external dependencies.

@waldgeist

Just as an idea: How about setting up a list of packages that have been forked and patched for 3.0, even if these forks might not be “official community packages” yet?

Both Atmosphere and Packosphere could do with an update to either filter or highlight the minimum Meteor version for each package version. The latter is the work of @copleykj, so I would imagine he may be able to do it, depending on availability and sponsorship.

[waldgeist]

@illustreets

If it is a package that looks unmaintained, perhaps try and search for a recent fork and contribute to that one if it is not already updated.

Yep, my idea was to make it easier to find them by maintaining a fork list.

@illustreets

Both Atmosphere and Packosphere could do with an update to either filter or highlight the minimum Meteor version for each package version.

Good approach. But this would only work if the fork would be pushed to these systems, wouldn’t it?

[illustreets]

@waldgeist

Good approach. But this would only work if the fork would be pushed to these systems, wouldn’t it?

You are right. But, lists are cheap, and in my opinion unreliable and quickly out of date. The best we can do is support the people behind Meteor Community Packages, who is the most suitable organisation for this kind of stuff (unless Meteor Software picks up the initiative, that is). I’d imagine they could do with some help at the moment.

[jam]

@illustreets

Both Atmosphere and Packosphere could do with an update to either filter or highlight the minimum Meteor version for each package version

. There was some discussion on this here: Proposal on Meteor 3 packages compatibility showcasing - #10 by grubba

Maybe the way to do it would be to highlight packages if api.versionsFrom equals or includes 3.0. I recently updated a package for 3.0 compatibility but my minimum version remains 2.8 so I don’t think we can rely solely on minimum version since I imagine there will be other packages that will maintain compatibility with 2.x for some time.

@illustreets

lists are cheap, and in my opinion unreliable and quickly out of date

True but maybe it’s a fast way to get started. Crowdsource a Google doc while the real solution makes its way into Atmosphere / Packosphere.