As the first PR got merged, I wanted to share how I see the whole story of making Meteor Fibers-free, so I wrote a blog post about it: “On the Road to Fibers-free Meteor”. Hope you like it!
(I also posted it in r/javascript and r/meteor; hopefully it brings more attention to Meteor overall!)
Thank you so much for making this happen!
Thanks! I’m excited about that teaser in the end!
Thanks a lot! Looking forward to the next step and slowly starting migrating my apps. Hopefully there will a be a nice guide soon.
Looks awesome! I wonder what impact this will have on useTracker.
Nice article man, I didn’t really get what all the fuss was about but in just a couple paragraphs I think you’ve enlightened me (or at least shed a hell of alot of light)
Am I right in thinking Node 16 will provide threads as a native way to do server/client communication aka fibers or did I get that wrong?
I think we should focus on cursors on the front and we’ll be fine. It’ll require more research (and probably a few more methods or hooks), but I guess it’s going to be fine for most cases.
Fibers are just a way of “implementing” (or rather “harnessing”) asynchronous operations. Promises are also one, and the entire ecosystem went more into them. There are also generators, and while I use them extensively, I don’t think they are even remotely usable without this proposal. (I’d prefer something more general, like algebraic effects, but…)
Anyway, the worker_threads module got introduced in Node 10 and stabilized in Node 12. Then, in Node 14.18.0, they introduced getEnvironmentData/setEnvironmentData and stabilized it in Node 17.5.0. Having that, we could easily replace current Fibers usage in Meteor, i.e., have a “thread-safe” storage. It’s still server-side only, though.
Ohh thanks man for taking the time to explain to my dumb ass, I didn’t realize it’s not thread safe. Got it now.
Seems this topic has been going on for a while, not the first time I’ve heard that.
So in a nutshell we’re going to move to thread safe promises in the near future and that may screw the pooch for many existing, well used packages?
The most important part is to make everything Fibers-free, i.e., use whatever JavaScript has to meet our needs (here: use Promises for everything). Then we could simulate the thread-safe storage we have with Fibers, but I don’t think there’ll be a need for that. (Maybe it’ll have a slightly different API.)
We should be able to adapt useFind and useSubscribe easily enough. It might be possible to do something neat by detecting whether useTracker’s tracked function returns a Promise (an async function would return a promise). Could be fun!
worker threads are still pretty different from fibers (at least how fibers are mostly used in their current form, for better or worse) - their truly shared memory can only be array buffers - so you’d have to either use very raw data, or implement data types ontop of that structure (doable, but not particularly easily).
A reasonable amount of meteor assumes there is a single copy of data - e.g., everything to do with observe multiplexers and handles - without this we’re just storing multiple copies of data for no reason at all. It’s not clear to me that worker_threads help with this at all.
That being said - I don’t think they need to, I believe the async_hooks portion can replace Meteor.EnvironmentVariable instances with relative ease - which (for the most part) is the only place meteor has per-fiber state. Though they also don’t work on the client 
Okay, but without “Fibers” will this still work?
It has to. And as the answer states:
Accounts.registerLoginHandler()is not ready for async/await. It expects a function, not a Promise. (An async function returns a Promise).
We’ll have to make it work with promises – “simple” as that.
This is so cool and you are killing it. One of the reasons I started learning meteor all those years ago was the fibers. That made querying the DB all that more productive and I loved how it was synchronous without blocking. So I’m sad to see it go but I sure hope it winds up being better in the end for everyone.
And not just any function, it needs at best to return an object. with Fibers I have the peace of mind of using promises, (err, res) => { }, or anything new they come up with every fortnight.
In my case this, this very old code, still works beautifully.
Accounts.registerLoginHandler('ldap', function ({ ldap, sAMAccountName, pw }) {
...
...
const future = new Future();
checkLogin(sAMAccountName, pw).then(
res => future.return(res),
err => future.return(err)
);
const user = future.wait();
...
...
return {
userId: user.sAMAccountName,
token: stampedToken.token,
};
});
Can anyone post a code example of a before and after of going fibers-free so we can see the actual implementation and what it means for us?
(Emphasis mine.)
I get it, really. We also have a lot of code that is there, running for years now without any changes. But Promises aren’t new, and it looks like they’re here to stay. What is more, your code of yours requires only a few changes, and is actually shorter once they’re done:
// Change 1.: Async function. Keep in mind that it WILL NOT work now!
Accounts.registerLoginHandler('ldap', async function ({ ldap, sAMAccountName, pw }) {
...
...
// Change 2.: Use the promise directly.
const user = await checkLogin(sAMAccountName, pw).then(
res => res,
err => err
);
...
...
return {
userId: user.sAMAccountName,
token: stampedToken.token,
};
});
(I find it weird, that your code does future.return(err), but I kept it. If you’d rather have the error thrown, then removing .then entirely is enough.)
Example of what? I mean, in 99% of cases it’ll most likely require making functions async and adding await here and there. Additionally, in the transition phase (i.e., when the fibers will still work, but we’ll have to have both APIs there), some functions may have *Async counterparts.