In addition to what @rjdavid said above, I think the most challenging part of doing it this way would be managing dependencies – suddenly, all Meteor packages and libraries would break and have to be updated, which could be challenging for those who are using Atmosphere packages that are not all necessarily well-maintained anymore. Granted, the lack of maintenance is a long-term problem for any dependency anyway. But this would be a painful breaking change since you’d be at the mercy of the maintainers to update every dependency before your app would work again.
2 Likes
Oh yeah, that would be devastating…ha…ha… Ok well here is a refinement that builds on the idea that we should not have 2 migrations and not clutter the API with gross *Async and does not destroy meteors entire reputation in the process:
During the transition you could add a second set of your Mongo collections underneath your OLD collections, so you can have the old sync API and the new async API in use at the same time:
// Create Mongo collections
const Posts1 = new Mongo.Collection('Posts'); // sync api
const Posts2 = new Mongo.CollectionAsync('Posts'); // async api
// Query data, no "fetchAsync"!
Posts1.find().fetch() // old sync api
await Posts2.find().fetch() // new async api
The advantage to this is, that when the developer has finished their migration and they no longer need Collection (because they deleted them all), they are done. Done, done; No second migration. Seems easier to me.
I just went back and started to refactor some fiber code to promises and…I love fibers. I don’t want to see them go. fibers is meteor
I don’t think it’ll be as severe – these migrations will have more than a few months in between.
That’s exactly what we wanted to do and decided not to. Go read the text 
1 Like
2 Likes
It’s not an easy problem. The real challenging thing is that some Meteor APIs are sync client side, but async server side. (Such as Mongo’s Collection.find() etc.) Changing to Promises means both sides have to change, in order to retain isomorphism, including all the various tools built on top of them. And that’s just not easy.
But we can’t keep Fibers, they just aren’t supported any more. It has to be done.
6 Likes
also be aware that compatibility with node is not the only problem with fibers.
fibers change the way javascipt code is executed, it changes some semantics. This makes it partly incompatible with the rest of the js ecosystem.
e.g. consider you are a library author and you have some api where the consumer can pass in a function:
const myLibFunction = (userCallback) => {
doSomething()
// call user callback
userCallback()
doSomethingElse()
}
without fibers its always guaranteed that even on multiple calls of myLibFunction the order will always be
doSomething → userCallback → doSomethingElse
The common JS-runtimes guarantee this behaviour (node, browsers)
If you enable fibers however, the user can pass in a userCallback function that fiber-yields. If myLibFunction is called multiple times in a short time, the order of execution might change:
doSomething (first) → userCallback (first, yields) → doSomething (second) → userCallback (second, yields) → doSomethingElse (frist) → doSomethingElse (second)
This is not a constructed problem, this happens. I have seen multiple occasion where this leads to unexpected bugs. Doing SSR with react is a prominent one, see this issue.
another one is memoizee
This is also the reason why we have to use async and await keywords. Those are not just to annoy you, those show an important contract about the execution model of a function.
10 Likes
I’ve read your outstanding article. Congratulations on the efforts !!!
I have seen you are trying to make the fibers-free Meteor still backwards compatible, including all existing packages, right?
What if you don’t (on purpose) ? Why not developing a new fibers-free Meteor, incompatible with all previous packages? That would at least force the rewrite of all existing, many of them already deprecated anyways, and the start of a new ALL working new atmosphere system?
Wouldn’t a brand new start make good for Meteor’s 10 year anniversary ?
1 Like
That is one side of the argument.
Then the other side is that hundreds (thousands?) of production apps relying on these packages that will become hostaged of that approach. Dozens of those packages will never be updated, node 14 support is ending, and a lot of Meteor apps will not be able to migrate to supported node versions.
2 Likes
Yep !!! Thousands of apps that depend upon these packages will not be able to upgrade to the new Meteor. So what ???
What if Tiny just splits Meteor in two, a fibers-one with it’s lock on Node 14 and a compromise to maintain it for the next 5 years and a second one, a fibers-free Meteor, with a new name, with a new Marketing strategy behind it, a new GitHub repository, new modernizing features etc…, no way for a newbie to find outdated packages, tutorials, StackOverflow answers, a new strategy to attract NEW developers? Check this forum post I started earlier (didn’t know at the time of the fibers issue): MeteorTosh.
Think of the possibilities !!! Not exactly a brand new start in terms of Tiny development efforts as it would issue an “almost” backwards compatible new Meteor. But a brande new start of NEWCOMERS to the platform. Within time… it could even achieve total backward compatibility in practice… as all the most important Packages could be migrated to the new MeteorTosh, quite like we are doing already with the many community initiates to keep unsupported packages still alive. The newly attracted developers would come with package innovations that would replace completely the old ones…
Think this not just a development strategy, but also as a Company / Marketing strategy.
Well, I’d rather not answer that question if you cannot even empatheticly comprehend what that means on existing businesses
3 Likes
And… I am doing all this AGAINST my own interests ! I have one big application written in Meteor from when Node was still on version 0.10. I am glad that Meteor kept its compatibility strategy over the years as it still working at Meteor 2.7.
But the road was filled with bumps… I had to change the File package 4 times for instance… from cfs:gridfs, to ostrio:files then to vsivsi:file-collection and finally to a version in npm called “cloudinary” that did not use grids… I took out at least 10 other packages and replaced by raw code, other packages or just got rid of that functionality…
So… the package compatibility issue IS PRESENT TODAY! It would not be a novelty for the new “MeteorTosh” **
**Sorry for using this name… It is a way I do to recall the Ambitious Apple strategy to replace the extremely successful Apple ][ for the completely new Macintosh… For years Apple was sustained by the Apple ][ while Macintosh was still loosing money, but eventually the bet paid off.
comprehend what that means on existing businesses
I had to take my business out of and restart over (among other reasons) because Grids packages were not compatible with the new Mongo versions and I din’t have the resources to change that much code at the time.
In any case you can offer TWO complementary paths ! Keep supporting the fibers Meteor for X time, and open the door for migrations and evolution to no-fibers Meteor followers…, The BEST OF BOTH WORLDS !
What you are not taking into account here are the following facts:
- Node 14 support is ending this year. That is the last version of Node that supports fibers
- There are large applications that cannot be updated in one complete update/upgrade. It must be done per module, per sprint, etc.
Once you take these into consideration, you will understand why the current path taken is the one already decided by the Meteor Team (after long consultation with the community; and long discussions which repeats every now and then, like this one)
5 Likes
already decided by the Meteor Team
I see… if it is decided, it is decided… it only rest us to hope and support that you succeed then !!!
I am glad to see you’re so confident with the path chosen !!! My fears that Meteor was trying to “boil the ocean” are over now…
Thanks !
Sorry, but I have to agree with @rjdavid here. Splitting Meteor into two parts would eventually lead to the “old one” dying out over time. This might be a good option if the new “MeteorTosh” was really the “cool new kid on the block” that everybody was jumping on.
Yet there’s quite a chance that it would be recognized as “yet another framework formerly known as Meteor” by the majority of the JS community, at least in the beginning until it picks up. And in the meantime, you lost all the existing enthusiasts who are still using it. So, moving the existing Meteor to a fiber-less base seems to be the better option to me.
I agree in one point, however. There has to be a “hard cut” on Atmosphere at some time to deprecate all packages that are not actively “revived” by their authors. Having a selection of “hand-picked” community packages is not enough, because the old unmaintained ones on Atmosphere are still a “trap”, especially for unexperienced devs.
3 Likes
That can be the next discussion but only AFTER the existing predicament that we are all into is solved because we have 8 months left before security support of Node 14 ends (active support has already ended so bugs are no longer fixed unless they are security issues).
We need that backward compatibility to make it possible to do this upgrade by section of our apps.
3 Likes
Well, the decision has been taken. I agree with the decision as long as it is implemented. It would be more than great to have a non-fibers version with backwards compatibility! I just though that this effort was so daunting that it wouldn’t be feasible, thus my suggestion… But as I see the commitment and trust the people engaged on it has demonstrated, all my fears a gone.
However, I respectfully disagree with you.
I think it wold depend more on how you sell the n platform to the world. Instead of “yet another framework formally known as Meteor”, it could be known as “Market changing new framework from the same people who developed the highly successful Meteor.js”
I love to learn from past experiences of other people and companies. It always comes to my mind the bold actions Steve Jobs took at Apple that eventually saved his company. Hasn’t he moved to Macintosh away from highly successful Apple ][, Apple would be gone by now.
Mac launching was exactly that… “Market changing new computer (which back then was basically a framework) from the same people who developed the highly successful Apple ][”
Die hard users will stay for as long as the fibers version is maintained and keep evolving, just like the Apple ][ community did for years after the Mac.
Apple kept the Apple ][ for years and even launched the Apple III and Apple ][GS (which I bought). Amazingly, even today there is hardware and software being developed for the Apple ][ line… like SD card drives an so on… But, of course, it is a very niche market… Not the one we want for Meteor…
I think it wold depend more on how you sell the n platform to the world. Instead of “yet another framework formally known as Meteor”, it could be known as “Market changing new framework from the same people who developed the highly successful Meteor.js”
I love to learn from past experiences of other people and companies. It always comes to my mind the bold actions Steve Jobs took at Apple that eventually saved his company. Hasn’t he moved to Macintosh away from highly successful Apple ][, Apple would be gone by now.
Mac launching was exactly that… “Market changing new computer (which back then was basically a framework) from the same people who developed the highly successful Apple ][”
Die hard users will stay for as long as the fibers version is maintained and keep evolving, just like the Apple ][ community did for years after the Mac.
Apple kept the Apple ][ for years and even launched the Apple III and Apple ][GS (which I bought). Amazingly, even today there is hardware and software being developed for the Apple ][ line… like SD card drives an so on… But, of course, it is a very niche market… Not the one we want for Meteor…
Meteor initially was driven by investments to build a platform. The landscape back then was very different than today, so meteor team had to develop a lot of tools and things from scratch and developed some great innovations back then. One such novelity was “real-time”-data as default and isomorphic code (That is still kind of unique to meteor). Consider what meteor had to invent back then: bundler, frontend-library (spark, blaze), datalayer (DDP), db-access, cordova-bundling, package manager, …
But the world moved on and better alternatives for the individual pieces came along, we have now a lot of frontend-frameworks, better datalayer abstractions, faster bundlers, etc.
Meteor partially had to open up to some of these technologies, because keeping up with the ecosystem around was no longer feasible. And demand from industry regarding scaling (both in code and performance) just got bigger and bigger.
The team around meteor had then the challenge of maintaining a lot of pieces and try to keep up with demand on the same time. Eventually, the original company kindof pivoted and focues only on the datalayer and eventually founded Apollo. While apollo (or graphql in general) was widely adopted outside of meteor, meteor itself never got the “magic graphql integration” we all hoped for. You could use meteor as an apollo server, but you did not need meteor for that and it added little benefit. So many either started to use more targeted tools like nextjs, apollo-client & server, etc. and went away from meteor or they stick to the initial core concept of meteor (maybe with a more modern ui-library).
Tiny eventually stepped in to maintain both the core meteor project and its hosting platform galaxy. I guess the motivation was to maintain it for the existing user base, evolving it where it makes sense, but without going too far away from its core. Given the complexity of the project, this is still a large area to maintain and room for improvement can be challenging.
Now regarding fiber and why this a big challenge:
As elaborated, many meteor users still use the “traditional” way of using meteor with mongo-collections, ddp, pub-sub, methods and tracker. Those code bases also often rely heavily on code-sharing, mainly in methods. Its these code-bases that are now affected by this fiber-deprecation. The path of migration might be easy in some projects, but it can get very hard in others (also given that probably many of those code bases are not in typescript). But there will be migration needed and @radekmie and others work hard on making this as painfree as possible.
Try to maintain fibers would also have been a possibility, but that would mean adding just another (very complicated!) piece to maintain for Tiny. And it would also mean that meteor would stay longer exactly where it is now and where it was for a long time. So I would take this breaking change as an opportunity for meteor to make a step in the future after a long time.
About creating some “meteor tosh” or “new meteor”: As initially mentioned, meteor had some big investments, it was not a community driven project nor something that evolved from another project (e.g. ruby on rails was abstracted out of Basecamp). It was built to create a disruptive new platform which eventually will get monetized.
Doing that again is not so easy nowadays and there are way too many alternatives (altough more fragmented) and meteor already proved whether such a platform can be successfull or not. So I doubt that we will see something like meteor any time soon.
But of course, everyone is invited to invision and start something similar! Even if its not a success, you will still learn from that!
3 Likes