Thingking about Meteor 2.0 🤔

[alawi]

That’s an overly negative impression which comes from the mindset that Open Source projects should always be committing or they’re abandoned. Meteor is being under development for 6 years now, it’s unrealistic to expect a commit/feature rate of the project to be similar to that under heavy construction. By your definition I’d say 90% of open source projects are just abandoned. It’s like looking at a building which is almost complete, and comparing it to a hole in the ground and saying look construction rate is not the same, that’s just absurd.

There are 2.2k views on this thread alone since it has been posted, obviously a lot of people care. Also the “The single developer” did an incredible work in bringing the system up to date, I’ve seen many projects with 10+ devs not accomplishing similar feat due to lack of skills, poor management and/or decisions. And for the packages, there has been clear push toward using NPM packages, but the community did create Redis oplog and other crucial Meteor specific packages when needed.

The biggest issue surrounding Meteor up to date are the FUD and the skewed negative opinions not being countered by PR.

[ignl]

If you think that the biggest meteor problem is lack of PR against negative opinions then we are looking at it completely different and the best is just to agree to disagree…

[cloudspider]

I feel that I need to do a lengthy one here. I’m sorry…

That’s an overly negative impression which comes from the mindset that Open Source projects should always be committing or they’re abandoned.

The javascript land is changing continuously. It’s not about being abandoned, its about it not being kept up to date with the rest of the system. Sailing upstream is awesome, but when the river starts flowing faster then your boat, it might be time to jump off and take another one. OR you start peddling. And I AM peddling, but for that I need to have peddles… Right now my pull requests arnt even being accepted. Do I really need to become the boat. fork the whole project, built my own sites around it, etc?

Meteor is being under development for 6 years now, it’s unrealistic to expect a commit/feature rate of the project to be similar to that under heavy construction. By your definition I’d say 90% of open source projects are just abandoned. It’s like looking at a building which is almost complete, and comparing it to a hole in the ground and saying look construction rate is not the same, that’s just absurd.

The biggest mistake developers and businesses make is to look at projects like this and mark it as “done”. These are projects that need continous updates, refactors, regular interactions with the community and if possible new features aswell. Ecmascript would have been swept away by now by other modern languages if it was still in its old v5 form.

I suggest you read Martin Fowlers “Refactoring” book. Its about ‘evolutionary systems’. It fits nicely on this whole topic: https://martinfowler.com/

There are 2.2k views on this thread alone since it has been posted, obviously a lot of people care. Also the “The single developer” did an incredible work in bringing the system up to date, I’ve seen many projects with 10+ devs not accomplishing similar feat due to lack of skills, poor management and/or decisions.

I totally agree with this. Meteor and some of its contributors have archieved much more then whole teams combined. Most projects also fail because of the reason you mentioned. I do feel however that its not the MDG people’s skill level. Its lack of Management, priority and Marketing that kills it atm. I totally understand their business decision though. Meteor with Galaxy is a cashcow and it will be for some time. It had its investments, but Apollo seems to be a more strategic option given its momentum.

However, Meteor is no exception. All popular frontend projects have some really bright people that could easily replace a team of 10+ devs. Evan You? Maybe Dan Abramov?

Do I agree with MDG’s decision to go full Apollo and leave the Meteor project? No! In the end it will bite MDG, because if they drop Meteor’s support, why wouldnt they do that with Apollo as soon as momentum fades when it enters the Trough of Disillusionment on the Hype Cycle?

And for the packages, there has been clear push toward using NPM packages, but the community did create Redis oplog and other crucial Meteor specific packages when needed.

You’re highlighting some of Meteor’s biggest pains here. Hence why I like this original topic. Should there be a v2? Maybe one that breaks with Meteor’s package system? What about Fibers? What about Minimongo and DDP? Would Angular 1 still be here if they didnt decide to break with its legacy stuff? Good question. Great topic to talk about with the community.

The biggest issue surrounding Meteor up to date are the FUD and the skewed negative opinions not being countered by PR.

Yes and no. Most of the things that people mention are true (harsh but true)! Meteor ‘was’ not suitable for most websites that simply required SSR and a static API. MDG ‘is’ not active on topics, pull-requests and not even social media atm.

Compare it with React, Angular, Vue, Ember, Nuxt, Next and Laravel (Laravel is I think older then Meteor and is still actively being developed, maintained and talked about). The competitors that were not even in Meteor’s water, are catching up with toolsets like Webpack that directly compete with one of Meteor’s current USPs: Hot Code Pushing and its build tool: vue-loader for example is using HMR and is really fast and easy to configure. SSR out of the box in Next and Nuxt anyone?..

MDG did so much great things. It has brought some awesome technologies on the table, but if they keep on ignoring us, it will become a dead end.

[gaurav7]

Technology:

• Get rid of fibers and convert all core APIs (including core packages) to be promise-based. (from here and here
• Source maps support
• Do something unique between Apollo and Meteor that only Meteor can do or is best at
• More front-end agnostic, meaning no Minimongo, DDP
• Hot module reloading and SSR
• Improvements to listen internal events not just by hacking and wrapping some methods. (from here)

Support & Strategy

• Official Meteor 2.0 LTS
• Make Meteor first class citizen
• Better communication especially form upper management

Documentation:

• Vue Guide/Tutorial

Good summary, @alawi! Some of these might not be so drastic as to necessitate a v2.0, but I agree with the list in terms of things that Meteor’s roadmap must include, for it to not only stay relevant but really attract new adopters.

[gaurav7]

MDG did so much great things. It has brought some awesome technologies on the table, but if they keep on ignoring us, it will become a dead end.

MDG looked like taking some significant positive steps around Meteor’s project governance just before starting the work on Apollo. But then Apollo did so well and became the de-facto GraphQL client, leaving even Relay behind – must have exceeded even MDG’s expectations. I totally understand the attention and effort they are putting behind the project and the ecosystem.

I think all that we need now is for MDG to state clearly what their plans are with Meteor. Updating the roadmap with goals for 2019 would be a good way to kickstart the new year.

[gaurav7]

On the first point (fibers) here is something to consider (nice Ben Newman’s presentation, links to slides in the abstract): https://www.youtube.com/watch?v=bxaOGDqVPKw

@hluz, I’m not sure if you’re using that reference to support continuing with Fibers? Could you clarify?

My interpretation of the presentation is that Fibers helped Meteor to adopt async/await in a backward compatible way. But I don’t think Benjamin made any arguments about Fibers being worth sticking with even in future. In fact he seems to support the idea that async/await is the solution (slides: 20, 21) for coroutines not being suitable for Javascript (slides: 18, 19)

I think nobody contests the reasoning behind adopting Fibers when Meteor was first released. That was probably a great decision at that time, especially since Promise was not common and async/await were unheard of in JS-land. But I also believe Meteor faltered in not incorporating the mainstream Node.js conventions fast enough. Situation improved drastically since Meteor 1.3, but I’m afraid that Fibers is one big obstacle still remaining (and releasing Meteor as npm packages would be the other thing, but Fibers is a bigger problem imho).

Also, the problems that I alluded to in my previous comments are in fact detailed by Benjamin himself in this presentation; see slides 40-50

If some developers still find Fibers (co-routines) making sense to their projects, it could be an opt-in (just like it is for other Node.js projects, using node-fibers) – maybe Meteor could make that a tad easier through a compiler plugin/package.

[alawi]

Martin Fowlers “Refactoring” book.

Good reference, it’s my second favourite after The Mythical Man-Month

These are projects that need continous updates, refactors, regular interactions with the community and if possible new features aswell

I agree with that statement, I never said it’d be marked as done, but my point was the rate of development can’t be the same as when the project first start, this can be observed in all system life cycles, the amount of energy it take to bring a system from nothing to something is far greater then what it takes to maintain and evolve it.

Meteor started 6 years ago, it’s certainly passed the hype cycle, and I speculate that it’s in the middle of it’s life span. Reflecting on the web technology industry history, it takes on average 10 years for a new wave of innovation to kick-in (PHP/Java/Net – 10 years --> Ruby on Rails – 10 years --> NodeJS — 10 years -> ? ) one might argue that the rate of change might be increasing but there are constraints in adopting technology, people build businesses around them, there is natural construction speed limit, selling and maintenance periods which can’t be avoided so the 10 years mark makes sense. So I suspect the NodeJS ecosystem (and Meteor is the most mature and developed framework within the Node ecosystem) has another 5 to 6 years before it starts trending down. Nuxt, Next and create-react-app are the niche frameworks relative to Meteor now, they’re doing what Meteor did few years back with Blaze each capitalizing on the raise of view framework, SSR or some other micro-trend, but really Meteor moved first to the Node frameworks and that first mover advantage can’t be underestimated (and MDG is doing the same with Apollo now).

I think all that we need now is for MDG to state clearly what their plans are with Meteor. Updating the roadmap with goals for 2019 would be a good way to kickstart the new year.

I agree with that statement, hence my comment for LTS support. While their push on Apollo is understandable but Meteor is a foundational tech and it’s an entry to an ecosystem yet MDG is pushing hard on Apollo and they’re allocating ex-meteor developers to Apollo which I personally not very thrilled about. But the transparency of Open Source projects is both a gift and a curse, this resource shuffling happens a lot in private companies as they invest in strategic future projects without the knowledge of the external stakeholders however in Open Source, everything is in the wild, and when the rate of commits drop, without clear communication from the leadership team, the community starts to panic and the FUD increase.

MDG did so much great things. It has brought some awesome technologies on the table, but if they keep on ignoring us, it will become a dead end.

MDG seems like a company with top tech talent and little appetite for marketing or PR. I agree with your statement, it’s not just about technology it’s also about trust and communication. With that said, I really hope as a community we keep the conversation constructive and see how we can help to make Meteor 2.0 a great release

[cloudspider]

MDG seems like a company with top tech talent with little appetite for marketing or PR. I agree with your statement, it’s not just about technology it’s also about trust and communication. With that said, I really hope as a community we keep the conversation constructive and see how we can help to make Meteor 2.0 a great release

Agreed. And its fair to say that I won’t be moving to any other platform / framework any time soon. I think its a nice end to “MDG” topic in this thread.

[robfallows]

I think its a nice end to “MDG” topic in this thread.

Agreed. Can we stick to the original topic please.

[storyteller]

Here is my attempt to consolidate a meaningful list from the discussion so far.

• storyteller items from here.

Technology:

• Get rid of fibers and convert all core APIs (including core packages) to be promise-based. (from here and here
• Source maps support
• Do something unique between Apollo and Meteor that only Meteor can do or is best at
• More front-end agnostic, meaning no Minimongo, DDP
• Hot module reloading and SSR
• Improvements to listen internal events not just by hacking and wrapping some methods. (from here)

Support & Strategy

• Official Meteor 2.0 LTS
• Make Meteor first class citizen
• Better communication especially form upper management

Documentation:

• Vue Guide/Tutorial

There is a lot of good stuff in this thread, I think a good doc can be created from it. Also it’d be nice if we can have this somewhere that we can vote on.

Couldn’t sum it up better myself. I think unless something major, that we have missed comes up, we have a list of things that are of the interest to community.
Support and strategy isn’t something that we can affect much, on the technical side though we can contribute as well, though it would be nice to hear @benjamn thoughts and as a lead developer he needs to be in agreement with this direction or we are not going to get anywhere. For the Apollo part that is something that will need collaboration from that part of the company, but we could probably flesh out what we see as the “special” thing between Meteor and Apollo. So this is where I would like to direct the discussion going forward.

[mattblackdev]

Love love love! Nice work community!

[storyteller]

Well, now comes the hard part. Actually making it happen…

[vlasky]

No, I don’t support killing DDP and Minimongo as these are core parts of what makes Meteor Meteor and provide functionality that many long term users (like me) rely on in our Webapps.

Providing the option to use/not use DDP and Minimongo or to programmatically turn them on and off is fine.

As for Fibers, I am not sure what the issue is. Promise.await() is easy to use and works perfectly when calling promise-enabled functions or those that use async/await

Ben Newman gave an excellent talk in 2015 to explain why Meteor should retain the use of Fibers.

Slides:

http://benjamn.github.io/goto2015-talk/#/

I sincerely ask - what has changed since Ben’s talk that would make a complete migration to async/await worth doing?

Are there any issues with Fibers in Node.js 10?

[cloudspider]

I totally agree on the Minimongo and DDP part. Its for me one of Meteor’ biggest strengths. Not sure about Fibers though. Its on one hand confusing for developers coming from other platforms. On the other its a very convinient way of writing.

[alawi]

of what makes Meteor Meteor

As I said before here, the concept Meteor means different thing to different people, so we’ve to keep the discussion in the perspective/context of how the tool is being used.

As for Fibers, I am not sure what the issue is.

No, I don’t support killing DDP and Minimongo as these are core parts of what makes Meteor Meteor and provide functionality that many long term users (like me) rely on in our Webapps.

Providing the option to use/not use DDP and Minimongo or to programmatically turn them on and off is fine.

I agree, I use Minimongo and DDP myself but that was @sacha statement from here. And I think they didn’t mean to kill minimongo/DDP but to make it an optional package since those using GraphQL don’t really need the web sockets open which consume server memory and has slight initiation delay at the client, so making it optional, in my opinion, is a reasonable ask since they’re using Meteor just as a backend for a GraphQL end point.

As for Fibers, I am not sure what the issue is.

I’m not sure either but folks who advocating for it are saying it’s a barrier to move Meteor to NPM, it’s holding some developers coming from the rest of the NodeJS ecosystem off guard, and they might also be barrier for contributing to Meteor core, those points have been echoed several times in this thread but I think that requirement could be flushed out more because, for me, it still sounds vague but maybe it’s my lack of experience with Meteor core. You can see @gaurav7 comment here when I questioned why Fiber is an issue. Also, Ben did highlight some of the issues that comes with using Fibers, especially catch 3 in his slides. I think in his last comment you can see that he favoured the tradeoff of simplifying Meteor usage (developers experience) on the expense of the effort it takes Meteor core contributors to learn and maintain Fiber (contributors experience).

[alawi]

Well, now comes the hard part. Actually making it happen…

Well I think you started a good thread, I think it’s worth outlining and capturing the community needs/pain points/sentiments. But yeah execution is another story, I’m seeing progress on the Vue guide, which I think is a real service to Meteor that the community is well positioned to independently help with.

[gaurav7]

As for Fibers, I am not sure what the issue is. Promise.await() is easy to use and works perfectly when calling promise-enabled functions or those that use async/await

Ben Newman gave an excellent talk in 2015 to explain why Meteor should retain the user of Fibers.

@vlasky see my earlier comment highlighting the parts of Ben’s presentation where he seems to acknowledge the points being echoed in this thread w.r.t Fibers.

Sure Fiber makes many things appear “easy”, as long as it works. But it’s a nightmare when it does cause a problem, the likelihood of which increases as you scale and/or use more 3rd party libraries. So far, the “ease of use” argument perhaps rightly trumped over the cons, justifying the tradeoffs. But there was no alternative then. Now we have one, in the form of async/await, and the best part is that it’s a proper standard!

Not only would removing Fiber make Meteor more approachable to other Node.js app developers and authors of libraries & tools, but it would also save the few precious resources currently working on the framework (mainly Ben, as per github stats) from spending time on unproductive work (i.e. no new framework feature) that Fiber keeps demanding… just look at some of the current open issues to get an idea:

• https://github.com/meteor/promise/issues/14
• https://github.com/meteor/promise/issues/9
• https://github.com/meteor/meteor/issues/10117
• https://github.com/meteor/meteor/issues/10279
• https://github.com/meteor/meteor/issues/9275
• https://github.com/meteor/meteor/issues/10032

All this just to pretend that we’re writing synchronous code on a platform that’s inherently asynchronous! Is that really worth all the trouble now that there’s a good/better alternative?

As an example of the overhead it would eliminate: the whole meteor/promise package won’t be needed if Fiber were to be removed! Note that Promise.await and Promise.async are not standard APIs.

[cloudspider]

agree, I use Minimongo and DDP myself but that was @sacha statement from here. And I think they didn’t mean to kill minimongo/DDP but to make it an optional package since those using GraphQL don’t really need the web sockets open which consume server memory and has slight initiation delay at the client, so making it optional since and using Meteor just as a backend

Yes exactly this is why I’m now supporting simpleDDP and am building my vue connectors for it.

It feels to me that the red line here is pluggability. It takes quite an effort just to understand Meteor’s internals. Making this easier and putting developers in control over what functionality they require is crucial. That’s one thing I love about Vue and Nuxt. The plugins

[alawi]

Do you think Fibers can be removed in a backward compatible way (i.e legacy app code still works as is)? It sounds to me that removing Fiber from the Meteor core is going to be a major refactoring given the potential impact/testing required.

But it’s a nightmare when it does cause a problem, the likelihood of which increases as you scale and/or use more 3rd party libraries

I’ve personally never experienced an issue with underlying cause pertaining to Fiber usage and we’ve tons of third-party npm packages so I can’t relate to that statement, and being on this forum for sometime, I don’t see many issues pertaining to Fibers usage either. Also it seems coroutines are emulating multi-threading and multi-threading bugs are nightmare in general. However, there are some hard corner cases issues reported on Github pertaining to Fiber use such as this one…

All this just to pretend that we’re writing synchronous code on a platform that’s inherently asynchronous!

Human mind is naturally sequential thus having Meteor backend synchronous by default is in an advantage and one less hurdles for newcomers before getting their todo list render on the screen.

Is that really worth all the trouble now that there’s a good/better alternative

Actually I’ve the same question toward removing Fiber, is that really worth the trouble? If I understood Ben’s talk correctly, coroutines (Fibers) is a more generic/flexible approach toward async code when compared to async/await and he demonstrated how async/await can be implemented using coroutines furthermore it does enhance the developers experience (in Java we’ve synchronized blocked, other languages (Kotlin for example) have co-routines as well as Ben’s alluded too). There is a great talk from Kotlin authors explaining why coroutines are superior to async/await, they made very similar points to Ben, you can find it here. So my personal take on the refactoring of Meteor Fibers, that it’ll be a major undertaking (which I think an effort could be spend elsewhere) that doesn’t improve the developers experience (in fact it might degrade it) and I really don’t think it’s the main hurdle for contributing toward Meteor (I think it’s more marketing issue) so for me the refactoring is not justified, but again I keep an open mind, I think it’s debatable topic.

[ramez]

@alawi, agreed.

Maybe we need to be public with a way to abstract it out for NodeJS developers thinking of using Meteor, via stubs for Promise. If I look at some of our method calls, we do use Fibers a lot for async calls (e.g. communicating with AWS, files etc.) so I can see why developers would currently have no choice but to interact with Fibers.

But refactoring Meteor to extract Fibers would be writing a whole new framework.