Thingking about Meteor 2.0 🤔

MDG seems like a company with top tech talent with little appetite for marketing or PR. I agree with your statement, it’s not just about technology it’s also about trust and communication. With that said, I really hope as a community we keep the conversation constructive and see how we can help to make Meteor 2.0 a great release

Agreed. And its fair to say that I won’t be moving to any other platform / framework any time soon. I think its a nice end to “MDG” topic in this thread. :slight_smile:

2 Likes

Agreed. Can we stick to the original topic please.

2 Likes

Couldn’t sum it up better myself. I think unless something major, that we have missed comes up, we have a list of things that are of the interest to community.
Support and strategy isn’t something that we can affect much, on the technical side though we can contribute as well, though it would be nice to hear @benjamn thoughts and as a lead developer he needs to be in agreement with this direction or we are not going to get anywhere. For the Apollo part that is something that will need collaboration from that part of the company, but we could probably flesh out what we see as the “special” thing between Meteor and Apollo. So this is where I would like to direct the discussion going forward.

10 Likes

Love love love! Nice work community!

1 Like

Well, now comes the hard part. Actually making it happen…

No, I don’t support killing DDP and Minimongo as these are core parts of what makes Meteor Meteor and provide functionality that many long term users (like me) rely on in our Webapps.

Providing the option to use/not use DDP and Minimongo or to programmatically turn them on and off is fine.

As for Fibers, I am not sure what the issue is. Promise.await() is easy to use and works perfectly when calling promise-enabled functions or those that use async/await

Ben Newman gave an excellent talk in 2015 to explain why Meteor should retain the use of Fibers.

Slides:

http://benjamn.github.io/goto2015-talk/#/

I sincerely ask - what has changed since Ben’s talk that would make a complete migration to async/await worth doing?

Are there any issues with Fibers in Node.js 10?

6 Likes

I totally agree on the Minimongo and DDP part. Its for me one of Meteor’ biggest strengths. Not sure about Fibers though. Its on one hand confusing for developers coming from other platforms. On the other its a very convinient way of writing.

2 Likes

As I said before here, the concept Meteor means different thing to different people, so we’ve to keep the discussion in the perspective/context of how the tool is being used.

I agree, I use Minimongo and DDP myself but that was @sacha statement from here. And I think they didn’t mean to kill minimongo/DDP but to make it an optional package since those using GraphQL don’t really need the web sockets open which consume server memory and has slight initiation delay at the client, so making it optional, in my opinion, is a reasonable ask since they’re using Meteor just as a backend for a GraphQL end point.

I’m not sure either but folks who advocating for it are saying it’s a barrier to move Meteor to NPM, it’s holding some developers coming from the rest of the NodeJS ecosystem off guard, and they might also be barrier for contributing to Meteor core, those points have been echoed several times in this thread but I think that requirement could be flushed out more because, for me, it still sounds vague but maybe it’s my lack of experience with Meteor core. You can see @gaurav7 comment here when I questioned why Fiber is an issue. Also, Ben did highlight some of the issues that comes with using Fibers, especially catch 3 in his slides. I think in his last comment you can see that he favoured the tradeoff of simplifying Meteor usage (developers experience) on the expense of the effort it takes Meteor core contributors to learn and maintain Fiber (contributors experience).

1 Like

Well I think you started a good thread, I think it’s worth outlining and capturing the community needs/pain points/sentiments. But yeah execution is another story, I’m seeing progress on the Vue guide, which I think is a real service to Meteor that the community is well positioned to independently help with.

4 Likes

@vlasky see my earlier comment highlighting the parts of Ben’s presentation where he seems to acknowledge the points being echoed in this thread w.r.t Fibers.

Sure Fiber makes many things appear “easy”, as long as it works. But it’s a nightmare when it does cause a problem, the likelihood of which increases as you scale and/or use more 3rd party libraries. So far, the “ease of use” argument perhaps rightly trumped over the cons, justifying the tradeoffs. But there was no alternative then. Now we have one, in the form of async/await, and the best part is that it’s a proper standard!

Not only would removing Fiber make Meteor more approachable to other Node.js app developers and authors of libraries & tools, but it would also save the few precious resources currently working on the framework (mainly Ben, as per github stats) from spending time on unproductive work (i.e. no new framework feature) that Fiber keeps demanding… just look at some of the current open issues to get an idea:

All this just to pretend that we’re writing synchronous code on a platform that’s inherently asynchronous! Is that really worth all the trouble now that there’s a good/better alternative?

As an example of the overhead it would eliminate: the whole meteor/promise package won’t be needed if Fiber were to be removed! Note that Promise.await and Promise.async are not standard APIs.

7 Likes

Yes exactly this is why I’m now supporting simpleDDP and am building my vue connectors for it.

It feels to me that the red line here is pluggability. It takes quite an effort just to understand Meteor’s internals. Making this easier and putting developers in control over what functionality they require is crucial. That’s one thing I love about Vue and Nuxt. The plugins

1 Like

Do you think Fibers can be removed in a backward compatible way (i.e legacy app code still works as is)? It sounds to me that removing Fiber from the Meteor core is going to be a major refactoring given the potential impact/testing required.

I’ve personally never experienced an issue with underlying cause pertaining to Fiber usage and we’ve tons of third-party npm packages so I can’t relate to that statement, and being on this forum for sometime, I don’t see many issues pertaining to Fibers usage either. Also it seems coroutines are emulating multi-threading and multi-threading bugs are nightmare in general. However, there are some hard corner cases issues reported on Github pertaining to Fiber use such as this one…

Human mind is naturally sequential thus having Meteor backend synchronous by default is in an advantage and one less hurdles for newcomers before getting their todo list render on the screen.

Actually I’ve the same question toward removing Fiber, is that really worth the trouble? If I understood Ben’s talk correctly, coroutines (Fibers) is a more generic/flexible approach toward async code when compared to async/await and he demonstrated how async/await can be implemented using coroutines furthermore it does enhance the developers experience (in Java we’ve synchronized blocked, other languages (Kotlin for example) have co-routines as well as Ben’s alluded too). There is a great talk from Kotlin authors explaining why coroutines are superior to async/await, they made very similar points to Ben, you can find it here. So my personal take on the refactoring of Meteor Fibers, that it’ll be a major undertaking (which I think an effort could be spend elsewhere) that doesn’t improve the developers experience (in fact it might degrade it) and I really don’t think it’s the main hurdle for contributing toward Meteor (I think it’s more marketing issue) so for me the refactoring is not justified, but again I keep an open mind, I think it’s debatable topic.

1 Like

@alawi, agreed.

Maybe we need to be public with a way to abstract it out for NodeJS developers thinking of using Meteor, via stubs for Promise. If I look at some of our method calls, we do use Fibers a lot for async calls (e.g. communicating with AWS, files etc.) so I can see why developers would currently have no choice but to interact with Fibers.

But refactoring Meteor to extract Fibers would be writing a whole new framework.

Sorry maybe a little off topic, but I’m a bit confused on why using Fibers for async calls on the server? I’m a bit puzzled why I didn’t have to interact with Fibers, I do make a lot of async calls to third party APIs/filesystem but I just use promises/async await or Meteor.wrapAsync, so when exactly do I need to use Fiber, am I missing something, why I haven’t experienced the “Fiber pain” yet :sweat_smile:?

1 Like

When you make a call to a Meteor method, and that method depends on an async call (e.g. pushing a file on S3) how else would you do it?

We return a future.wait() in the Meteor method, and in the callback from S3 we return future.return(result) or future.throw(error)

How else would would you do it?

EDIT: Fixed text to replace fiber with future

I just checked Meteor.wrapAsync (which we don’t use – maybe we should change that). It takes care of that for you. It also does the Meteor.bindEnvironment for the callback.

Maybe this is the solution. Maybe Meteor.wrapAsync could be the starting point of using Promise (developers don’t care what’s under the hood)

1 Like

Yes, I used Meteor.wrapAsync for the Stripe calls convert the async calls to sync and I wait for the results, the code block reads sequentially thus making it easy to reason about, that’s why I said removing Fiber might actually degrade the developer experience. For file uploads, I push to S3 from the client. In both cases, I wasn’t even aware of Fiber when working on them!

By the way I highly recommend Rob Fallows article here, really helped me to wrap my head around Meteor’s async.

1 Like

Yeah very likely it would be a rewrite at least for the parts that are tightly coupled with fibers. That’s why I brought this up only in the context of a hypothetical Meteor 2.0 (which I wish doesn’t stay hypothetical for too long :slightly_smiling_face:).

The latest versions of AWS SDK have .promise() methods. So you could also do something like

async function upload() {
   const response = await s3.upload({}).promise()
}

or pretending synchronous code through implicit use of fibers:

function upload() {
   const response = Promise.await(s3.upload({}).promise());
}

So I must clarify in my push for removal of fibers, that it’s entirely possible to write async/await code in Meteor today. And with the non-standard helpers like Promise.await and Promise.async you are able to interact with external promise-based APIs.

// node_modules/external/index.js:
export function asyncCall(callback) {
  callback(err, result);
}

// non-Meteor Node.js
import { asyncCall } from 'external';
asyncCall((err, result) => {})

// Meteor Node.js
import { asyncCall } from 'external';
Meteor.wrapAsync(asyncCall, this)((err, result) => {})

Which one has a better Developer Experience? :slight_smile:

To be fair, DX is also a function of what you’re already used to. When Meteor was launched, Node.js was also fairly new, there was no (popular) Rails-eque fullstack web-framework, callback hell was one of the early criticisms of Node.js. So giving the appearance of synchronous code might have been a masterstroke then to attract developers from other platforms like PHP and Java.

But today, Node.js has arguably surpassed the popularity of the traditional platforms. Co-routines are not idiomatic to Node.js for whatever reason. So if Meteor wants to go back to “mainstream” and lower the barrier to entry, I believe it should eliminate aspects that are not familiar to Node.js developers trying out Meteor.

So the use of fibers not only enforces technical tradeoffs that are not worth anymore, I believe it has also made Meteor framework “not worth the extra learning curve at a fundamental level”, as modern alternatives keep coming up.

3 Likes