Meteor Scaling - Fusion Mode [Status: Beta]

Make sense, to get that going, I’d assume we’ll need to change Meteor core DDP package?

Yes, it can still be improved, I was able to get around 800ms on my local machine on fresh Meteor react project. However, as stated, as far as I know, we currently don’t have control over the merged CSS injected by Meteor (those you in-line in the JS files). I’ve moved the imported public CSS out of the critical path, and in-lined all the CSS I need for the server rendered login page, but any CSS I wrote in the code gets merged and injected at the head, and I can’t instruct Meteor to delay that.
imagehttps://forums.meteor.com/uploads/default/optimized/2X/2/2c18eb8b91441687b8c5a19ca5cf46dc2e8232b7_1_690x349.png

The first interactive paint is what I think you’re aiming to improve, and DDP/Sockets initialization seems to be slowing that.

1 Like

Truth. I posted a question here: https://github.com/meteor/meteor/issues/9349#issuecomment-343724758 as I need some input from the guys that coded ddp-client (Looking at you @sashko)

Indeed, this will also improve server performance and ability to handle more users that view the page at the same time. My problem is why establish an websocket if you don’t really need it ? This makes Meteor suited only for some type of apps. What about presentation pages, shops, many others ? Why not have the ability to become reactive only on some special pages ?

1 Like

I’m sold on conditional sockets! makes total sense. Also I think MDG might face similar decision/DDP refactoring as soon as they plan tighter integration of Meteor with Apollo. If I’m not mistaken, I think Apollo by default runs on HTTP and only leverage web sockets when subscriptions are defined on a page.

I’ve been toying around with it for a bit and I made a bit of progress. Managed to integrate it with Accounts Password, Meteor calls now work with this.userId. So I managed to hack the authorization layer via the HTTP.

The problem I’m now tackling is the automation of engaging in a DDP connection once the first .subscribe() is done. However, there is a big problem, because Accounts automatically subscribes to ‘meteor_loginAccountsConfiguration’ by default, so basically it would mean that if you have Accounts, the first subscription automatically engages the DDP, so it’s useless.

I think it would be best to manually do this:
DDP.engage()
DDP.disengage()

And let the developer full control over this. For example, you can do things like (if you’re on the page for a long time without any activity, just cut the ddp loose, and resume it when you come back)

BOOM!

Conditional websockets in Meteor are now a reality. I’m curious how many bugs you find, it’s currently a POC, but it should work just fine in 99% cases. Read the README. (It works with Accounts)

4 Likes

This already works so nice with “server-render”. Basically you can use react-router 4, with a BrowserRouter for client and StaticRouter for server, and pass in the same routes! It’s the first time I used it and I’m amazed (need to add static-html and remove blaze-html-templates) for it to work.

And if for example you are subscribing or using createContainer() || withTracker(), you can fetch data directly server-side and pass it to the components, no change required! But you have to be careful to only fetch what’s needed. And also maybe do a Meteor.subscribe = function () {}; a dummy function on the server.

I am in awe, Meteor is now so performant I can’t believe it. We have ability to work without websockets but still benefit of authentication and meteor calls, we have server-side rendering that blends so nicely with React. We can now reach very large scalability, especially if we put .css and .js into a CDN.

And you can very easily cache the SSRs so it’s way way speedier:

Absolutely beautiful.

1 Like

Wow, great work as usual @diaconutheodor ! I’ve being using React Router V3 and manually handling server side routes, so this setup is definitely an the right way forward.

React is becoming a standard in front-end development and it’s growth is not slowing down, so first-class support of React by Meteor will definitely increase the framework adoption. In Meteor’s roadmap they’ve a section for “Out of the box support for advanced React features”. If we’ve something like meteor create react-app that would generate a React skeleton with SSR, React Router V4 with dynamically imported components at the route and conditional DDP then I think it’d be a great head start for any app. If you’ve a blog or static page without reactivity, you can just hit that command, deploy and be sure that your page will load instantaneously and it would scale as any node server (no tailing or the extra overhead of socket management at the server). As the site/app grows in features, reactivity can be added on demand using DDP engage and Redis-Oplog. I think all the solutions for react are here, they just need to be packaged and presented the Meteor way to the user and it’d be better then Next.js for React since it’s more flexible.

I’m really excited about this, I’ll give this a spin with a side project to see how it behaves.

1 Like

Regarding meteor create react-app :slight_smile:

Since I already had it for testing purposes… why not?

2 Likes

Thank you!! I knew you had something for testing, you read between the lines, I just didn’t want to ask for it directly :wink:

But I think we need to have something below, right now it has only simple-todos-react.

1 Like

We should have a sort of aggregated list of meteor boilerplates somewhere.

And sorted by the number of stars, and completely searchable.
If you want to add a new boilerplate, just add a GitHub Repo link.
Meteor will interogate the github api, sync stars daily. And comments (via disqus) enabled at each repository.

And ofcourse it’s going to use the “fusion” package because we don’t really need an WS opened :smiley:

Any thoughts? I think this would be extremely useful, instead of just adding a boilerplate to react app, some people may want a react+redux+redux-router, others want a react+mobx+XXX

I think this will be useful for people starting new projects and old-school Meteor devs (like me) to try out cool new stuff.

1 Like

I think the idea is great (I’m currently in the process of moving my project to Vue and having others’ setups to look at is of much help), but it would probably be easier to make this happen without the direct involvement of MDG - i.e. not have this list shown in the shell with meteor create and instead just have a simple website available with links to repos with the clone command right in the middle. I’d say git clone is not much more trouble than meteor create in the shell.

Having the boilerplates tied directly to meteor create would mean that MDG probably would need to verify the code to ensure that it meets a certain standard (both with regard to its quality and absence of malicious code), as users would assume that such example code is approved by MDG and safe to use. This would be burdensome on MDG and even if MDG would have the resources to do that (which I doubt) the whole process would be too slow for the list to gain momentum. So I think this is a perfect opportunity for the community to step up and make something happen, and help MDG out in the process.

1 Like

I think this is great idea as well. So basically something like NPM/Atmosphere but instead of packages it’s for Meteor boilerplates.

Do other modern frameworks have something like that? does anything similar exist?

@vooteles
You understood correctly my idea.

@alawi
I haven’t seen, I searched a bit but no ciggar.

This can be a great opportunity to test fusion and ssr :smiley:

1 Like

Next.js maintain a list of example repos on Github, but as @vooteles stated this approach might end up with bottleneck in the maintenance and growth of the boilerplates list in addition to draining of resources from the core dev team, so I’m completely agree, a standalone community powered platform makes a lot of sense.

Agreed! an instantly loaded SSRed/Fusioned meteor app with all the active meteor boilerplates, sounds like another great addition to the Meteor community!

Here is the closest thing I found for react that implements some of those requirements, since the data in that tool is maintained centrally, you can see many requests to add boilerplates in the comments section, so it could definitely be improved and made into a standalone platform.

Another approach I found here in which they’re using a GitHub to maintain a list of boilerplates involving mobx, again this is not ideal since it’s centrally managed without any search capabilities.

And some developers just blog (example of a blog made a month ago) about the latest state of the boilerplates ecosystem which is obviously less then ideal.

This seems like a gap in the entire ecosystem frankly and there is opportunity for Meteor to lead the way - yet again :wink:

A little of topic, but I wanted to mention it’s already possible to use DDP for Apollo. For queries and subscriptions:

To be honest, there is no reason to expect a much tighter integration from MDG. But that’s for another topic.

1 Like

Great idea. Render it server-side everytime. If the user clicks real fast, serve up the next one again from the server. Later the app takes over. Nice.

@tomachi you know what’s the beautiful part? This is all currently possible. Fusion works amazingly well, and SSR with React Router v4 works very good too! You can now host your meteor app on a 10$ instance and handle a lot of traffic!

1 Like

@diaconutheodor You might be interested to know that there is currently some related internal Meteor decoupling work underway. Since the WebSocket connection used on the client side of a Meteor app is heavily tied to DDP, you can’t use Meteor’s WebSocket connection without also using the ddp-client package (which is fairly large). All of this is about to change:

This doesn’t bring in conditional WebSocket support (like Fusion), but it does provide a way to leverage Meteor’s WebSocket connection without having to depend on Meteor’s DDP related libraries. This should open up another wide range of possibilities for Meteor apps.

5 Likes

@hwillson
I believe that separating websocket from ddp is a natural idea, keeping things decoupled. But it won’t help us in anyway if we don’t need a websocket connection.

I also saw that there’s a lot going on in ddp-client, so after Meteor’s 1.6.1 release I will have to adapt the package as well to make it fit. And I hope that before we reach 1.7 you can add fusion as a package and automatically extend this behavior.

1 Like