Agreed. I was able to vastly improve the number of concurrent users for the Blonk app (to 400 or so) by using Meteor methods to fetch data and then upsert it into a local minimongo collection.
This allows you still use the same query techniques but the data won’t update… saving unneeded re-rendering and CPU cycles in a lot of cases.
[quote="jacobin, post:20, topic:13716"]
They also discussed jumping through hoops to limit subscriptions.
[/quote]
Yea, you def. won’t have the same developer experience as on the tutorials using autopublish.
One company I heard of was sloppy and could only get 10 concurrent users per box and was spending like 15k a month on servers.
This is interesting and something I’ve been thinking about for a while myself. Does anyone know of a neat, simple way to stop a publication being reactive?
Realistically, only things like forums or chatrooms need to be fully reactive. Most other things (such as user profiles etc) don’t need to be.
If it stays as is it will be a major issue as successful apps will be forced to move on from Meteor relatively early (and that will ultimately lead to people avoiding Meteor altogether).
With that said, because of the above I think both the community and MDG will find solutions to these problems. It’s just a question of how long that takes.
I believe @msavin still has a package to this but it uses a reactive var instead of minimongo.
Which, at times can be better than minimongo because the data doesn’t get mixed together. For example in Blonk we needed to pull in a users profile in the chatroom but also didn’t want it to get mixed up in the profiles feed. Since they could both be on screen at the same time it takes hacks to mark the different data before it hits minimongo.
Surely there should be an easier way to do this? I think it’s great that Meteor is reactive, but you’d think it would be easier to make it un-reactive as well…
I would estimate that only 5-10% of webapps need Meteor’s reactivity. I think Meteor should make a simple api for static data retrieval.
When developing my app, which will have tens of thousands of simultaneously connected clients, I try to avoid reacitve connections as much as possible, due to server overhead they incur and use Methods everywhere.
Full Stack Reactivity. In Meteor, realtime is the default. All layers, from database to template, update themselves automatically when necessary.
You can see this when you look at the application protocol itself. Pub/Sub/Unsub/addedbefore… If you get rid of reactivity entirely then you might as well just use REST over the wire instead of DDP.
The problem though isn’t so much the reactivity, but that the oplog tailing Meteor has sitting behind DDP is more a hack than a technology.
When people have to jump through hoops using their own hacks to purposely avoid using Meteor’s core feature then something is wrong.
Hopefully one day Hansoft releases their Meteor fork so Meteor will at least scale to 1000 users per node with reactivity.
In the meantime you could instead just use Phoenix feeding Redux on the client, keep reactivity everywhere and scale to hundreds of thousands of concurrent users on dedicated $60 a month commodity hardware.
Regarding that Hansoft part I feel like we mainly need to change our thinking regarding applications and databases.
Mainly splitting various functions not only to microservices, but also separate databases.
Like every view to be composed of few apps. And every of that app is tailing just it’s own DB.
Maybe RethinkDB will/can solve it, but for now playing with app architecture should be able to help a lot.
But there are not very well known patterns developed yet how to connect to XY microservice apps.
We are already used to template level subscriptions and variables, I think we should develop/learn to use template level DDP.
In the meantime you could instead just use Phoenix feeding Redux
I think learning an entirely new framework and its tooling - that’s what called ‘jumping through hoops’.
You can see this when you look at the application protocol itself.
I am mainly a front-end architect and use Meteor because I don’t want to have my head full with non-frontend stuff like data transfer, db integration, build chain, and so on.
Sometimes I have to look for non-standard ways for my goals, but for the most part Meteor does its job and does it well.
And - God forbid! - I don’t want to be anywhere near technical stuff like protocol intricacies.
Even architects need to have at least a basic knowledge of structural engineering. Otherwise they end up handing the poor civil engineer totally impractical designs.
I guess “frameworks” in general are generics tools used mainly for prototyping. Once your app is online and successful, you might have to rewrite part of the code to adapt it to your specific use (i guess, i have never been past that point).
Meteor is great for RAD prototyping of ideas. I would call meteor more of a prototyping platform than a framework.
Refactoring and rewriting isn’t something that stops once you’re in production. Everyone from facebook on down has to switch out technologies and try new things as they scale up.
My experience with gamedev was no different. Techniques that worked for 700 concurrent users burst at the seams when it hit 800. What worked for 1000 fell apart at 1200. When it hit 1400 users though, I’m sure it would have been fine for 2k.
With a framework or hand crafted stack you can switch out pieces easily as you see fit. Don’t want to use XYZ api as it doesn’t work for you? Just use something else. Code to interface.
With a tightly coupled platform you don’t have that luxury. That’s why with meteor, you’re stuck with Mongo even though sql support has been the #1 request for ages. It’s not outpatient surgery but would require an extended hospital stay and rehab.
MDG does seem to be trying to rectify that on the frontend. I just wish it were the backend.
I doubt they want to be seen forever as as a framework that is “just for prototyping”. Especially as other frameworks pop up that offer rapid prototyping and the potential to scale. But, you have to start somewhere.
