I’d like to get in touch with others that have had challenges scaling meteor.
It’s a topic I’ve had to deal with myself and expect to continue to deal with in the future.
I’m interested in skyping with people about this topic and if this something that interests you please private message me. I’d be happy to share how I’ve dealt with the challenges till now and would be very interested to learn how others have dealt with it to be prepared for the future.
The sort of scale I’m talking about is 1000+ concurrent active users.
If people are also interested in discussing in this thread I’d be happy to. The simplest yo start scaling meteor is just to run more and more meteor instances and load balance between them.
I agree, horizontal scaling is one of the easiest way, and Meteor can indeed scale. There may be indeed some problems with MongoDB oplog and it’s reactivity when you have a lot of traffic.
We used classic elb from aws, and it’s working great for us.
The next steps we want to take is to have all of our assets delivered from a CDN so basically each time we deploy migrate /public folder and also the bundled javascript/css that loads, so our meteor servers are only responsable with the websocket. This is in my opinion the way to go, bc Node sucks at delivering statics assets
Another question I would like to address is MongoDB location, we are now experiencing some issues, because we use mlab, the data comes in very slow. We are thinking of creating a MongoDB cluster ourselves, how others solved this ?
I use nginx to do the load balancing. I host meteor on digitalocean and
have my mongodb with compose in the digitalocean data centre in the same
location and have never noticed any speed issues between meteor and the db
but I’m not sure how I would measure that either.
For CDN that’s an easy fix. You can use cloudfront or cloudflare for free.
The issue with cloudflare is you need to make sure the load balancing still
works if you stick cloudflare in front of it since if you use ip_hash for
the load balancing algorithm with nginx, that depends on knowing the user’s
ip address.
If you’re on aws then just using cloudfront may make more sense.
I serve all my static assets using cloudfront. The one exception is the
main html, js and css files for the site. The reason I don’t use cloudfront
for these is that if I have two meteor instances running different versions
of the site, then cloudfront may not be able find the the correct files to
serve the user and then the site won’t load for the user. One way to get
around this would be to always make sure that all meteor instances are on
the same version but this would be even be an issue during updates of the
site.
In addition to this you can get nginx to cache static assets for even less
load on node.js.
But the real issues I’ve felt have been adding more servers no longer helps
things. Apparently connecting too many instances to your mongo db can cause
performance issues.
Oplog and the cpu required for publications observing the oplog seem to be
the biggest scaling pains.
One thing I’m interested to hear about is if people have tried splitting up
publications into different meteor instances. Basically all this instance
does is run publications for a single collection. That would be a really
nice way to scale. I just wonder how many connections each instance could
handle and how exactly you’d connect to the multiple instances from the
client. What the best way to structure such a system would be. Sending DDP
messages between servers may be the way to go with this approach and that
may be what meteor cluster package is all about.
I’d also be interested to hear about hitting mongo bottlenecks with too
many reads or writes per second. And how people have dealt with that and
when you can expect to hit these limits.
If I’m doing an update to 20 servers, each server gets updated one by one,
and there will be at least a 20 minute period where servers are on
different versions of the code.
There’s no reason that all 20 instances need to be serving the exact same
code to be honest. There have been times on my site where servers may have
been on different versions for a few days even. If you want to slowly roll
out an update you might do this for example.
I’m interested in following the redis oplog discussion. I hope it’s
something that works out. Still very early days.
Redis-oplog reached beta. is currently stable, been in sync with a lot of people they tested it in their environments we found some edge-case bugs but we fixed them.
It wouldn’t hurt the package to just plugit in your app and do some testing, not necessarily put it in production.
is currently stable, been in sync with a lot of people they tested it in their environments we found some edge-case bugs but we fixed them.