Question with regards to scaling Meteor


#1

If I’m not mistaken, one of the biggest issues with running Meteor is that it takes observers use a lot of CPU. This means getting even 50 connected clients to a single server can be an achievement.

Since 1.3 Meteor has the option to disable oplog on a per query basis as follows:

disableOplog Boolean
(Server only) Pass true to disable oplog-tailing on this query. This affects the way server processes calls to observe on this query. Disabling the oplog can be useful when working with data that updates in large batches.
pollingIntervalMs Number
(Server only) How often to poll this query when observing on the server. In milliseconds. Defaults to 10 seconds.
http://docs.meteor.com/api/collections.html#Mongo-Collection-find

I was wondering if setting pollingIntervalMs to 5 minutes for mostly static publications would help with this scaling issue. Most data for most apps doesn’t need to be updated in real time, so would setting a high polling interval solve the higher CPU caused by many observers? And then only using oplog for publications that absolutely need it?

Any thoughts on this topic in general would be appreciated.


#2

Sorry to bump this, but no answers?


#3

Hey @elie , I believe the answer to your question is yes.
You can get more information here: http://info.meteor.com/blog/tuning-meteor-mongo-livedata-for-scalability


#4

Next, follow this thread: Meteor Scaling - Redis Oplog [Status: Prod ready]

I will post updates there as soon as I have them.

I found a way to use Redis to leverage the pub/sub system + make it fully BC with all of our code, so we dont have to rewrite anything.

And ofcourse finally make Meteor + it’s reactivity scalable.


#5

Looking forward to seeing the redis solution.