The current Redis implementation of Oplog, redis-oplog lets developers control reactivity efficiently - this has given back reactivity to the app’s control. Meaning you can now build all kinds of apps from games, chat, and other collaborative apps without much scaling issues as opposed to using the mongo oplog for reactivity, which is the default behaviour of the Meteor reactivity stack.
Note: A cache layer can probably be better implemented in Redis. For more schema-able data, MongoDB is better. [Note: both MongoDB and Redis are technically schemaless].
I know this isn’t usable yet, but I’m just throwing something else into the mix. Mongo 3.6 will have a new Notification API - as in a real notification API that will support subscribing to collections (and handling events) out of the box. Depending on how this turns out, Meteor might be able to move away from tailing the oplog. This really could be an incredible step forward. It’s definitely early days still, but super exciting! A FR is in place here, if you’re interested in following along:
Finally, the oplog is a mongod-wide component which under heavy traffic means I may be tailing much more data than I actually need if my application is focused on just one collection or DB.
Ideally, creating an activity stream that is optionally published out would expose this through an API layer independent of the oplog. This would allow for applications like this to no longer directly rely on the oplog.
If we could also filter on the activity stream similar to how audit filters are applied it would allow for only reading the changes you care about as oppose to filtering on the application side and transmitting a ton of data needlessly.
Wow! I’d never heard of this before, but it sounds like exactly what Meteor needs for scaling! That gets me pretty excited
People have been using regular oplog tailing with Redis for years and it’s worked for them. Redis is just another option for (possibly) better scaling, at the cost of more manual and conscious choices around reactivity. No need to go down that path until you hit scaling problems in production which necessitate a change. Premature optimization is often a waste of valuable time that you could instead be spent building a product.
However, I’m STILL not clear on what Redis is. Does 1 instance of Redis automatically oplog tail? Or like MongoDB, do I need to setup 3 instances to create the tail?
I’m currently experiencing database issues with performance, I built an MMO on Meteor. www.StarCommanderOnline.com - you’ll notice connection problems as the server for 10 consecutive people takes 14+ gigs of RAM.