Meteor Scaling - Redis Oplog [Status: Prod ready]

@alawi you asked how this works in my view, it’s super barebones:
Mongo.Collection.update|insert|remove -> push to redis
publications listen to redis -> push to client

@ramez
Those “2” monitors that listen to oplog, are required to do the following:

  1. Maintain a list of all the active subscriptions in the system (from all 5 nodes)
  2. When anything is inserted/updated/removed they have to scoop through all active subscriptions and see if its related and update them

What you are doing this way, abstracting the oplog reader, is a good idea, and it may work, but it’s totally different from my idea, that’s all, and it’s much harder to implement.

In my view, oplog will be removed completely, there will be no more need for oplog tailing :slight_smile: at ALL, the app will publish changes to redis.

Regarding additional risk and complexity, we are now talking about scaling up, ofcourse you will need different layers to maintain as you scale horizontally/vertically, so I will disregard this as a concern. Plus, it’s super easy to install redis locally, and redis can work very well via an internal network.

2 Likes