Oplog usage on Mongo cluster with multiple databases

help
1

Hi guys,

we recently switched to the Meteor Cloud hosting and enabled the APM Tool.
Running quite good for now, but further optimizations can and should be done.
Some execution traces indicate, that we should use the MongoDB Oplog feature to boost the performance related to the observers.

Situation:
We have a MongoDB Cluster, that is hosting 4 different databases. The Oplog, at least how i understand it, is the operations log for the replica set with all the entries about the recent operations (obviously). That’s supposed to be for EVERY operation done, not caring about the database that got a change (read, write, update, delete).

Problem / question:
Is it bad practice or even not possible to use the Oplog functionality for a shared cluster or is Meteor smart enough to filter the right operations related to its used database? Would it be better to setup dedicated clusters for each database that should benefit from the Oplog feature?
Is there any way to monitor the usage of Oplog from the Meteor side?

Some help or pointing in the right direction would be greatly appreciated.

1 Like
2

Hello,

I have the same setup as you ( Atlas MongoDB cluster ) with three databases.
I have the same OPLOG URL connection for all thoses databases and monti APM is telling me all those databases are using the OPLOG.

So maybe this is a setup in your configuration ?

I will be happy to know if more people are doing this also

Cheers

1 Like
3

Using Monti APM is a great way to see accurate statistical analysis

image

4

Thanks a lot for your answer Victor!

Well, I am enabling the Oplog for two of my apps.

But just thinking, if it really is best practice and maybe you can jump in or have some more technical details on that.

oplog
oplog1760×1420 392 KB

Imagine there is a big app with like 1000 operations per second and a tiny app with around 1 operation per second.

  1. Isn’t the big app massively spamming / cluttering the Oplog?
  2. Is there still a benefit for the tiny app to use the Oplog feature?
  3. all the entries from the Oplog are transferred to every client that has the Oplog feature enabled → high bandwidth demand?

By the way thanks for mentioning the monti apm. Didn’t know about it. But for the moment we gonna stick to the Meteor APM that comes with the Meteor Cloud Pro subscription.

Cheers :wink:

5

  1. Isn’t the big app massively spamming / cluttering the Oplog?
    Yes indeed but if it is a solid cluster there is no problem.

  2. Is there still a benefit for the tiny app to use the Oplog feature?
    It does depends on the publications you have and not really on the number of connections I guess.

  3. all the entries from the Oplog are transferred to every client that has the Oplog feature enabled → high bandwidth demand?
    Yes but again your cluster should be sufficient.

I do not know what your applications does but redis-oplog might be better for you you just have a shared redis instance for all the servers ?