Let's talk about Oplog Observe Performance


A quick google search and you’ll find a ton of threads about oplog observe driver’s performance. Most of it stems from the lack of an observe filter, and the fact that the oplog tails the entire DB with no selectivity.

I ran meteor create vanillaApp and imported mongo on the server, loaded up the mongo shell, and bulk inserted 10,000 documents with about 200kb of string data per document into the ‘posts’ collection. There was no activity (CPU) fluctuation on the server, which is what I expected.

Simply adding this line of code:

let Posts = new Mongo.Collection('posts')

and performing the experiment over again resulted in the server reaching 80%+ CPU for the duration of the insert. Keep in mind, I had no queries declared on the server and I had no clients connected. This CPU fluctuation seems awfully unnecessary.

Does the oplog observer continue to process callbacks even if there is not a single livequery declared? If so, why? Does anyone know of any progress in this area? Using {disableOplog:true} appeared to make little difference here. Is there something wrong in my setup? (Literally just a vanilla meteor app with that default counter example)