CPU/Memory high when shared MongoDB bulk data inserted(not pub/sub collection)

performance
1

Condition

There are customer server and admin server.

The two server shares same mongodb and same database,
but the projects are separating Meteor project.

When the admin server insert a bulk data (over 70million) into ‘A collection’,
the customer server’s cpu/memory runs extremely high during the admin server inserting the data.

So I could assume that the customer server fetching the inserted oplog data by admin server.

Question

The point that I don’t understand is,
why the customer server is forking or syncing the oplog data,
even there’s no ‘A collection’ declair in customer server.
I expected that the customer server never fetching anything when the admin server insert a data, because ‘It is not decliared in the customer server’.

So I want to know Meteor project is syncing oplog even the project has not declaired the collection on the same database.
Or is there other reason for this?

2

If you’re still using the regular mongo oplog, and both servers are connected to the same database (not just the same cluster, the same database) then both servers will observe all changes made to any collection, regardless of whether there is a subscription. This is a known limitation of the regular mongo oplog package and one of the reasons most people have opted to use the redid oplog package.

An alternative would be to store these high volume data in a separate database.

1 Like
3

Okay. I didn’t know this known limitation :wink: So I adapt some separate db in local, but redis-oplog is good to go for chat reactive and control oplog status. I’ll try, thanks.