I have started using Kadira and I noticed that some of my queries are slow. And the main performance issue is fetch on oplog.rs
In my app I have collection called Projects, it contains all data related to user project (id, access, createdAt, members , tags , python_modules , filesystem , lastActibitiTime)
Every 5 seconds we update field lastActivityTime with current timestamp,
setLastActivityTime: function (projectId) {
check(projectId, String);
if (!this.isSimulation) {
Projects.update(projectId, {
$set: {
'lastActivityTime': new Date().getTime(),
'isIdle': false
}
});
}
},
but when I have checked Kadira it shows that this simple query takes over 1000ms.
You are performing a regex search, which can exhibit poor performance. At least yours is anchored to the start of field, which is good, but you need an index on ns if you haven’t already got one.
You also need an index on op.
There are often better ways to track time than updating a database document every few seconds. That continuous “noise” will adversely affect oplog tailing.
As a separate idea, you could look at redis-oplog which may allow you to avoid oplog tailing altogether.
Yes, I have checked and indeed I don’t have any indexes on oplog.rs. I didn’t realize that oplog doesn’t create any automatically.
I have to show for other project members when particular user had been seen recently but You make me start thinking how I could change it, thanks for the clue.
You’re correct. I’d missed that key part. You can’t add any secondary indexes to oplog.rs. It’s normally only the first query to oplog.rs that may be slow. Once it’s running and only retrieving updates it’s usually pretty fast. Do you see that delay continuously, or only when you start the server code?
