Higher DB CPU usage: from 1.6.1 to 1.9.1?

Hi,

Our DB is on Atlas it it’s been always kept between at 10-15% CPU usage. As soon as we have one node process running Meteor 1.9.1 that has users connected to it, the CPU averages at 35-40% with peaks at 60-80% (never seen before).

I have not found any ticket or post regarding this. Has anyone else experienced this ?

We’re hesitating between rolling back to 1.6.1.

Regards,

Burni

1 Like

As soon as we restarted one of our node processes, Mongo started to receive “aggregates” queries from Meteor on a collection we have never used aggregates.

So it seems the mongo driver is doing something that is translated into an aggregate ?
Since those are slow, well, the CPU spikes a lot.

Any idea ?

1 Like

That’s the aggregate

That looks like a count of the collection

Exactly! we’ve just found it.

The count was “ligher” in Meteor 2.6.1 . Now it seems to use the aggregate and it’s way more CPU/RAM intensive.

We had a one that was running every time a user would connect. That happened every couple of seconds and it was taking a toll on the DB.

We changed the code to something else and it seems to be fixed.

We’ll monitor for a day to make sure everything is on track!

Regards,

Burni

2 Likes

Same issue here, since we migrate from 2.7.3 to 2.9.1, we got a lots of slow collscan queries (same kind as burni13, queries using aggregate)
I rolled back to 2.7.3 four hours ago, and no more collscan

I think creating indexes will help

@burni13 I don’t see what kind of query on js side trigger the issue
Could you provide a concrete example of what you changed in you code please ?

it was db.Collection.find().count();

The goal of the query was only to count all documents in a (now) quite large collection.
That’s way slower than it used to be!

So we changed the code behaviour to look for something else.

Do you have such queries ?

Regards,

Burni

On what would you recommend doing an index on for a find().count() ?

Regards,

Burni

collection.find().count() has a similar performance as collection.find().fetch() wherein the db has to go through the entire cursor to count the results.

The ideal way to count is collection.countDocuments() https://www.mongodb.com/docs/manual/reference/method/db.collection.countDocuments/

1 Like

I note that countDocuments() is not yet implemented in minimongo.

It is since v2.9 (changelog). They are not in the docs, though… @grubba could you look into it?

2 Likes

if you just want to read the numbers of document in a collection then you should not scan all the documents.
You can try this: https://www.mongodb.com/docs/manual/reference/method/db.collection.estimatedDocumentCount/
or you can simply store the number of documents somewhere, you will need to update it everytime you insert or remove document(s).

@minhna thanks!

We’ll switch to that!

Regards,
Burni

Done! I’ve done in 2.11 branch Collections | Meteor API Docs will cherry picking it to devel

1 Like