There’s no inherent or architectural reason this issue occurs. Based on the reading of the issues here that @copleykj helpfully found, there could be regressions in the way
node/fibers and newer versions of
node interact with each other. But that’s probably a red herring.
To answer your specific question @imagio, the number of documents is probably a red herring too. What’s likely happening is between 2,000 and 3,000, you retrieve a user document that is:
Extremely large: It will appear to be at 100% CPU, because of the way the instrumentation works, but it’s actually transferring a multi-megabyte amount of data over a network connection that has surprisingly low bandwidth (on the order of 50 kB/s). Or, maybe you have a link to a forum image or something like that, which you “helpfully” download on the server, which coincidentally is a huge file. Or, you’re trying to do a “join”-like operation, and you’re accidentally retrieving way more data than you think you are.
Not valid: You use something like simple schema, or all these validators, which may accidentally cause an infinite loop trying to validate something. That stuff is really glitchy sometimes.
Paging won’t fix this. Try downloading your production database (using
mongorestore, perhaps using Mongo’s Compass application to set up the SSH tunnel to your protected DB for you) and inspecting it.
From an application architecture point of view, if your client’s need all the user documents (which naturally they don’t), create a single document in your database containing everything about all the users a client needs. If you’re reaction is, “That would be a huge document!”, well, then your original architecture going to be super slow anyway Even if you use paging! A simple rule of thumb is to have as close of a correspondence between a Mongo document and the things the web client needs to see on screen as possible. I’m otherwise not really able to comment on whether you’re using
fields to filter out stuff you don’t need or whatever.