Hey,
I’m having an issue with one of our Meteor methods. If I call it, the response time takes about 20 seconds. The method fetches some data from the database and sorts it via underscore. The problem is, that the same action takes on my local dev machine about 3 seconds (connected to the same database). All CPU cores on the production server are running on 50% load, bandwidth usage is about 50MBit/s of 256MBit/s.
It only happens on prime time. PM2 shows a loop delay about 0.89ms. Any idea what could cause the difference between my productional server and the dev machine? I would understand it, if the prod server has 100% CPU load, but that’s not the case.
Could it be a problem with the MongoDB connection? The only difference I can see is that the production server does more requests to the database than my local machine.
Have you tried using something like mongotop or db.collection.find(yourQuery).explain() to see if the production database is having a holdup somewhere? If your local machine takes 3 seconds for a method call (which is a very long time…) I would guess you’re doing some heavy blocking computation in that method. I could see another 2 seconds getting added due to various latencies in the production env, and then another several seconds getting added if you have multiple clients trying to do the same heavy sorting.
Just tried to find slow queries, but the slowest was about 200ms. Both servers are using the same database, so I don’t think it is a query issue, since the same query is running well on the local machine.
Query time also depends on the mongodb load, though. Your load in your local environment (I’m guessing) is very low (unless you’re doing a simulation or something).
How many documents are you sorting on? I’m curious as to why your method takes 3 seconds! Are you waiting in your method on an async function (besides the query)?
I would also highly recommend this compound index, instead: {url:1, server:1, createdAt:1 /*or -1, but it shouldn't matter*/}, otherwise mongodb will have to spin up a sort generator, generate a sort key, and sort for every single query (which involves bring your 1m docs into memory)… Indexing the sorted field should vastly improve performance, but remember to put that index last.
EDIT: Oops! If you are requesting {server: {$in: availServers}}, you should actually put this index last, so your index will look like {url:1, createdAt:1, server:1}. In general the sorted field should go last in an index unless you are querying the other indexed fields using something other than $eq (i.e. if you’re using $lt, $in, $ne, etc.), in which case those should go after your sorted field. The reason for this is that the n+1th index in a compound index is sorted separately for each key in the nth index in the compound index.
Whoops, sorry, I’ve posted the wrong table. We query against app.data and this one had a read time about 200ms. But I’ve created new indexes, I’ll check if it solves the problem. Thank you very much for your help.
Do Meteor methods block each other by default? If I have one hundred users all call the same Meteor Method at the exact same time would those hundred Method calls run sequentially by default without the use of this.unblock()?
So if the method takes 200ms to complete per user, would I expect 20 seconds to go by before all users’ method calls were complete - again by default and not using this.unblock()?