[critical] The Mongo server and the Meteor query disagree

help
1

I’m having a problem with meteor saying:

Error: The Mongo server and the Meteor query disagree on how many documents match your query. Maybe it is hitting a Mongo edge case? The query is: REDACTED - SEE BELOW
    at packages/mongo/oplog_observe_driver.js:881:15
    at Object.Meteor._noYieldsAllowed (packages/meteor.js:848:12)
    at OplogObserveDriver._publishNewResults (packages/mongo/oplog_observe_driver.js:851:12)
    at OplogObserveDriver.proto._publishNewResults (packages/znewsham_kadira.js:3207:38)
    at OplogObserveDriver._runQuery (packages/mongo/oplog_observe_driver.js:758:10)
    at OplogObserveDriver._runInitialQuery (packages/mongo/oplog_observe_driver.js:658:10)
    at packages/mongo/oplog_observe_driver.js:191:10
    at packages/mongo/oplog_observe_driver.js:15:9
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
    at packages/znewsham:justplay-common/imports/api/defer.js:17:25
    at runWithEnvironment (packages/meteor.js:1356:24)

Below is the query I think is causing the problem, I have some data in the DB with a null name - I thought this was what was causing the issue, as the queries stopped failing after I removed the data. However, I then tried adding back an item with a null name and it worked just fine.

  1. What could cause meteor to diasgree like this?
  2. It seems that meteor throwing an error here causes the entire server to get into a bad state. Some publications work, but many things fail. When this happened on our production environment just now, all our autoforms stopped responding (the insert/update worked, but the onSuccess call was never received). Only a server restart fixed the issue. Are there any protections against this happening again?
Collection.find(
{ teamId: "string", unit: "string", classification: "string", version: 0 },
{ sort: { name: 1 }, fields: { name: 1 } }
)
2

We had similar problems and this error really screws up your server.

I’m not sure at which point it’s removed but any properties in the selector which are undefined will be removed and cause this issue. We check all of our props in every selector of every publication to be certain. If your data is null in the DB maybe it gets converted into undefined when fetched? null should work AFAIK but I’m not certain.

3

I just found the same thing - turns out the names are undefined, but searching for {name: null} in mongo shell finds them, so I assumed they were null - but the problem is exactly what you said, we search for an undefined name as part of an aggregation type process, thanks for your help!

2 Likes
4

This behaviour was added in 1.6.1:

Meteor’s Node Mongo driver is now configured with the ignoreUndefined connection option set to true , to make sure fields with undefined values are not first converted to null , when inserted/updated. undefined values are now removed from all Mongo queries and insert/update documents…

1 Like
5

Thanks! Does this also include find/findOne? I suspect thats where the problem is for us rather than insert

6

The behaviour with find is described further on in that link (I just snipped part of it).

7

Is there any way of disabling this functionality - I’m familiar with passing in a custom driver to individual collections, but would prefer not to do that to all collections in our system - we had this problem hit on a different query today that will be much harder to track down. Similarly, do you know if it would be possible to at least modify the behaviour of the oplog observe driver so it matches the functionality of mongo (i.e., remove undefined values slightly earlier)

8

The code is very explicit (it’s not a parameter):

You could try cloning the package and changing that option, which should revert the behaviour.

9

Whats really odd is this seems to be coming from meteor core - we’re querying the user collection for an { _id: "userId", somethingElse: undefined}, I can’t see anywhere in our code we do that, but I DO see it in accounts_server.js:

var observe = self.users.find({
        _id: userId,
        'services.resume.loginTokens.hashedToken': newToken
      }, { fields: { _id: 1 } }).observeChanges({

The code is wrapped with an if(newToken) so I don’t think this can be it either, I’m wondering if there is an edgecase here where newToken can be undefined but pass that test - or if somewhere else in meteor something similar happens without the check.

This code would make sense as it is triggered when our admin users spoof in as someone, when already logged in - so its a limited problem for sure. We log all our login calls, and have never received one with both the userId set and a resume token NOT set

10

Ok, this wasnt it - it was our code but in a way that wasn’t obvious - it was triggered by a race condition on the client. Previously the query would have returned nothing and now it crashes - the solution (exactly as the documentation suggests) is to go through all our queries and ensure we don’t allow undefined values anywhere.

1 Like