That console log value of _where parameter is working perfectly on Studio 3T that we are using to manage the DB. It returns same as the .count() method result in my codes. So the issue is on the .fetch() method of meteor, sometimes it can read the data correctly and sometimes not. We have more than 20K documents in a collection we tried to test or under this collection Jobs_status_identifier. We still encounter this issue until now in some other parts of our system.
I’m sure it is. If you copy and paste into a query tool, including the MongoDB shell, or Studio 3T, the conversion from string to object is managed for you.
However, if for example, you use a form to collect fields, which you then assemble into a query, that will result in a string which looks like an object, but isn’t. I’m not saying you’re doing that - it’s an example of how this might occur.
Change the top of your method to this:
_get_per_colour_jobs : function(_where, _clr){
console.log(typeof _where);
...
and report back.
Less likely alternative answers are:
- MongoDB is broken.
- This happens to everybody, but no-one’s noticed.
- You’ve dropped the unique constraint or entire index on
_id.
hi @robfallows , … the console log results is ‘object’. I put it back to pub/sub method since the result is the same. Now, here’s the _where value I have on subcription.
{"current_status":"In Progress","in_progress":{"$gte":100},"complete_details.address.state":{"$in":["NSW","QLD","SA","TAS","VIC","WA","NT","ACT"]},"complete_details.claim.type":{"$in":["Promise Claim","Other","Maintenance","Insurance"]}}
The data in the DB is 53 but I got only 44 results when executing .fetch() function.
Additional details, on how I execute it.
_subsc_Jobs_status_identifier = Meteor.subscribe('Jobs_status_identifier', _where, {sort : {_number: 1}}, function(){
let _get_lists = Jobs_status_identifier.find(_where, {sort : {_number: 1}}).fetch();
Session.set('job_get_lists', _get_lists);
});
Thanks
A couple of things to try:
-
Look at the data from
findandfetch- what’s missing? Can you see a pattern, or anything odd? -
Go back to methods and (in the method) use the
rawCollectionfunctions, which bypass minimongo. Note that the underlying MongoDB library hastoArrayinstead offetch:Meteor.methods({ async _get_per_colour_jobs(_where, _clr) { _get_lists = await Jobs_status_identifier.rawCollection().find(_where).toArray(); return _get_lists; } });Note also, the use of
asyncandawait. I’ve also removed_clr, as it’s empty and not a valid parameter in the MongoDBfind(Meteor’sfindoptions are done differently). If you want to research this, check the documentation forfindand cursor methods.
I think this has already been confirmed as a bug:
hi @robfallows, as I’ve explained in previous reply I’ve moved it back to pub/sub method of getting the data from database. Maybe it can elaborate this issue by posting my publication (see below).
Meteor.publish("Jobs_status_identifier", function (params, _clr) {
if(params){
_get_resp = Jobs_status_identifier.find(params, ((_clr) ? _clr : {}));
return _get_resp;
}
});
Was just using the server side to test if the query is working or not and since it’s working the same as my pub/sub method then I decided to put it back as before for some reason, we need it to be reactive.
You mention that _clr is not valid to place in the .find()? The _clr here is flexible sometimes I put order by and limit params and sometimes not, that’s why I put a condition to convert it to {} if not defined. Can you explain why should I have to remove it there? 
Anyway, I still use the same construction of _where to this pub/sub method and unfortunately the issue still exist, not in all queries but in some of my queries like I’ve posted above previously.
Thanks!
@robfallows, maybe is convered in the issue posted by @storyteller ? Hopefully it can be fixed soon.
Only if you want to test with rawCollection().
None of your examples or information have said this. In fact, you have explicitly stated that _clr = {}.
If you are not prepared to help by stating exactly what you’ve done, then I’m not prepared to spend any more time on this.
hi @robfallows, sorry if I missed something you requested. That server method is not being used already like what i’ve said because we want it reactive. Here’s what I did on how I execute the query.
first I constructed this _where params, on server side
_where = { ‘complete_details.address.state’: { ‘$in’: [ ‘NSW’, ‘QLD’, ‘SA’, ‘TAS’, ‘VIC’, ‘WA’, ‘NT’, ‘ACT’ ] }, ‘complete_details.claim.type’: { ‘$in’: [ ‘Promise Claim’, ‘Other’, ‘Maintenance’, ‘Insurance’ ] }, current_status: ‘In Progress’, in_progress: { ‘$gte’: 100 } }
then I run these two, this time on server side @ main.js
console.log(Jobs_status_identifier.find(_where).count());
_get_lists = Jobs_status_identifier.find(_where).fetch(); console.log(_get_lists.length);
and the result still different. Don’t be confused with _clr I already remove that.
Another interesting query that give me only more than half of the real results. The total data is 998 but the .fetch() method only give me 558. This is funny, I created only a simple query (on server side) which is below:
let params = {"$and":[{"$or":[{"waiting_for_appointment":{"$gte":100}},{"waiting_for_scope":{"$gte":100}},{"waiting_for_accepted_scope":{"$gte":100}},{"waiting_for_go_ahead":{"$gte":100}},{"waiting_for_confirmation":{"$gte":100}},{"in_progress":{"$gte":100}},{"ready_to_be_invoiced":{"$gte":100}},{"to_be_paid":{"$gte":100}}]}],"current_status":{"$in":["Waiting For Appointment","Waiting For Scope","Waiting for Accepted Scope","Waiting For Go Ahead","Waiting For Confirmation","In Progress","Ready to be Invoiced","To Be Paid"]},"complete_details.address.state":{"$in":["NSW","QLD","SA","TAS","VIC","WA","NT","ACT"]},"complete_details.claim.type":{"$in":["Promise Claim","Other","Maintenance","Insurance"]}};
And run/execute
_get_resp = Jobs_status_identifier.find(params);
console.log(_get_resp.count());
console.log(_get_resp.fetch().length);
And the console log results
I20171129-03:15:03.083(0)? 998
I20171129-03:15:06.377(0)? 558
Any thoughts on this?
If you run this on the server, what do you see?
console.log(Jobs_status_identifier.find({}).count());
console.log(Jobs_status_identifier.find({}).fetch().length);
Although, as @robfallows suggested, I think the actual differences between the datasets is the key to figuring this out, not examining the counts.
running this
let _get_resp = Jobs_status_identifier.find({});
console.log(_get_resp.count());
console.log(_get_resp.fetch().length);
got this
I20171129-06:01:38.405(0)? 1421
I20171129-06:01:47.855(0)? 865
Just to confirm, Jobs_status_identifier is the collection, right?
yeah, that’s why there’s a number of data returns 
Is "complete_details.address.state" an array? Maybe that part of the query is causing your results to vary. I wonder what your results will be if you add the appropriate fields filter? https://docs.meteor.com/api/collections.html#fieldspecifiers
That’s only a string. I don’t think that’s the issue, as you see the result was being pulled correctly, and how to prove that? Using the .count() method and for sure it’ll works also using the rawCollection. The issue here is on how the .fetch() function handle the result from the database. I’m guessing it re-reads the where statement in that function and unfortunately it interpreted it in a wrong way…
Or maybe wasn’t able to wait the data from mongodb? We have more than 20k data in db so maybe that’s the issue?
Is it an object or a string in your schema? I don’t think a string should have dot notation in a query?? I suggested the fields filter because I’ve noticed its helped to make some query results more accurate when it comes to arrays and probably objects as well.
Don’t worry about that, I’ve posted a version above without where statement, and still the same result.
Any thoughts on this?