How can I filter out duplicate and empty documents as well as order a MongoDB Collection?

1

Getting all the data in a MongoDB collection is as easy as it gets. You just use the Collection instance name and call find:

TimeAndSpace = new Mongo.Collection('timeAndSpace');

if (Meteor.isClient) {
	. . .
	Template.placesLived.helpers({
		places: function () {
			// this helper returns a cursor of all of the documents in the collection
			return TimeAndSpace.find();
		}
	});
}

But my Collection has some duplicate data and some empty records. How do I filter these out? Also I want to order by two fields. My Collection is structured and written to thus:

TimeAndSpace = new Mongo.Collection('timeAndSpace');

if (Meteor.isClient) {
    Template.addTimeSpaceForm.events({
        'submit form': function(event){
            event.preventDefault();
            var city = event.target.city.value;
            var state = event.target.state.value;
            var yearin = event.target.yearin.value;
            var yearout = event.target.yearout.value;

            Meteor.call('insertLocationData', city, state, yearin, yearout);
        }
    });
}

if (Meteor.isServer) {
  Meteor.startup(function () {
    // code to run on server at startup
  });
  
  Meteor.methods({
      'insertLocationData': function(city, state, yearin, yearout){
          TimeAndSpace.insert({
              ts_city: city,
              ts_state: state,
              ts_yearin: yearin,
              ts_yearout: yearout
          });
      }
  });
}

I want the documents to be ordered first by yearin, then by yearout (so that if there are multiple documents with the same yearin value (such as 1984), they would be returned in this order:

Helena		Montana		1984	1984
San Andreas	California	1984	1987

(not vice versa)

So how do I filter out empty and redundant documents, and order documents by specified fields?

I tried this:

return TimeAndSpace.find(
  {ts_city: {$exists: true, ne: ""}, ts_state: {$exists: true, ne: ""}},
  {sort: {ts_yearin: 1, ts_yearout: 1}}
);

…but it returns no data.

When I enter this in the console:

TimeAndSpace.find(
  {ts_city: {$exists: true, ne: ""}, ts_state: {$exists: true, ne: ""}},
  {sort: {ts_yearin: 1, ts_yearout: 1}}
);

I get:

Exception in template helper: Error: Inconsistent operator: {"$exists":true,"ne":""}
    at http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1269:15
    at Function._.each._.forEach (http://localhost:3000/packages/underscore.js?0a80a8623e1b40b5df5a05582f288ddd586eaa18:164:22)
    at isOperatorObject (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1263:5)
    at compileValueSelector (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1459:14)
    at http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1439:9
    at Function._.each._.forEach (http://localhost:3000/packages/underscore.js?0a80a8623e1b40b5df5a05582f288ddd586eaa18:164:22)
    at compileDocumentSelector (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1422:5)
    at _.extend._compileSelector (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1399:12)
    at new Minimongo.Matcher (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1342:27)
    at new LocalCollection.Cursor (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:144:20)
XHR finished loading: GET "http://localhost:3000/sockjs/info?cb=8zcc5akr4u".
return TimeAndSpace.find(
  {sort: {ts_yearin: 1, ts_yearout: 1}}
);
Uncaught SyntaxError: Illegal return statement
    at Object.InjectedScript._evaluateOn (<anonymous>:905:140)
    at Object.InjectedScript._evaluateAndWrap (<anonymous>:838:34)
    at Object.InjectedScript.evaluate (<anonymous>:694:21)
TimeAndSpace.find(
  {ts_city: {$exists: true, ne: ""}, ts_state: {$exists: true, ne: ""}},
  {sort: {ts_yearin: 1, ts_yearout: 1}}
);
Uncaught Error: Inconsistent operator: {"$exists":true,"ne":""}
    at http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1269:15
    at Function._.each._.forEach (http://localhost:3000/packages/underscore.js?0a80a8623e1b40b5df5a05582f288ddd586eaa18:164:22)
    at isOperatorObject (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1263:5)
    at compileValueSelector (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1459:14)
    at http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1439:9
    at Function._.each._.forEach (http://localhost:3000/packages/underscore.js?0a80a8623e1b40b5df5a05582f288ddd586eaa18:164:22)
    at compileDocumentSelector (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1422:5)
    at _.extend._compileSelector (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1399:12)
    at new Minimongo.Matcher (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:1342:27)
    at new LocalCollection.Cursor (http://localhost:3000/packages/minimongo.js?af9eb9d7447544ca9b839a3dcf7ed2da2209b56c:144:20)

So what’s wrong with the code?

2

Uncaught Error: Inconsistent operator: {"$exists":true,“ne”:""}

You’re missing the $ in $ne.

Reading up on $exists a bit more, I don’t think you need it. I think you only need {'$ne':''}

3

Excellent - just what the intern ordered! - Clay ShannonMark Twain Central