Polling for external Mongo changes

help
1

I’ve been pointing my meteor instances at an existing mongod, i.e.:

$ MONGO_URL=mongodb://localhost:27017/some_db meteor

This mongod is written to by processes external to meteor, and I want meteor to pick up those changes and update the page instantly.

I’ve found that meteor is only picking up updates every 10 seconds, evoking several old threads about pre-1.0 meteor behavior.

Debugging this, I noticed that when I let meteor start its own mongod, and have my external processes connect to and modify that, meteor does update things instantaneously (which is awesome).

I’m guessing that meteor sets up some oplog-tailing that I am missing with my external mongod.

I think I can get by for now having meteor manage the mongod in this way, but wanted to document this since I couldn’t find any since-1.0 info about what behavior was expected, and at first I thought that the 10s delay was the best I was going to get.

If anyone wants to chime in about how to glue an external mongod to meteor in a way that makes this work, that would also be cool. Thanks!

2

The mondodb instance you are connecting to should have oplog enabled.

The way to do that is initiate a replica set on that mongodb instance and set the oplog url

$ export MONGO_URL mongodb://localhost:27017/some_db
$ export MONGO_OPLOG_URL mongodb://localhost:27017/local
$ meteor
1 Like
3

quick observation: I’m also connected to a separate mongo instance, on another server, and I see my updates happen anywhere from within 2 seconds to 8 seconds (when I make the update directly on mongo)… however, if I make my update on another instance of the meteor application, the updates are instantly reflected in the browser…

4

@wesyah234 it sounds like you’re seeing the same issue I originally posted about; your changes are only reflected when Meteor does a poll every 10s.

@serkandurusoy’s instructions worked for me:

  • start your external mongod with a replica set: mongod <options> --replSet=rs0
  • connect to it and initiate the replica set: > rs.initiate() (verify with rs.conf())
  • run meteor with MONGO_OPLOG_URL=mongodb://<host>:<port>/local (db name has to be local for the oplog URL; set MONGO_URL the way you normally would).

Then you will see external mongo updates reflected immediately in your app.

5

Thanks… definitely worth a try, however, I now have a question:
What are the performance implications of doing it the way it is currently, vs doing the replica set setup?

Is the 10 second poll taking more resources overall, thus it’s always recommended to set up the replica set?

Or, is the 10 second poll taking less resources overall, thus, if you’re ok with an up to 10 second delay in updates, you should just leave it as is.

Also, as I noted in my reply, when I work with 2 instances of my application, hitting a remote mongo instance, the updates are instantaneous… so how does that happen? How does my app know the difference between an update made to mongo directly in mongo (which takes 10 seconds) vs an update made to mongo through another web client (which is instant)?.. that’s puzzling.