I’ve chosen to move forward with deploying Meteor on Kubernetes, mainly for the uptime advantage across multiple pods + nodes + etc. However, I’ve recently stumbled on an issue.
At this point, it’s important to remember that while data changes will usually happen through the Meteor app itself, it’s also possible for external data changes to occur. These could be triggered by another instance of the same app (when scaling horizontally), a different app altogether, or even manual changes to the database through the Mongo shell.
So every 10 seconds (or whenever the server thinks that the Posts collection may have changed), the LRS will re-run the Posts.find() query against Mongo (with a corresponding hit to your Mongo server).
This is a big issue, as I want to deploy any number of pods. I’m currently running 3 nodes on Kubernetes, and a total of 3 pods. This issue is easily replicable when I scale out to a large number of pods (ex. 15).
One client (client A) connects to one pod, and another client (client B) connects to a different pod. However, the results made by client A are not immediately reflected on client B. This defeats the purpose of using Meteor and the oplog for me, as real-time is needed.
There could be a possible solution, to direct all clients in on area/physical location to the same pod, however I’m not entirely sure I can currently do that with Kubernetes. Any clues?
If you’re seeing a lag of up to 10 seconds in the situation you’ve described, make sure your mongo deployment is running as a replica set and that you’re starting your app instances with the MONGO_OPLOG_URL environment variable set correctly.
Real-time is definitely available with horizontal scaling.
@sashko@babrahams that’s the thing, it appears oplog tailing is all setup properly. How can I confirm the oplog is working correctly? I’m using kadira and it’s showing oplog notifications.
When I scale down to one pod (vs 3 or 15), everything syncs instantly. When I scale to something like 15 to be able to replicate the issue nearly every time, I receive the delay. It does appear that oplog isn’t working, however I’m showing oplog notifications through Kadira and when everyone is connected to one pod, everything syncs instantly.
FYI, I am running a MongoDB replica set, and MONGO_OPLOG_URL is set.