Meteor: Why is Oplog so slow?

Hey,
we are running multiple Meteor instances for a live chat application. Since 2 days I’ve noticed some very high CPU usage on our servers, even the pub/sub response times have increased (from about 20ms up to 200ms). After checking the Kadira logs, I’ve found this:

Fetching oplog.rs takes up to 2600ms. This is a very high value, so I’m wondering if there are any ways to improve it? Or should we go back to Meteors polling solution and set a small pollingInterval value (f.e. 1 second)?

We have the same issue when running the latest version of npm-mongo (1.5.48 or newer). I think there is an issue with the latest version of the node mongo driver. We’ve reverted to 1.5.47 for now.

1 Like

So you are using Meteors mongo@1.1.11 instead of mongo@1.1.12?

Some other Kadira graphic that irritates me is this one:

I could swear that it was full of oplog notifications just 2-3 weeks ago. We didn’t change any of the pubs or subs.

Yes, we are using mongo@1.1.1.

1 Like

Thanks, just downgraded to 1.1.1 - for the moment it seems like it solves the problem, I’ll keep my eye on it.

@benjamn Is this a known issue, or should we open a new issue on GitHub? Don’t know if we can make an easy reproduction.

Hi,
I’m trying to downgrade my mongo version from 1.1.12 to 1.1.11 by setting it in my .meteor/versions and .meteor/packages files. But everytime I start my app, it upgrade it back to the 1.1.12.

Thanks for help

Did you try it with this command?

meteor remove mongo
meteor add mongo@=1.1.11

No I didn’t, I do it now :slight_smile:

So it doesn’t work. I got a huge message error when I do the adding command :

MINISAT-out: Cannot enlarge memory arrays. Either (1) compile with -s TOTAL_MEMORY=X with X higher than the current value 67108864, (2) compile with ALLOW_MEMORY_GROWTH which adjusts the size at runtime but prevents some optimizations, or (3) set Module.TOTAL_MEMORY before the program runs.
MINISAT-err: Cannot enlarge memory arrays. Either (1) compile with -s TOTAL_MEMORY=X with X higher than the current value 67108864, (2) compile with ALLOW_MEMORY_GROWTH which adjusts the size at runtime but prevents some optimizations, or (3) set Module.TOTAL_MEMORY before the program runs.

/Users/robin/.meteor/packages/meteor-tool/.1.4.1_1.1l0s2pc++os.osx.x86_64+web.browser+web.cordova/mt-os.osx.x86_64/isopackets/ddp/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:165
      throw error;
      ^
abort() at Error
    at jsStackTrace (packages/logic-solver/minisat.js:3:1)
    at stackTrace (packages/logic-solver/minisat.js:3:1)
    at abort (packages/logic-solver/minisat.js:33:1)
    at enlargeMemory (packages/logic-solver/minisat.js:3:1)
    at Function.Runtime.dynamicAlloc [as alloc] (packages/logic-solver/minisat.js:3:1)
    at _sbrk (packages/logic-solver/minisat.js:3:1)
    at Sd (packages/logic-solver/minisat.js:7:1)
    at Ud (packages/logic-solver/minisat.js:7:1)
    at gc (packages/logic-solver/minisat.js:7:1)
    at pc (packages/logic-solver/minisat.js:7:1)
    at Kb (packages/logic-solver/minisat.js:7:1)
    at Dc (packages/logic-solver/minisat.js:7:1)
    at Object.dd [as _addClause] (packages/logic-solver/minisat.js:7:1)
    at packages/logic-solver/minisat_wrapper.js:74:1
    at Object._native.savingStack (packages/logic-solver/minisat_wrapper.js:37:1)
    at [object Object].MiniSat.addClause (packages/logic-solver/minisat_wrapper.js:68:1)
    at [object Object].Logic.Solver.solve (packages/logic-solver/logic.js:1326:1)
    at [object Object].Logic.Solver.solveAssuming (packages/logic-solver/logic.js:1369:1)
    at minMaxWS (packages/logic-solver/optimize.js:52:1)
    at [object Object].Logic.Solver.minimizeWeightedSum (packages/logic-solver/optimize.js:139:1)
    at packages/constraint-solver/solver.js:368:1
    at Function.time (/tools/tool-env/profile.js:305:10)
    at [object Object].CS.Solver.minimize (packages/constraint-solver/solver.js:352:1)
    at [object Object].CS.Solver.minimize (packages/constraint-solver/solver.js:346:1)
    at [object Object].CS.Solver._getAnswer (packages/constraint-solver/solver.js:735:1)
    at packages/constraint-solver/solver.js:603:1
    at Function.time (/tools/tool-env/profile.js:305:10)
    at [object Object].CS.Solver.getAnswer (packages/constraint-solver/solver.js:602:1)
    at packages/constraint-solver/constraint-solver.js:165:1
    at Object.Logic.disablingAssertions (packages/logic-solver/logic.js:45:1)
    at Function.CS.PackagesResolver._resolveWithInput (packages/constraint-solver/constraint-solver.js:164:1)
    at [object Object].CS.PackagesResolver.resolve (packages/constraint-solver/constraint-solver.js:128:1)
    at /tools/project-context.js:518:35
    at Function.time (/tools/tool-env/profile.js:305:10)
    at /tools/project-context.js:514:19
    at /tools/utils/buildmessage.js:359:18
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:352:34
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:350:23
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at Object.enterJob (/tools/utils/buildmessage.js:324:26)
    at /tools/project-context.js:486:20
    at /tools/packaging/catalog/catalog.js:100:5
    at /tools/utils/buildmessage.js:271:13
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:264:29
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:262:18
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:253:23
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at Object.capture (/tools/utils/buildmessage.js:252:19)
    at Object.catalog.runAndRetryWithRefreshIfHelpful (/tools/packaging/catalog/catalog.js:99:31)
    at ProjectContext._resolveConstraints (/tools/project-context.js:485:13)
    at /tools/project-context.js:268:9
    at /tools/utils/buildmessage.js:359:18
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:352:34
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:350:23
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at Object.enterJob (/tools/utils/buildmessage.js:324:26)
    at ProjectContext._completeStagesThrough (/tools/project-context.js:258:18)
    at /tools/project-context.js:250:12
    at Function.run (/tools/tool-env/profile.js:489:12)
    at ProjectContext.prepareProjectForBuild (/tools/project-context.js:249:13)
    at /tools/cli/commands-packages.js:2034:20
    at /tools/utils/buildmessage.js:271:13
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:264:29
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:262:18
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at /tools/utils/buildmessage.js:253:23
    at [object Object].withValue (/tools/utils/fiber-helpers.js:89:14)
    at Object.capture (/tools/utils/buildmessage.js:252:19)
    at Command.func (/tools/cli/commands-packages.js:2033:27)
    at /tools/cli/main.js:1410:23
If this abort() is unexpected, build with -s ASSERTIONS=1 which can give more information.

Did you try it with the equal? In the first version I forgot this.

Yes yes I tried exactly that :confused:

Mhh okay, I’m also on 1.4 and Mac, everything here works fine. Maybe you can try a reboot :slightly_frowning_face:

haha ok. I gonna try :slight_smile:
Thx. I come back to you soon :slight_smile:

So I just rebooted, and I still have the same error when doing the meteor add mongo@=1.1.11 :frowning2:

Mhh okay, but going back to

meter add mongo

works without problems?

yes I just tried and works fine

Mh okay, my last idea would be to remove all caches/build in your .meteor folder, f.e.

.meteor/build
.meteor/plugin-cache
.meteor/bundler-cache

and then run the remove/add command again.

1 Like

still the same :frowning2:

Nevermind I will try to find how to fix that. Thanks a lot for your help.