Redis Oplog issue in Meteor 3.1

help
1

Hi all,

just played around with Meteor 3.1 a little bit and noticed a problem during server boot up. Here’s my log:

error on boot.js TypeError: self._mongoHandle._createSynchronousCursor is not a function
    at new PollingObserveDriver (packages/cultofcoders:redis-oplog/lib/mongo/PollingObserveDriver.js:67:47)
    at MongoConnection._observeChanges (packages/cultofcoders:redis-oplog/lib/mongo/observeChanges.js:113:21)
    at Cursor.observeChanges (packages/mongo/cursor.ts:75:24)
    at _getMultiplexer (packages/montiapm:meteorx/src/fiberless/mongo.js:35:32)
    at _getObserverDriver (packages/montiapm:meteorx/src/fiberless/mongo.js:42:29)
    at exposePollingDriver (packages/montiapm:meteorx/src/fiberless/mongo.js:57:24)
    at exposeMongoAsync (packages/montiapm:meteorx/src/fiberless/mongo.js:19:9)
    at initAsync (packages/montiapm:meteorx/src/server.js:37:3)
    at /tools/static-assets/server/boot.js:453:5
    at /tools/static-assets/server/boot.js:504:7
    at startServerProcess (/tools/static-assets/server/boot.js:501:3)
TypeError: self._mongoHandle._createSynchronousCursor is not a function
    at new PollingObserveDriver (packages/cultofcoders:redis-oplog/lib/mongo/PollingObserveDriver.js:67:47)
    at MongoConnection._observeChanges (packages/cultofcoders:redis-oplog/lib/mongo/observeChanges.js:113:21)
    at Cursor.observeChanges (packages/mongo/cursor.ts:75:24)
    at _getMultiplexer (packages/montiapm:meteorx/src/fiberless/mongo.js:35:32)
    at _getObserverDriver (packages/montiapm:meteorx/src/fiberless/mongo.js:42:29)
    at exposePollingDriver (packages/montiapm:meteorx/src/fiberless/mongo.js:57:24)
    at exposeMongoAsync (packages/montiapm:meteorx/src/fiberless/mongo.js:19:9)
    at initAsync (packages/montiapm:meteorx/src/server.js:37:3)
    at /tools/static-assets/server/boot.js:453:5
    at /tools/static-assets/server/boot.js:504:7
    at startServerProcess (/tools/static-assets/server/boot.js:501:3)
=> Exited with code: 1
=> Your application is crashing. Waiting for file change.

Btw, are there any plans to make redis-oplog part of meteor core? It would definitely put my mind at ease…

2

Hey @bratelefant , the method has been renamed to represent the correct behavior and I forgot to make it backwards compatible, I will write a patch for redis oplog ASAP

image

And yes, there are plans to bring to the core or make it an optional built-in strategy, however that’s more long term

4 Likes
3

@leonardoventurini Wanted to check in on this since it’s blocking us upgrading to Meteor 3.1. It also seems like there are now multiple versions of redis-oplog for Meteor 3:

There is also a new PR that just came into the Meteor 2 branch: v2.3 by StorytellerCZ · Pull Request #412 · cult-of-coders/redis-oplog · GitHub

I’m wondering which is the recommended fork moving forward, since this is such an important package!

4

Hey @brianlukoff I have a fix for it but need to get it propagated to the original package, will setup a PR for it today and @storyteller will help me get the changes released.

I also noticed the mess it is right now, I was almost opting on going for my own fork, but it’s best we use the original package in Atmosphere.

You can review the changes here:

Need to create another PR pointing upstream.

5

Thanks! Would it be possible to get an official 3.0 release published and maybe also include the EJSON performance update here? It sounds like this may help with GC.

6

Yes, I will make sure I have the latest upstream changes

2 Likes
7

I adapted the changes:

Will proceed to create a PR against MCP now

8

There we go cc @storyteller

4 Likes
9

Things progressed, but there is a merge conflict when trying to merge to master:

10

I seems that cultofcoders:redis-oplog is still not fully compatible with Meteor 3.1.x

I just tried using it and shortly after starting my Meteor server I get this error (crashing it):

W20250325-15:23:14.693(0)? (STDERR) meteor://💻app/packages/cultofcoders_redis-oplog.js:456
W20250325-15:23:14.693(0)? (STDERR)           throw new Meteor.Error("We could not find the collection instance by name: \"".concat(this.collectionName, "\", the cursor description was: ").concat(JSON.stringify(cursorDescription)));
W20250325-15:23:14.693(0)? (STDERR)                 ^
W20250325-15:23:14.962(0)? (STDERR) errorClass [Error]: [We could not find the collection instance by name: "users", the cursor description was: {"collectionName":"users","selector":{"_id":"edWpRtBRnJSkHtENy","services.resume.loginTokens.hashedToken":"+xxxxxxx="},"options":{"transform":null,"projection":{"_id":1}}}]
W20250325-15:23:14.963(0)? (STDERR)     at new ObservableCollection (packages/cultofcoders:redis-oplog/lib/cache/ObservableCollection.js:39:13)
W20250325-15:23:14.963(0)? (STDERR)     at RedisOplogObserveDriver.init (packages/cultofcoders:redis-oplog/lib/mongo/RedisOplogObserveDriver.js:37:33)
W20250325-15:23:14.963(0)? (STDERR)     at MongoConnection.<anonymous> (packages/cultofcoders:redis-oplog/lib/mongo/observeChanges.js:125:25)
W20250325-15:23:14.963(0)? (STDERR)     at MongoConnection.mongoConnectionProto._observeChanges (packages/montiapm:agent/lib/hijack/wrap_observers.js:170:44)
W20250325-15:23:14.963(0)? (STDERR)     at Cursor.observeChanges (packages/mongo/cursor.ts:75:24)

It seems that this call:

this.collection = Mongo.Collection.__getCollectionByName(
       cursorDescription.collectionName
     );

Is unable to find the “users” collection (and only that collection, apparently).

This can be easily tested by launching the Meteor shell and querying it:

Mongo.Collection.__getCollectionByName("users")
11

Seems like a fix would be to use the Mongo.getCollection(...) function introduced in Meteor 3.0.2 instead of that one. I don’t use this package though.

12

I can confirm that using Mongo.getCollection() seems to indeed fix the problem. I’ve opened a PR with that change: Fix issue with Mongo.Collection.__getCollectionByName not resolving 'users' when used in Meteor 3.1 by dmolin · Pull Request #419 · cult-of-coders/redis-oplog · GitHub

2 Likes