Updating Redis Oplog Outside of Meteor

chiefchef · January 15, 2020, 10:57pm

We have seen 100% CPU spikes in our app and would like to implement the Redis Oplog solution. I was able to install the package and configure it with little issue. We’ve signed up for a small account with RedisLabs to test. I am noticing that the CPU is no longer spiking, however, when we are making changes to MongoDB from outside of Meteor (C# or Studio3T) we are not seeing the changes automatically update on the screen as they would previously with MongoDB Oplog.

If I’m updating MongoDB from outside of Meteor how do I notify Redis about the changes? Has anyone dealt with this? There must be something that I’m missing…

copleykj · January 16, 2020, 1:12am

I think you will probably want to give this a read…

github.com

cult-of-coders/redis-oplog/blob/master/docs/outside_mutations.md

## Outside Mutations

There may be scenarios where you have an external service that is not Meteor, which
performs mutations to MongoDB. If you want your subscriptions to "catch" the changes you have to send 
manually the changes to redis channels.

However, this is pretty straight forward.

Given I have the following publication:
```js
return Tasks.find({groupId});
```

If an outside worker updates the MongoDB, it should also send changes to redis like this:

```js
// update task document, that changes status inside MongoDB
redis.publish('tasks', JSON.stringify({
    e: 'u',
    d: {_id: taskId},

This file has been truncated. show original

drew · January 16, 2020, 2:07am

We built a simple form for admins that lists collections: Object.keys(Meteor.connection._stores);

And a few text fields to specify event / field changes.

znewsham · January 16, 2020, 2:24am

I’ve been working on an NPM package - it’s not quite production ready (we’re using it in production in a limited sense, but I wouldn’t want to guarantee that it will work).

It can be used to either push updates to redis and/or provide synchronous DB access. https://bitbucket.org/znewsham/mongo-redis-future/src/master/

chiefchef · January 16, 2020, 10:46pm

@copleykj I think this sounds like what I need to do. I will have to play around with the code to see if I can figure it out. I’m currently making many changes to the MongoDB with C# and these instructions don’t seem very helpful for that. I am also curious how I would update Redis through the Mongo Shell as we do many operations just directly on the database. It seems like they are expecting you to make this redis.publish call for every document, I’m not sure how that works if I were to update an entire collection for instance…

It seems like this article is also describing the same thing that I want done:

However, when I follow the instructions and set the properties in the meteor config file I get the same behavior…