Strange phenomenon: Subsequent methods or subs won't succeed

Do you still remember which side-effects you were facing? For now, I noticed that documents are deleted and re-added unnecessarily. But it made me suspicious that there might be some other traps I don’t know about yet.

You receive delta on updates, or when reuse data from other subscriptions. With zoom in/out you’ll receive new data? right?

By using event-driven without Mongo oplog, we’r using redis pub/sub.

I can’t solve you issue, but if you want to use efficient way for large data set, for me, its not about pub/sub.

The problem here was: this is only true if you don’t unblock them. Which was a recommendation to solve the original problem mentioned here (methods and pubs hanging up after one user interaction, but only after a couple of weeks).

Do you happen to know if Redis Oplog supports geo-spatial queries? I wasn’t aware that it is not supported by the normal oplog mechanism.

We’r not using redis oplog, because we use transactions with custom MongoDB methods, which doesn’t work with redis oplog package.

Instead, we use more raw and efficient way - redis pub/sub(Vent).

For example, you can dispatch any updates for points via Vent:

Points.update({_id:pointId}, { $set:{ title:newTitle } });
//publish event to all listeners of pointId
Vent.emit(`points:${pointId}:updateTitle`, { title: newTitle });
//publish event to all listeners of all points
Vent.emit(`points:updateTitle`, { title: newTitle });

Also, you can split points by the region (US,EU) or the country (UK,DE,BY,RU) or smallest part like area.

Vent.emit(`points:${point.region}:updateTitle`, { title: newTitle });
Vent.emit(`points:${point.country}:updateTitle`, { title: newTitle });
Vent.emit(`points:${lat+lon+radius or smth else}:updateTitle`, { title: newTitle });

When you change viewport, just load new data and subscribe to channels you need and mutate your cache on new events from channels.

1 Like

Could you please point us to which package this Vent stuff comes from? Or is Vent a custom MongoDB collection? I am unable to connect the dots based on your example, but I’m very keen to learn more about your implementation of redis-based pubsub as an alternative to regular Meteor publications.

RedisOplog’s Vent redis-oplog/vent.md at master · cult-of-coders/redis-oplog · GitHub

1 Like

If anyone interesting in this topic, i can do a live to talk about how we using it for a large project where we also using pubs.

3 Likes

That’s great, thank you. I’m sure that for real-time or near real-time applications, especially with a large number of users Meteor pubsub isn’t the proper tool. Redis is superfast, so using it as a message backbone to build reactivity is probably the way to go. I’ll have a go at it.

1 Like

I’m definitely interested, count me in.

1 Like

lets move live idea info Slack #lives cannel )

Definitely, count me in.

1 Like

That looks interesting. Will have a deeper look at it!

1 Like

Seems like we have another case that looks very similar to the original phenomenon mentioned in this thread:

Hi, out in the end, have you worked out the problem, I want to know any reason, as long as it is able to solve, this is the original main cause I left the pub/sub, and I now use the Apollo of non real-time response to data docking, if you have experience, give me a more detailed talk about, I might try again using the pub/sub

This problem is not directly related to pub/sub, it also affects methods.

So how did you solve it in the end, I want to learn :grin:

I don’t have a solution yet. I added this.unblock() to all methods (except those that delete data), and now I am waiting if this helped or not. Which is hard to tell, because the problem only appeared after weeks of normal operation.

I look forward to the message again.

Any updates on resolving this?

I’m encountering a similar issue with Galaxy where I make a simple Meteor.call() and the callback is never triggered, despite confirmation that the result is reaching the client (I can see it in Meteor DevTools Evolved DDP tab). It happens after the server’s been up for a few days or so.

Never happens on dev.

I’m going to deploy my app now with a unhandledRejection handler to see if I catch anything funky. :guitar:

Would love any tips if anyone’s learned anything since Feb 2021 :slight_smile:

I’m running Meteor 2.14, with latest packages and npm packages. I’m using Atlas for DB with an oplog.