Strange phenomenon: Subsequent methods or subs won't succeed

[waldgeist]

@mozibrand

in the end there would be more serious side effects

Do you still remember which side-effects you were facing? For now, I noticed that documents are deleted and re-added unnecessarily. But it made me suspicious that there might be some other traps I don’t know about yet.

[afrokick]

@waldgeist

Well, I think it’s the beauty of pub/sub that you only get the deltas you actually need, if you have a large data set to display.

You receive delta on updates, or when reuse data from other subscriptions. With zoom in/out you’ll receive new data? right?

@waldgeist

How should this work efficiently with methods?

By using event-driven without Mongo oplog, we’r using redis pub/sub.

I can’t solve you issue, but if you want to use efficient way for large data set, for me, its not about pub/sub.

[waldgeist]

@afrokick

You receive delta on updates, or when reuse data from other subscriptions.

The problem here was: this is only true if you don’t unblock them. Which was a recommendation to solve the original problem mentioned here (methods and pubs hanging up after one user interaction, but only after a couple of weeks).

[waldgeist]

@afrokick

By using event-driven without Mongo oplog, we’r using redis pub/sub.

Do you happen to know if Redis Oplog supports geo-spatial queries? I wasn’t aware that it is not supported by the normal oplog mechanism.

[afrokick]

@waldgeist

Do you happen to know if Redis Oplog supports geo-spatial queries? I wasn’t aware that it is not supported by the normal oplog mechanism.

We’r not using redis oplog, because we use transactions with custom MongoDB methods, which doesn’t work with redis oplog package.

Instead, we use more raw and efficient way - redis pub/sub(Vent).

For example, you can dispatch any updates for points via Vent:

Points.update({_id:pointId}, { $set:{ title:newTitle } });
//publish event to all listeners of pointId
Vent.emit(`points:${pointId}:updateTitle`, { title: newTitle });
//publish event to all listeners of all points
Vent.emit(`points:updateTitle`, { title: newTitle });

Also, you can split points by the region (US,EU) or the country (UK,DE,BY,RU) or smallest part like area.

Vent.emit(`points:${point.region}:updateTitle`, { title: newTitle });
Vent.emit(`points:${point.country}:updateTitle`, { title: newTitle });
Vent.emit(`points:${lat+lon+radius or smth else}:updateTitle`, { title: newTitle });

When you change viewport, just load new data and subscribe to channels you need and mutate your cache on new events from channels.

[peterfkruger]

Could you please point us to which package this Vent stuff comes from? Or is Vent a custom MongoDB collection? I am unable to connect the dots based on your example, but I’m very keen to learn more about your implementation of redis-based pubsub as an alternative to regular Meteor publications.

[afrokick]

@peterfkruger

Could you please point us to which package this Vent stuff comes from?

RedisOplog’s Vent redis-oplog/vent.md at master · cult-of-coders/redis-oplog · GitHub

[afrokick]

If anyone interesting in this topic, i can do a live to talk about how we using it for a large project where we also using pubs.

[peterfkruger]

That’s great, thank you. I’m sure that for real-time or near real-time applications, especially with a large number of users Meteor pubsub isn’t the proper tool. Redis is superfast, so using it as a message backbone to build reactivity is probably the way to go. I’ll have a go at it.

[peterfkruger]

I’m definitely interested, count me in.

[afrokick]

lets move live idea info Slack #lives cannel )

[waldgeist]

@afrokick

If anyone interesting in this topic, i can do a live to talk

Definitely, count me in.

[waldgeist]

That looks interesting. Will have a deeper look at it!

[waldgeist]

Seems like we have another case that looks very similar to the original phenomenon mentioned in this thread:

Meteor loggingIn stuck on true even though user is logged in

Hi everyone, seeing this weird issue on a new Galaxy deployment of a working codebase. Meteor.loggingIn() is stuck on true, but Accounts.onLogin callback is executing, Meteor.user() is defined. Meteor.logout() has no effect, user remains even after reloading the page. Meteor.status() says connected I looked at the raw messages in the websocket and everything looks normal: Client calls connect, calls login with resume token, and subs to loginServiceConfiguration and autoupdate client versions …

[mozibrand]

Hi, out in the end, have you worked out the problem, I want to know any reason, as long as it is able to solve, this is the original main cause I left the pub/sub, and I now use the Apollo of non real-time response to data docking, if you have experience, give me a more detailed talk about, I might try again using the pub/sub

[waldgeist]

This problem is not directly related to pub/sub, it also affects methods.

[mozibrand]

So how did you solve it in the end, I want to learn

[waldgeist]

I don’t have a solution yet. I added this.unblock() to all methods (except those that delete data), and now I am waiting if this helped or not. Which is hard to tell, because the problem only appeared after weeks of normal operation.

[mozibrand]

I look forward to the message again.

[drone1]

Any updates on resolving this?

I’m encountering a similar issue with Galaxy where I make a simple Meteor.call() and the callback is never triggered, despite confirmation that the result is reaching the client (I can see it in Meteor DevTools Evolved DDP tab). It happens after the server’s been up for a few days or so.

Never happens on dev.

I’m going to deploy my app now with a unhandledRejection handler to see if I catch anything funky.

Would love any tips if anyone’s learned anything since Feb 2021

I’m running Meteor 2.14, with latest packages and npm packages. I’m using Atlas for DB with an oplog.