Other Exciting Meteor News (RedisOplog crucial upgrade)

Hello gentlemen,

Released redis-oplog 2.0.4

RedisOplog has received today an update that improves performance even further. Taking it to the next step into its evolution. It may sound weird but by offering to opt-out of race-condition protection, we can open the gateway to incredible speeds in sending updates to large scale clusters, without generating spikes in the database.

Collection.configureRedisOplog({
   protectAgainstRaceConditions: false,
});

The greatest impact will be seen in very large clusters, every live-change implied a database request on the receiving end. Therefore, database gets bombed. Another problem that this solves, is allowing to send events where the listeners read from a slave database (which may have not yet have the full data)

Race conditions happen in extreme cases (almost never to be honest), and if it happens in most cases it’s not a big deal.

There isn’t anything out there that makes reactivity so simple and so scale-able. It’s crazy how powerful is the Meteor code-foundation, without it being written so well, redis-oplog may have never existed, or we couldn’t have full BC with it.

Cheers!

27 Likes

Hey @diaconutheodor, thanks for all your hard work on these optimizations.
Could you explain a bit more what “when it happens it’s not a big deal” means? What are the possible downsides of turning this on. I believe this would be used for a very large production-scale application, where “extreme cases” are everyday problems :wink:

Thanks again!

Imagine you see number of likes on a Post, if you see 230 instead of 231. It may not be a big deal. You can always do some invalidation at a certain interval to ensure that your data is still up-to-date.

It’s not a big deal in most cases. I have never encountered a case (in the apps I’ve built,architected) so far where a race condition like this could have business impact. It can be in others for sure!

So 1: Even in huge apps, it almost never happens, it’s not like 1000 people are editing the same doc at the same time. If this happens maybe it’s time to rethink your db model?

So 2: If it happens in most cases it’s not a big deal.

You always have the option to bombard the db not a problem.

1 Like

I like the new feature. What are the ideal use cases for this? I’d guess something like chat, where edits are not common, and speed really matters.

1 Like

Socialize packages have Redis Oplog functionality baked in and activate if you have it setup. So this include feed, likes, comments,messaging and other.

I will have to look, what we need to update there (or can add to functionality) with the new release.

1 Like

Used this as it made sense. Saw the race condition happen twice or thrice in the console message (I think when running cypress tests). Hopefully we reach the point when this is already a big deal (we are still far from it)

1 Like