For example, an admin-user updates a game scoreboard on a server-container running our admin-tool micro-service. Meanwhile, we have about 500 connected clients to our game-viewer micro-service that are watching real-time updates to the scoreboard of some live game.
We use redis-oplog’s Vent tool because it’s precise and easily manages events across all micro-service containers. And the subscribe-able game channel is the same across the entire app and services. It also lets us remove native pub-sub from the game-viewer micro-service to reduce overhead.
So when the above scoreboard update happens, it sends the update out to Redis on a specified game channel, then each server-container of the game-viewer micro-service receives that update and sends it down via DDP to all connected clients that are subscribed to that channel.
The issue we’re running into is that each server-container of the game-viewer can handle hundreds of connections, but the turn-around time for the DDP messages to each connected client are taking sometimes up to 15 to 20 seconds (in this case it’s about 500-600 clients connected to a Galaxy Double Container - so it has some horsepower). As redis-oplog sends each connected client the DDP update in a linear sequential order (if I understand its code correctly). This linear sequence is time-consuming with hundreds of clients and likely other requests happening on the server.
My question is - I’m assuming once you get down to the DDP level, you have to update each connected client one-by-one? There’s doesn’t seem to be a way around this. I noticed there’s some packages like streamy and meteor-custom-protocol that mention broadcasting of messages. Does anyone know if this ultimately also does a one-by-one DDP update to each connected client? Or is there some way to actually do a true broadcast that would only require the server to send one update that is then received by all clients in some highly efficient way than the one-by-one updates.
@diaconutheodor Am I understanding what’s happening under the hood with redis-oplog correctly? Thanks for a response.