Request for new publication strategy SHARED_MERGE

performance
1

I’ve been reviewing publication strategies in order to do some app optimization. I think I have a common use-case that could prompt a new publication strategy that would be highly optimal. I’m curious if anyone from Meteor (@nachocodoner) could sound off on how hard this would be.

The use-case is a game with a high-number of real-time simultaneous clients, say 10,000, who all subscribe to the same game document. The publication returns 70 various fields that can all change in real-time but not all at once (e.g. status, playersAlive, phase, powerUpsLeft, etc.). All 10,000 players use the same publication observer and, at least for the game subscription, receive the exact same client data and updates. There’s nothing unique, per player, about the game document, nor is it subscribed to more than once by any other subscription, nor is it ever removed. We just need to follow real-time diffs on the fields.

It seems all the current publication strategies do not optimize for this seemingly common use-case:

  • SERVER_MERGE: This is overkill as all clients have the same game document, that is only subscribed to once, not used in other subscriptions, and not removed. So the server doesn’t need to keep a copy of every client’s game document. Yet, this is the only publication strategy that sends field diffs down, which is what we want.
  • NO_MERGE_NO_HISTORY: This sounds good in that it only send updates and doesn’t store each client’s data, but it sends the entire document each time, so 70 fields, when we just need single field updates.
  • NO_MERGE: This is like NO_MERGE_NO_HISTORY but includes removed support, which we don’t need. Still sends all 70 fields.
  • NO_MERGE_MULTI: This tracks for multiple publications, which we don’t need, and still sends the entire document, so all 70 fields.

Would it be possible to add something like a SHARED_MERGE or SINGLE_MERGE that for a given publication observer, it keeps one copy of the document that has been sent, diffs any changes, and then shares that with all clients of the same publication observer?

It seems something like this would deliver the pub/sub magic of Meteor but greatly reduce the memory overhead of a scale-level of connections, albeit with the right use-case/circumstance (that seems fairly common to Meteor’s use-case).

Would just using NO_MERGE_NO_HISTORY still be way more optimal than SERVER_MERGE even with sending 70 fields to every client?