Limit rate at which publications transmit data

1

I have a Meteor app that performs a lot of data processing and constantly updates MongoDB while displaying the ongoing results to the client. Problem is, this of course takes a lot of resources and doesn’t even work well for a single client connected from the other side of the globe due to the latency.

I’m wondering if I can somehow only publish the changes at an interval and then mock the ongoing updates on the client to make it seem as if there is a seamless flow of data. So I basically just want Meteor.publish to send data at a given interval.

Anyone have some idea how to go about doing this? Any advice is greatly appreciated.

2

After reading this blog post, I disabled oplog tailing and used polling like so:

Meteor.publish('data', str => {
    check(str, String);
    return Data.find(
        {instanceId: str},
        {disableOplog: true, pollingThrottleMs: 1000}
    );
  });
1 Like
3

You might also have a look at redis-oplog package if you wanted to keep things more real time. It might also be more performant than poll and diff since this too can be a pretty expensive task.

1 Like