Cost of subscriptions

Hi! I know this will be hard (or maybe impossible) to answer, but maybe someone has experience.

I already have a meteor application, but it’s only used by ~10 concurrent users so it runs perfectly even in the smallest galaxy container. I’m in the process of making a new version. This one will be used by at least 2-300 concurrent users, maybe even more (6-700).

I have a collection with ~50k items and another one with ~500k items. Both collections can grow over time (around +50k/+500k in 6 months). Every item is linked to a user with a userId field. The updates are not super-intensive. There’s a background process that downloads new data from an external API and updates the items when necessary. It runs every few minutes and updates a few hundred items in each run.

The users can see a list of these items (initial items: 40 with an infinite-scrolling to get more) and they can apply filters/sort to the list. Given the bad reputation of subscriptions, I don’t know if I should get the necessary data with a subscription or with method calls.

Right now I’m using easy-peasy which is basically a redux store and populate the store using grapher queries. But I have no clue what would it cost in terms of CPU/containers if 400 users would subscribe to their documents. It would be nice if they could see the updates in real-time but I’m too afraid to use subscriptions :slight_smile: Is there a general rule of thumb? What can I expect? I added redis-oplog with the default settings (i.e. without changing app logic).

One thing you have to ask yourself is whether “nice to have” is enough of a justification to stuff your memory with thousands of documents that will for the most part not be updated (unless I am mistaken). If the docs we are talking about are news articles for example, it is pretty wasteful to use subs, as they are mirrored in memory on the server for each user. Is it necessary?

Subscriptions are great for what they do, give the user live updating documents, but it is an expensive process, and should be well justified. If two users are collaborating on a doc for example. If it is “dead” data, that does no change, it is not a good use case for subs.

I would blanket recommend using methods unless you have a use case that requires the docs to always be up to date.

Even if the server was able to handle it, using subs just to fetch data is noticeably slower than fetching with methods.

In our use cases, we very rarely sub on more than one doc at a time for example.

1 Like

The documents in the collections represent reservations and messages from guests. There are multiple external services where these documents are retrieved from, e.g. Airbnb, Booking.com, etc.

The “users” are airbnb hosts and they see a list of their reservations and messages from their guests. Most users will only have 2-3 reservations/day so their list of reservations won’t update often, but it’s certainly good to see a new reservation at the top of the list when it arrives (instead of seeing it only when they refresh/return).

My main question is that what would it cost to have a list of documents with ~300 concurrent users. Do I need 2 medium app containers or 10 large?

In that case it is definitely more in the range of 2 medium containers than 10 large for sure, but in any case I would opt for the largest container, and have 1 or two of those. We noticed a huge difference in raw response times and performance independent of users between the different containers.

In a low-update intensive app I would expect you will be able to handle 300 users with one large container

1 Like