Scaling Meteor on AWS for a short burst

How about this: failed to execute transaction on idbdatabase - Google Search

If I google this, I get info about a lot of other frameworks, but none that we are using. So I am wondering if this might be related to the recently applied REDIS oplog?

However, I found this now:

But this didn’t give me a real clue either.

We do use dynamic imports, which is mentioned in the beginning of that thread. Yet in this case, it will most likely be something Meteor related.

We’re not using the Firebase SDK, though.

1 Like

It’s possible but has limitation. If you have 2 -3 meteor instances then it’s still okay but you can see the cpu load increased on mongodb database instances. It’s better to use redis oplog.
I think if we could use change stream feature from mongodb then it’d be much better. We don’t need to use redis oplog package to scale horizontally.

1 Like

It is not. Is is related to the local storage being used by the browser. We cannot replicate this but it is still happening once or twice a month.

1 Like

Interesting. I never saw this before. But we also did not have that such a high load before. :sweat_smile:

This is not a paid advertising, but I am really really happy with Scalingo and just would like to support those guys.

Basically manages all that Dev Ops stuff for you and scaling works like a charm.

Tried to get AWS EBS running, for me it was a PIA; setting Scalingo was super easy and deploying now is just a matter of merging a branch in github, with no downtimes ever. :slight_smile:

1 Like

Meteor apps with many changes in the database will suffer with Horizontal scaling if Redis oplog is not used as all the containers (or servers) will catch up with all the changes.

In this case, a lot of changes, horizontal scaling is probably not going to help much as you have more power to change the database even more.

Note 1: this varies a lot from app to app but this is true in all cases, what I mean is that you can find apps scaling horizontally much more than others depending on how they publish the data.

Note 2: most of the apps will never suffer with MongoDB oplog. The only way to discover these issues in advance is by load testing your app. This was the main topic I was helping Tom.

1 Like

I’m using local database. Our database is a little bit large so it’s cheaper to setup a replica set.

For those kind of one-off events it could be very useful to have an additional (semi-) static / cached landing page which handles the first load. Then you already filter out many of the quick views which people do which then go away. That instantly reduces real “session” users. Serving a static or cached page is about as cheap as it gets.

I’ve noticed in the Showcase section of the Meteor website, many projects use a simple landing page and the Meteor app is under https://app.xxxxxx.com. This helps a bit with crawlers and general visitors but it is completely useless if you share a link from the Meteor app, something like an event or a post, etc. But this is why we use CDNs to deliver the Meteor bundle files.
At one point I was looking to get the dynamic imports to work with CDNs too but as usual, got caught into other things and dropped it. This is one of those things with a massive impact in the performance and cost saving.

I’m using loadbalancer and auto scaling feature from Google Cloud and it works well with Meteor. Normally I run only one Meteor instance but when the load increased, it will auto add more instances.