Scaling Meteor on AWS for a short burst

How about this: failed to execute transaction on idbdatabase - Google Search

If I google this, I get info about a lot of other frameworks, but none that we are using. So I am wondering if this might be related to the recently applied REDIS oplog?

However, I found this now:

But this didn’t give me a real clue either.

We do use dynamic imports, which is mentioned in the beginning of that thread. Yet in this case, it will most likely be something Meteor related.

We’re not using the Firebase SDK, though.

1 Like

It’s possible but has limitation. If you have 2 -3 meteor instances then it’s still okay but you can see the cpu load increased on mongodb database instances. It’s better to use redis oplog.
I think if we could use change stream feature from mongodb then it’d be much better. We don’t need to use redis oplog package to scale horizontally.

1 Like

It is not. Is is related to the local storage being used by the browser. We cannot replicate this but it is still happening once or twice a month.

1 Like

Interesting. I never saw this before. But we also did not have that such a high load before. :sweat_smile:

This is not a paid advertising, but I am really really happy with Scalingo and just would like to support those guys.

Basically manages all that Dev Ops stuff for you and scaling works like a charm.

Tried to get AWS EBS running, for me it was a PIA; setting Scalingo was super easy and deploying now is just a matter of merging a branch in github, with no downtimes ever. :slight_smile:

1 Like

Meteor apps with many changes in the database will suffer with Horizontal scaling if Redis oplog is not used as all the containers (or servers) will catch up with all the changes.

In this case, a lot of changes, horizontal scaling is probably not going to help much as you have more power to change the database even more.

Note 1: this varies a lot from app to app but this is true in all cases, what I mean is that you can find apps scaling horizontally much more than others depending on how they publish the data.

Note 2: most of the apps will never suffer with MongoDB oplog. The only way to discover these issues in advance is by load testing your app. This was the main topic I was helping Tom.

1 Like

I’m using local database. Our database is a little bit large so it’s cheaper to setup a replica set.

For those kind of one-off events it could be very useful to have an additional (semi-) static / cached landing page which handles the first load. Then you already filter out many of the quick views which people do which then go away. That instantly reduces real “session” users. Serving a static or cached page is about as cheap as it gets.

I’ve noticed in the Showcase section of the Meteor website, many projects use a simple landing page and the Meteor app is under https://app.xxxxxx.com. This helps a bit with crawlers and general visitors but it is completely useless if you share a link from the Meteor app, something like an event or a post, etc. But this is why we use CDNs to deliver the Meteor bundle files.
At one point I was looking to get the dynamic imports to work with CDNs too but as usual, got caught into other things and dropped it. This is one of those things with a massive impact in the performance and cost saving.

I’m using loadbalancer and auto scaling feature from Google Cloud and it works well with Meteor. Normally I run only one Meteor instance but when the load increased, it will auto add more instances.

This is not a paid advertising, but I am really really happy with Scalingo and just would like to support those guys..

.

That’s exactly how I do it. The landing page is plain Vanilla HTML and scores very high in the Lighthouse. If a visitor wants to sign-up for my app (which is the whole purpose of the landing page) then indeed the URL is yourDNA.family - Discover your DNA family and while it takes a bit longer on mobile (6.4s FCP / 10.8s TTI / 46 performance score - it’s delayed by a blocking 3rd party URL’s like Zendesk) it’s much better on desktop (2.6s FCP / 2.6s TTI / 68 performance score - now it’s more FontAwesome that delays it).

Memory consumption (39%) and CPU is very low on the smallest AWS EC2 server and as this server is only used during sign-up process (and password forgotten) it never breaks a sweat or has to scale up (though it can).

So IMO it’s best practice still for Meteor to split up landing pages (and optimize them ideally with Vanilla HTML or Svelte eg) and the actual Meteor app.

I still get close to 100 in Lighthouse both mobile and desktop with a self hosted Prerender. In cases such as the AWS website where the site is massive already before a log in, I feel a “landing page” is useful. Otherwise, I prefer to give the user the whole bundle “in advance” delivered from a CDN close to the user. Trying to imaging who these would look like: app.facebook.com or app.linkedin.com etc …

1 Like