Meteor cost at scale will this become a major issue?

I have never been worried about scaling meteor apps and each time I hear the scaling questions about meteor I feel it one of those buzz questions and in truth not a smart question to ask in most context. My thoughts are clear on this in the followng post.

But lately my biggest concern with Meteor and this might not be a full meteor issue but more about websockets and reactive applications. Is cost at scale. I believe everyone has been ignoring this. But maybe this isn’t a big deal and every language, framework or tech is similar. Why I’m asking about this here.

From what I can tell taking my info from around the web and differnet posts like the following post, First Experiences Scaling A Meteor App, Scaling Meteor to 20,000+ Users in 7 Days

The average VM depending on your code can handle anywhere between 150 to 350 concurrent users. So if you have 10,000 concurrent users which I believe is not crazy consumer scale yoru looking at the following

I’m using Modules.io as Meteor Galxey in my point of view is not for startups and not realstic I think they missed the ball here.

Using the 512 RAM servo size which is $28.80 per servo and $288 for 10 so if each servo can handle 350 current users (10,000 / 350) I need about 28 servers for simple sake let round up to 30 that would be $288 * 3 = $864 per month.

And let’s look at factor of 5 that would be $4,320 per month

I wonder if this is ok and in pair with other languages / frameworks?

I know from my previous start which was Ruby on Rails with Angularjs front end we were able to scale to millions of requests at the cost of $1,000 to $2,000 per month (to clarify that was on Microsoft Azure, VM Linux. We had replicate set for Mongo primary and two secondaries and then for the app a load balancer and two machines plus running and another for backup if one went down. For the client just one machine and running mode of the static files from CDN).

Thanks for this request. I am also wondering how Meteor does actually scale when user numbers are getting bigger. And how much this will cost.

1 Like

I want to know when they say “concurrent” what do they mean? Are the users actively fetching data or are just connected?

If we use little to no pub/sub and rely on methods to fetch data, how many users can we handle at a time?

And what about if they are all real time pub/sub? Then how many concurrent users?

For simple sake we can say a concurrent is a user that is connected to your site pub/sub to data.

Of course you have to scale your DB for read and writes but that’s another topic.

I believe that “cheap” scaling comes with proper architecture of the application. Granted one of meteors big draws is that it is easy to pick up for almost anyone and proper architecture is not webdev 101.

I have built and deployed meteor systems that were able to handle 10’s of thousands of writes per second while maintaining around 4,000 concurrent users across 3 m3.medium EC2 instances. We use a microservice architecture with kubernetes as the container management system.

This solution costs just over $100/month and could scale up to at least 10,000 concurrent users for not much more (I don’t think).

I know Plenty of Fish (POF) isn’t written in meteor but it gets 1.2 billion page views/month, and 500,000 average unique logins per day and it only uses 2 load balanced web servers with 2 Quad Core Intel Xeon X5355 @ 2.66Ghz, 8 Gigs of RAM (using about 800 MBs), 2 hard drives, runs Windows x64 Server 2003 and 3 DB servers.

I personally believe that the “cheap” scalability of an application is the duty of the developer and not of the framework. That being said, some things inside of a framework can make scaling easier or more difficult (pub/sub).

3 Likes

If you have 150 to 350 concurrent users, is $864 per month cost-prohibitive?

no1 is stopping you from grabbing 1gb droplet on DO and try how many authenticated DDP connections it can handle
for example https://github.com/meteorhacks/meteor-down comes to my mind

Oplog tailing

You might want to watch the video linked in the above thread.

Oplog tailing is the bottleneck for scaling with meteor. Scaling your mongo cluster isn’t going to do much for you.

That’s the impetus behind the Phoenix Thread. Reactivity that scales both vertically and horizontally.

2 Likes

@shock As I suggested in my post this data is somewhat already out there it might not be exact but no need to run a test

1 Like

@jacobin I know this why my post was not directed about the DB scaling I’m interested in comparing the cost of scaling the app.

Horizontally scaling does not alleviate the oplog tailing O(no) issue. Discussing the cost of doing something you can’t actually do is rather silly.

The cost, will be hiring devs to remove meteor’s reactivity and replacing it with Redis/RethinkDB/Postgres pub/sub.

Depends on your app. Is it by its nature really intensive/realtime/dataheavy? Plus the nature of your target audience; is this a niche project/or does it appeal to the general public. Really, your business model. Can I get X $ from Y clients to cover Z $ in server fees.

$864 sounds way over what I’d like for 150-350 concurrent users, but then again I don’t know what exactly your app does. @khamoud’s answer makes me optimistic! Im not sure I’ll be able to reach that efficiency, but it’s nice to see that its certainly possible. And @khamoud thanks for the data! This isn’t something you can really find by googling, especially not with this specificity :blush: Most people try to keep their business expenses as a guarded secret.

One of the problems is that by default most everything with meteor seems to be reactive. Cursors for user profiles? I even see people doing insane things like pushing reactive chat through the database instead of rabbitmq/mttq.

Certainly using a stateless rest/microservices architecture like khamoud will get you to scale a bit higher… but at some point the oplog tailing is still going to cry O(no!) when you add in reactivity.

redis + MySql cold storage is my Solution for a lot of the heavier read/write data. Meteor supports multiple dbs, but it’s far from firstclass.I think it’s time meteor got some new significant db functionality besides minimongo and oplog trailing…

Also, I think you’ll like:
https://meteor.hackpad.com/Proposal-for-deprecating-user-profile-3UljX1VayvV

Ran into this myself earlier today:

1 Like

I agree. Hansoft’s solution mentioned in the video uses Redis as a broker between meteor and the database. It was tested up to 10k users per node with flying colors.

@SkinnyGeek1010 is working on using Phoenix in a similar manner, but only for rest requests not reactivity.

MDG supposedly had one guy ‘starting work investigating’ a database abstraction abstraction layer a few weeks back. That just isn’t enough. Putting people to work on a new frontend when your backend doesn’t really work is insanity.

6 Likes

Pls give me the link where is written how many concurrent users 1gb droplet can handle without anything else than pub/sub on accounts for users to log in.
all the other is application specific and does not automatically need to use any other pub/sub.

I believe it was calculation for 10k users

That makes a lot more sense lmao.

Codefights has 200k users. I wonder how much they’re paying for Galaxy.

I think it also depends on how much you’re trying to do with your app. I read a comment by Arunoda recently that said that 70 percent of publications don’t need to be reactive. This means every app is wasting a lot of resources. If you hit really big scale you probably want to work out other solutions that don’t make every call for data reactive. He’s coming out with a post soon on how they made use of graphql to solve this problem for kadira.

Either way, the more real time your app, the more it’ll cost you to run each month. Scaling any site might mean having to cut down on some features to make it work at massive scale for a low cost. Each site will have its own scaling challenges. Not sure how smart it is to compare meteor and rails. If rails is less offering less features, then of course it’ll be easier to get more concurrent users.

That’s total registered users. If you watch their video talk, Codefights peak concurrent users was 1000 over 20 Galaxy containers.

iirc The meteor podcast on Galaxy said the entry tier gives you 10 containers at $599. So perhaps they use the $999 tier.

They also discussed jumping through hoops to limit subscriptions.

2 Likes