Performance of Meteor web application against 100000 parallel users

Waw. Thank you so much. Much appreciated.

just make an imgur album for screenshots

Why would you need 100k concurrent users? And why are you benchmarking on a laptop?

If youā€™re wanting high concurrency then I would go along with what other people are saying in the article and go with something like Elixir, Go or plain old C. However, even then thereā€™s little chance your laptop will handle 100k concurrent users.

Testing for 100k concurrent users on a Dell laptop is like a blacksmith forging a sword with a butter-knifeā€¦ Use the right tools.

If you decide to use Meteor then Iā€™d avoid MongoDB as @diaconutheodor makes some good points about oplog tailing above. Iā€™d recommend going with Apollo (GraphQL) and using Redis for speed with something like Postgres for your data integrity (essentically, use Redis as a cache for data that needs quick access).

If you are trying to bench for 100k users, this would take extremely cautious measures in the design of your application (your data structure/schema & pub/subā€™s especially) to be able to even attempt that.

Without seeing how you created your project, we are completely in the darkā€¦ so how are we supposed to help you?

We would have to see all the pub/sub code in the project, to even start to begin giving advice. And thatā€™s just a starting point. With Meteor (or any other framework) you would need to make many, many drastic measures to support that many users.

Thatā€™s why people think you are trolling/are not sure if serious. You are asking for advice, giving us very little to work with, providing very poor quality screenshots, giving far too little information, and testing on a system that was not built to handle such stressful tests.

How is your laptop possibly supposed to emulate a web server that you would use to support 100k concurrent usersā€¦? You would likely be deploying on cloud systems with external databases. Your laptop is not comparable!

Wow this is really impressive. Iā€™m so glad to know that you are making Meteor scalable, because I (and I guess a lot of other people) really want to use Meteor in production for big projects.

Keep the good work, we really need it and we appreciate it so much!! Thank you a lot! :))

1 Like

Please, please, please keep us updated on this. Start an email list or something so that I know when this is coming out.

Follow this guys: @cluxter @mz103

Here is my draft code:

Not reliable, not working for now, itā€™s more than I expected :slight_smile: but the POC is done. It just works smooth as butter :slight_smile: now, itā€™s just the ground work, TDD approach, lots of tests, etc.

3 Likes

Out of curiosity, what made you decide on the NPM redis package, rather than slava:redis-livedata and slava:miniredis?

Thanks to a lot of people here who shared very good constructive help and hints. I will continue my IT deep investigation with node.js and Meteor. If I will finish my testing and technical investigations, happy to share it with you all.
BR, Mohi

The responses here are rude and ridiculous.

Only a handful of helpful ones.

Yes, you will struggle to scale Meteor past 1000 concurrent users. We run 20 or so 512MB instances on DO and we can deal with a few hundred concurrent. Hitting around 700 or so and itā€™s been a struggle to get there.

Elixir is a good option if you want to scale massively and might be a better option than Meteor if you need to hit big scale.

And to those asking why youā€™d want to hit 100,000 users, the answer should be obvious, but if you ever have a popular website youā€™ll want that. Of course thereā€™ll be scaling challenges no matter what platform you use, but I canā€™t really fathom someone handling 100k concurrent with Meteor atm.

3 Likes

Thank you so much Elie, for the respect and for deep understanding for my initial thread. I totally agree with you. 5 years ago we did a similar load-testing with Government web-application based on Java enterprise framework and a lot of open source tools. It was really tough and a struggle, clustering Oracle db with JBoss and later we finally reached that using IBM Web-sphere, configuring the application server with commercial expensive load-balancer. It was really tough job scaling that web-application on that technologies, at that period of time.

I will check with your recommendation about Elixir. Again many thanks.

@mjarada2016 @elie

Yes, Elixirā€¦ Erlang VMā€¦ enough said itā€™s good, but:

https://www.quora.com/When-did-Facebook-switch-away-from-using-Erlang-for-Facebook-Chat

Give this a read.

2 Likes

Iā€™m subscribing to this thread :slight_smile:

@rhywden - you could tone it down a bit, iā€™ll second that opinion
@diaconutheodor - awesome - whether Redis or which technology is not the point, correct? Your improvement, if I am restating it correctly, is that you can push out diffs sooner by bypassing the Mongo layer is that right? Then tailing the oplog becomes redundant because the app maintains its own list of invalidations.

Redis is just a pub/sub system. We could use any pub/sub system. I just wanted to pick the fastest and the most stable there is.

You get the idea but Itā€™s way more than that. The diffs indeed may arrive sooner than with oplog, bc pub/sub of redis == killer speed, but there is so much more, namespacing, channels. No more ā€œprocessingā€ information we donā€™t need. We can finally build chat apps in Meteor, that are as fast as we would code them from scratch in an express() + websockets + pub/sub system + cpu & memory friendly :smiley: + It is database agnostic, meaning we could finally have reactivity on MySQL/Oracle/etc for now just focused on Mongo. :slight_smile:

7 Likes

Maybe a Apolloā€™s competitor is happening here :smiley:

All the best for you guys in this project. But, Iā€™m in doubt that MDG is desiring to help out with this. For obvious reasonsā€¦

1 Like

All this talk about making Meteor scaleable reminds me of the early days of Ruby on Rails. Iā€™m hoping to see the same explosion and level of impact on web development sometime very soon.

4 Likes

It already started. Redis Oplog was the first step of the equation, Iā€™ve finished the main functionalities, I am now working on writing tests to make it stable.

@aislan
Apollo Competitor ? I donā€™t really know about that. I just want to make Meteor great.

5 Likes

Iā€™m also planning on load testing our services before launch. Would it be possible to hear more on how you implemented the testing with jmeter and possibly see some testing script(s)?