Waw. Thank you so much. Much appreciated.
just make an imgur album for screenshots
Why would you need 100k concurrent users? And why are you benchmarking on a laptop?
If youāre wanting high concurrency then I would go along with what other people are saying in the article and go with something like Elixir, Go or plain old C. However, even then thereās little chance your laptop will handle 100k concurrent users.
Testing for 100k concurrent users on a Dell laptop is like a blacksmith forging a sword with a butter-knifeā¦ Use the right tools.
If you decide to use Meteor then Iād avoid MongoDB as @diaconutheodor makes some good points about oplog tailing above. Iād recommend going with Apollo (GraphQL) and using Redis for speed with something like Postgres for your data integrity (essentically, use Redis as a cache for data that needs quick access).
If you are trying to bench for 100k users, this would take extremely cautious measures in the design of your application (your data structure/schema & pub/subās especially) to be able to even attempt that.
Without seeing how you created your project, we are completely in the darkā¦ so how are we supposed to help you?
We would have to see all the pub/sub code in the project, to even start to begin giving advice. And thatās just a starting point. With Meteor (or any other framework) you would need to make many, many drastic measures to support that many users.
Thatās why people think you are trolling/are not sure if serious. You are asking for advice, giving us very little to work with, providing very poor quality screenshots, giving far too little information, and testing on a system that was not built to handle such stressful tests.
How is your laptop possibly supposed to emulate a web server that you would use to support 100k concurrent usersā¦? You would likely be deploying on cloud systems with external databases. Your laptop is not comparable!
Wow this is really impressive. Iām so glad to know that you are making Meteor scalable, because I (and I guess a lot of other people) really want to use Meteor in production for big projects.
Keep the good work, we really need it and we appreciate it so much!! Thank you a lot! :))
Please, please, please keep us updated on this. Start an email list or something so that I know when this is coming out.
Follow this guys: @cluxter @mz103
Here is my draft code:
Not reliable, not working for now, itās more than I expected but the POC is done. It just works smooth as butter now, itās just the ground work, TDD approach, lots of tests, etc.
Out of curiosity, what made you decide on the NPM redis
package, rather than slava:redis-livedata
and slava:miniredis
?
Thanks to a lot of people here who shared very good constructive help and hints. I will continue my IT deep investigation with node.js and Meteor. If I will finish my testing and technical investigations, happy to share it with you all.
BR, Mohi
The responses here are rude and ridiculous.
Only a handful of helpful ones.
Yes, you will struggle to scale Meteor past 1000 concurrent users. We run 20 or so 512MB instances on DO and we can deal with a few hundred concurrent. Hitting around 700 or so and itās been a struggle to get there.
Elixir is a good option if you want to scale massively and might be a better option than Meteor if you need to hit big scale.
And to those asking why youād want to hit 100,000 users, the answer should be obvious, but if you ever have a popular website youāll want that. Of course thereāll be scaling challenges no matter what platform you use, but I canāt really fathom someone handling 100k concurrent with Meteor atm.
Thank you so much Elie, for the respect and for deep understanding for my initial thread. I totally agree with you. 5 years ago we did a similar load-testing with Government web-application based on Java enterprise framework and a lot of open source tools. It was really tough and a struggle, clustering Oracle db with JBoss and later we finally reached that using IBM Web-sphere, configuring the application server with commercial expensive load-balancer. It was really tough job scaling that web-application on that technologies, at that period of time.
I will check with your recommendation about Elixir. Again many thanks.
Yes, Elixirā¦ Erlang VMā¦ enough said itās good, but:
https://www.quora.com/When-did-Facebook-switch-away-from-using-Erlang-for-Facebook-Chat
Give this a read.
Iām subscribing to this thread
@rhywden - you could tone it down a bit, iāll second that opinion
@diaconutheodor - awesome - whether Redis or which technology is not the point, correct? Your improvement, if I am restating it correctly, is that you can push out diffs sooner by bypassing the Mongo layer is that right? Then tailing the oplog becomes redundant because the app maintains its own list of invalidations.
Redis is just a pub/sub system. We could use any pub/sub system. I just wanted to pick the fastest and the most stable there is.
You get the idea but Itās way more than that. The diffs indeed may arrive sooner than with oplog, bc pub/sub of redis == killer speed, but there is so much more, namespacing, channels. No more āprocessingā information we donāt need. We can finally build chat apps in Meteor, that are as fast as we would code them from scratch in an express() + websockets + pub/sub system + cpu & memory friendly + It is database agnostic, meaning we could finally have reactivity on MySQL/Oracle/etc for now just focused on Mongo.
Maybe a Apolloās competitor is happening here
All the best for you guys in this project. But, Iām in doubt that MDG is desiring to help out with this. For obvious reasonsā¦
All this talk about making Meteor scaleable reminds me of the early days of Ruby on Rails. Iām hoping to see the same explosion and level of impact on web development sometime very soon.
It already started. Redis Oplog was the first step of the equation, Iāve finished the main functionalities, I am now working on writing tests to make it stable.
@aislan
Apollo Competitor ? I donāt really know about that. I just want to make Meteor great.
Iām also planning on load testing our services before launch. Would it be possible to hear more on how you implemented the testing with jmeter and possibly see some testing script(s)?