You must be new around here.
1 Like
Actually I remember watching that in real time.
Thatâs your whole argument? blinks
1 Like
Try understanding how oplog tailing works and itâs limitations. There are no arguments, just statements of fact. There are multiple threads on the topic.
Iâm now the second person you have tried to troll in this thread. blinks
4 Likes
So the people who are successfully using Meteor with more than 100 concurrent users are, by extension, trolling you because I donât agree with you?
Sent from Samsung Mobile
1 Like
Your ignorance of the issue has only to do with your arguing from a place of ignorance, nothing else.
Applications with more than 100-200 concurrent users, such as Codefights with a claimed 690 max, has had to a) deploy across 10-20 meteor nodes and b) jump through hoops to eliminate oplog driven reactivity everywhere possible.
Phoenix can handle thousands on a single node using commodity hardware and Rethink or Redis pub/sub instead of oplog tailing. Hence the interest among meteor veterans.
Considering oplog tailing being a limiting factor on horizontal scaling is a well known issue, the only person you are trolling at this point is yourself.
2 Likes
So write the app in Meteor then when you hit #1 in product hunt hire geniuses to rewrite it in whatever. Pheonix, Go, C, Assembly, etc.
Facebook started out on PHP because it was simply a comfortable language. It could have been written with C even back then and would have started with better performance. None of that matters. Itâs more important that you can add features and evolve your product quickly in the beginning. Once the time comes you could even invent your own framework to solve your specific problems.
1 Like
In general I agree that you shouldnât micro-optimize for things that may never happen. However, going too far in that direction can be dangerous. I naively thought that MDG would fix scaling before I hit issues. Iâm only offering anecdotal evidence from my own experiences but hereâs my take:
I had the same attitude when I thought that I would only have a few hundred concurrent users. However, business requirements changed before I even got the project launched and now I need to handle several thousand concurrent users from the start and within a year tens of thousands of concurrent users are reasonable (due to the customer driving their own traffic to the app).
This puts me in a pickle. Itâs also the reason I made the meteor_elixir repo, as a spike to prevent re-writing the whole app before itâs even launched. It basically just cuts out subscriptions and delegates that to Phoenix while the rest is handled by Meteor.
Second use case:
An app built in Meteor. The data transfer is very minimal/optimized because of the mobile restraint. We thought we wouldnât have a lot of traffic but alas, one day we had 400 concurrent users blasting the system. Crap I had to spin up 4-5 instances to keep it from crashing (5 keeps 404 errors to little/none). 5 - 1gb servers cost $285 on Modulus and I couldnât host on DO at the time (mup didnât exist).
In this case Meteor is able to keep up but it was not efficient and expensive. It was for a âfreeâ app so no revenues to pay for hosting (thatâs another topic 
).
I was able to reduce costs by cutting realtime on some features and using long polling every 30s (even though getting a new chat message instantly would be a better experience).
Phoenix is more efficient for these use cases and even though itâs more work upfront, over time it pays off. Thereâs no doubt that you can build an app faster with Meteor, thereâs a tradeoff to be made. No free lunch.
11 Likes
@SkinnyGeek1010 I wonder if it makes sense to just build a wrapper in Node that emulates Meteor for Methods and Publications but speaks to a Phoenix layer instead of doing OpLog, Mongo, etc.
A framework built on Pheonix that fires up on the server and listens via API on which database entities to create, monitor, etc, and passes back the data to the Meteor server which then processes it and streams the data down the wire. If the bottle neck is OpLog processing this would solve the issue, no?
The winner of the hackathon made an app that pulls data on rest end points and deliver a ddp connection.
Here there is no pulling here. We are listening to channels between servers (Phoenix and Meteor) and returning a ddp websockets connection from Meteor on the server to client.
Canât it be completely automated, requiring developer thinking strictly in JavaScript, with the proper compliment written on Phoenix?
1 Like
While spiking out this solution I was trying to do something similar. One of the solutions was to subscribe to Phoenix on the meteor server in the publish, and use low-level methods to send it down the socket.
However, youâre opening yourself up to a myrid of race conditions and limiting how much one server can scale. Node is really great at stateless requests like a JSON API but as one of the video from earlier showed, itâs hard to scale stateful websocket connections.
Itâs quite possible to serve a million concurrent users with one Phoenix server and a few Meteor servers (enough to basically log in the user and return their profile). Offloading the initial Meteor index.html to a CDN can free up Meteor resources (iâve benchmarked ~67 pages a second on a DO 1gb box to just server the index page).
If you proxied Phoenix through node youâre meteor cluster count would skyrocket because you can only scale so high vertically with Node.
Erlang was built from the ground up to handle this so it makes sense to let it handle it. Ironically this makes the system much much more simple than trying to proxy it though node/meteor.
At the end of the day I decided that all the extra work doesnât buy you anything and doesnât make life easier. Both ways require a knowledge of enough Elixir and Phoenix to setup a channel and query data ( a bit less if youâre using Rethink as it will push new changes down).
On the meteor client side all you need to do is:
// we'll use a local collection to store incoming data
Chats = new Mongo.Collection(null);
channel.join()
// prime minimongo with last 20 chats on join
.receive("ok", resp => {
resp.initialChats.forEach(doc => {
Chats.insert(doc);
});
})
/// ...
channel.on("new_msg", doc => {
// upsert because messages we sent will result in duplicates if we insert
Chats.upsert(doc._id, doc);
})
The amount youâd have to learn is minimal and itâs far easier than trying to write your own reddis pubsub to scale livequery. Like I said, no free lunch 
5 Likes
Thanks for the details. I still donât see why not âcross that bridge when needed.â Like you said the server end of Meteor is often fairly straight forward. Itâs great that Phoenix is there and it would be even better if there was a simple easy-breezy way to scaffold a Meteor-Phoenix project where you write your publications and methods on Phoenix and subscribe in Meteor using the same /Client /Server folder structure and then when you âFoobar Deployâ it does what it needs to do.
I donât mind learning Erlang at all, Iâm sure its a fine language.
Is there a sample project with Phoenix as the backend and Meteor on the front with instructions on how to get the Phoenix part up and running on Mac? Iâd give that a try for sure!
[quote=âBabak, post:273, topic:13519â]
Thanks for the details. I still donât see why not âcross that bridge when needed.â[/quote]
Most meteor projects will never get to that bridge. If they do, itâs not a realistic expectation to monetize those few hundred users and hire âgeniusesâ to rewrite your app from scratch in the few short days before performance complaints on twitter kill itâŚ
Two backends simultaneously is somewhat silly. Choose one, use occamâs razor on the other. Skinnygeek had to try it as his project was too far along.
If you chose node because of skillset limits, use one of the myriad of npm libraries that integrate redis/postgres/rethink pub/sub feeding reactivity through Redux/React on the client.
That path will let you at least scale horizontally into the thousands before it explodes as you need to share state between nodes through socket.io. This limitation was discussed in the video Skinny mentioned.
For brand new apps where scalability is a requirement, Phoenix on the backend and Redux/React on the frontend is the path of least resistance/time/money/complexity.
4 Likes
A lot of the time waiting until you get there is just fine. I just chose wrong. Twice. For most people the efficiency/cost will creep up faster than no service. Thatâs an easier path to have as migrating would only cost more $ and not losing users. I guess my main point was to plan ahead and leave enough buffer for the unexpected (that was my problem).
and it would be even better if there was a simple easy-breezy way to scaffold a Meteor-Phoenix project where you write your publications and methods on Phoenix and subscribe in Meteor using the same /Client /Server folder structure and then when you âFoobar Deployâ it does what it needs to do.
You can almost do this. Meteor doesnât have any scaffolding generation but itâs very fast in Phoenix (with passing tests!). Once you learn how it works you can wire the two together in less a couple of mins. Deploying is as simple as a heroku push⌠itâs actually easier than Meteor to deploy (on Heroku, live code patching with Exrm is a bit tricky at first).
Is there a sample project with Phoenix as the backend and Meteor on the front with instructions on how to get the Phoenix part up and running on Mac? Iâd give that a try for sure!
If you use the meteor_elixir repo above you could pull that down and run mix deps.get to pull in packages and then mix phoenix.server to get it running. However, since Elixir is not installed yet that can be installed here. It also required mongo to be running in the background mongod. Once itâs running then you can meteor run and the chatroom should be working 
.
However, it may be easier to run through the up and running guide first as this will get a hello world app running (albeit with Postgres by default).
5 Likes
Why do you sound so condescending? Do you have a human communication âlimitâ or what? Re-read your comment and see if it sounds convincing, if it does, stop talking to me.
Reddit itself has pretty much crashed a few times, people still use it. Twitter too. Iâve had Reddit crash my server before. No big deal, just scaled up and that was that.
Facebook was written with PHP because it was convenient. Github could have been written with in C++, but it was written with Rails. Instagram started with Django. So what?
Being lazy can be a programmerâs virtue, this is one reason why getting start with Meteor was wisely made easy. Iâm not going to go jump through hoops to learn Phoenix especially with people like you trying to promote it.
Maybe you guys should streamline your presentation and get the elevator pitch in order and stop compensating for that marketing âlimitâ by attacking people who actually warm up to the idea of trying your Meteor forum spam product.
3 Likes
I know all about Oplog and its âlimitationsâ. You shouldnât assume someone doesnât know about something just because they donât mention it.
Youâre also the one who responded to me in point of fact (trolling?), and is now resorting to Ad Hominem.
Very mature!
WhateverâŚPhoenix Good, Meteor Bad. I get it.
Iâll just stick to the facts that they both have identified use cases thanks.
2 Likes
Some of these guys donât sound like guys youâd want to have a beer with.
3 Likes
Any thoughts on meteor vs phoenix pros/cons for a p2p type app?
1-1 or 1-small group , not 1-1000âs++
2 Likes
Cool, thanks, good info.
2 Likes
Itâs hard to say without knowing more of the business requirements. Would the p2p app be like a chat app or something similar where messages are passed around? Or realtime updates? Would an SQL or graph database (like Neo4J) be a better fit than a document database?
Also if you have a large amount of 1-1 type of publications that can eat up resources too, though not for a while.
If the app was more of a startup where youâre trying out a new product to see if thereâs market fit I would almost certainly choose Meteor because you can iterate quickly to see if itâs going to work, and likely it will change a lot.
However, if one knew each framework equally well, and also was really quick with a non-blaze frontend (like React/Angular/etcâŚ) then I think it would be a draw. React + Redux replace Blaze + Minimongo, Channels replace publications (and are more simple IMHO), Ajax (or channel messages) replaces methods.
Some areas that donât overlap are authentication and Mobile support. No easy cordova support without Meteor but you have easy React Native support with the existing Phoenix API.
Authentication is tricky. If you need anything other than a standard username/password or Facebook Oauth then youâll spend more time doing it with Meteor than with a lower level framework.
It was frustrating to build an SMS pin code signup/login with Meteor because auth is so hidden. You canât just work with the hooks very easily (in Metoer) and theyâre not documented well.
If you havenât used JWT tokens yet (theyâre new) than it will take some time to get used to it but itâs very very similar to Meteorâs _loginToken thatâs stored in local storage. You login, the sever hands you a token and you save it in storage and pass it back with each request to prove the user is logged in.
Iâm getting close to the point where I could build a React/Redux/Phoenix app as fast as a Meteor app. However, if youâre building a throw away prototype or hackathon app⌠Meteor will win hands down (meteor add accounts-ui is hard to beat! unfortunately none of my clients want to use it)
Sorry for the rambling but hopefully that answers your question!
5 Likes
Not rambling at all. Thanks for your reasoned and intelligent perspective.
2 Likes
Two more things I just thought of that might tip the scales toward Phoenix.
If you enjoy the âUnix wayâ, Unix piping, and (pragmatic) functional programming, the serverside might be easier to write in Elixir (dare I say more fun). This functional style could be a love it or hate it thing so thatâs something to consider.
To play devilâs advocate, Elixir is new and there isnât a library for everything like in NPM so you may need to write some packages yourself.
The second thing is micro-services. If thatâs your jam then OTP will be your best friend. Itâs the most impressive piece of machinery iâve seen in a while.
I tried using headless Meteor apps for microservices and I regret it. The 80% use case works great but the edges will burn you. Itâs just not setup for that.
I personally think itâs better to build a monolith that is abstracted out enough so that switching to a microserivce is only a few hour job. Having several services just means more work in the beginning.
2 Likes