Mate, I don’t want to get down in the dirt with you, so I’ll take that with a smile.
Listen, if you read what I wrote then it becomes clear your comment actually takes such a narrow view of what I said it becomes a distortion.
The thing is I’m not going to chunk what I have now for something new (React) just because it’s new mate, I’m going to slowly migrate over to React because if I don’t, I’ll be in a real quandary at some point not too far off for all the reasons I mention in that comment you linked to.
@sonicviz, the posts that I’ve read from you have been balanced and well thought out so far mate. Your last post, where you go into React, I think would be welcomed in the ‘Next steps on Blaze and the view layer’ thread too.
The “Phoenix tutorial” that is declining in popularity is the software for flashing Nokia phones called Phoenix. (See the related keywords below the main chart.) I don’t think that the Elixir framework is represented on that page.
(Sorry, I didn’t see the other replies when I posted this. Here’s another (imperfect) query though.)
I’m not as far along as you business wise as I’ve only done an MVP with Meteor, but I understand the position MDG has put you in and others in.
MDG needs to put business stability first over features.
I’m still 50/50 on whether big React push actually a solid technical decision driving it (why not Angular or as people point out keep their own) or Investors are pushing it in that direction hoping to move it closer to being an acquisition target.
re: Phoenix
I have some other potential projects on the boil so I’m willing to investigate Meteor alternatives.
I understand why people get excited about new technology and platforms but I wish the evangelism would take a backseat to objective tech and business eval criteria.
Meteor is making a sound technical decision by future proofing the tech stack, allowing you to use whatever front-end you feel like. It’s not exactly code to interface, but it’s thankfully moving in that direction.
This includes angular, which will likely end up having better integration than react in the long run. MDG hired a single React dev, who doesn’t appear to really do much. Meteor claims to be working directly with the Angular team through Ugio, who appears to be doing quite a bit.
You seem to think there are not objective criteria at play and it’s all about shiny objects. Nothing could be farther from the truth.
For some folks being able to scale beyond 100 concurrent users is a business requirement. For some folks having an ACID compliant database is a business requirement.
Phoenix solves both of those meteor limitations.
Many of the folks discussing Phoenix in this thread have running meteor production applications and wish to solve those issues. They are in fact objective business and technology requirements.
Not at all, but you don’t have to read to far into the thread to see elements of both. Which is what I was referring to.
Scaling and DB are two great examples.
However…
Pulling numbers out of thin air doesn’t help your argument, especially after what you just wrote above.
It makes you look like you’re suffering from Maslow’s Instrument law.
Where’s your evidence Meteor peaks at 100 current users? Because you’re implying Meteor is not the correct tech solution for > 100 concurrent users. I don’t think I misinterpreted that, seeing as you explicitly say “Phoenix solves both of those meteor limitations.”
Ditto for ACID db. MongoDB may be the official db but there seem to be people using it with PostgreSQL, for example.
Also:
“This meant around 200 concurrent users on the site for a whole day, with peaks around 250. And I have to say the app held up just fine, with no noticeable slowdowns or issues.” +
“A large Meteor app like Telescope will comfortably scale up to at least 200-300 concurrent users, which is more traffic than being #1 on Product Hunt will get you (in other words, more than 99% of websites out there will ever receive at once).” http://www.telescopeapp.org/blog/screenings-launch-meteor-scaling-case-study/
Kadira has also done some heavy meteor scaling way beyond these levels afaik.
I get it that Phoenix/Elixir has been engineered for high concurrency.
Not surprising considering where it came from.
There’s plenty of use cases where using Meteor isn’t the best solution.
I’m sure the same also goes for Phoenix.
I fully agree with you on objective business and technology requirements.
Try understanding how oplog tailing works and it’s limitations. There are no arguments, just statements of fact. There are multiple threads on the topic.
I’m now the second person you have tried to troll in this thread. blinks
Your ignorance of the issue has only to do with your arguing from a place of ignorance, nothing else.
Applications with more than 100-200 concurrent users, such as Codefights with a claimed 690 max, has had to a) deploy across 10-20 meteor nodes and b) jump through hoops to eliminate oplog driven reactivity everywhere possible.
Phoenix can handle thousands on a single node using commodity hardware and Rethink or Redis pub/sub instead of oplog tailing. Hence the interest among meteor veterans.
Considering oplog tailing being a limiting factor on horizontal scaling is a well known issue, the only person you are trolling at this point is yourself.
So write the app in Meteor then when you hit #1 in product hunt hire geniuses to rewrite it in whatever. Pheonix, Go, C, Assembly, etc.
Facebook started out on PHP because it was simply a comfortable language. It could have been written with C even back then and would have started with better performance. None of that matters. It’s more important that you can add features and evolve your product quickly in the beginning. Once the time comes you could even invent your own framework to solve your specific problems.
In general I agree that you shouldn’t micro-optimize for things that may never happen. However, going too far in that direction can be dangerous. I naively thought that MDG would fix scaling before I hit issues. I’m only offering anecdotal evidence from my own experiences but here’s my take:
I had the same attitude when I thought that I would only have a few hundred concurrent users. However, business requirements changed before I even got the project launched and now I need to handle several thousand concurrent users from the start and within a year tens of thousands of concurrent users are reasonable (due to the customer driving their own traffic to the app).
This puts me in a pickle. It’s also the reason I made the meteor_elixir repo, as a spike to prevent re-writing the whole app before it’s even launched. It basically just cuts out subscriptions and delegates that to Phoenix while the rest is handled by Meteor.
Second use case:
An app built in Meteor. The data transfer is very minimal/optimized because of the mobile restraint. We thought we wouldn’t have a lot of traffic but alas, one day we had 400 concurrent users blasting the system. Crap I had to spin up 4-5 instances to keep it from crashing (5 keeps 404 errors to little/none). 5 - 1gb servers cost $285 on Modulus and I couldn’t host on DO at the time (mup didn’t exist).
In this case Meteor is able to keep up but it was not efficient and expensive. It was for a ‘free’ app so no revenues to pay for hosting (that’s another topic ).
I was able to reduce costs by cutting realtime on some features and using long polling every 30s (even though getting a new chat message instantly would be a better experience).
Phoenix is more efficient for these use cases and even though it’s more work upfront, over time it pays off. There’s no doubt that you can build an app faster with Meteor, there’s a tradeoff to be made. No free lunch.
@SkinnyGeek1010 I wonder if it makes sense to just build a wrapper in Node that emulates Meteor for Methods and Publications but speaks to a Phoenix layer instead of doing OpLog, Mongo, etc.
A framework built on Pheonix that fires up on the server and listens via API on which database entities to create, monitor, etc, and passes back the data to the Meteor server which then processes it and streams the data down the wire. If the bottle neck is OpLog processing this would solve the issue, no?
The winner of the hackathon made an app that pulls data on rest end points and deliver a ddp connection.
Here there is no pulling here. We are listening to channels between servers (Phoenix and Meteor) and returning a ddp websockets connection from Meteor on the server to client.
Can’t it be completely automated, requiring developer thinking strictly in JavaScript, with the proper compliment written on Phoenix?
While spiking out this solution I was trying to do something similar. One of the solutions was to subscribe to Phoenix on the meteor server in the publish, and use low-level methods to send it down the socket.
However, you’re opening yourself up to a myrid of race conditions and limiting how much one server can scale. Node is really great at stateless requests like a JSON API but as one of the video from earlier showed, it’s hard to scale stateful websocket connections.
It’s quite possible to serve a million concurrent users with one Phoenix server and a few Meteor servers (enough to basically log in the user and return their profile). Offloading the initial Meteor index.html to a CDN can free up Meteor resources (i’ve benchmarked ~67 pages a second on a DO 1gb box to just server the index page).
If you proxied Phoenix through node you’re meteor cluster count would skyrocket because you can only scale so high vertically with Node.
Erlang was built from the ground up to handle this so it makes sense to let it handle it. Ironically this makes the system much much more simple than trying to proxy it though node/meteor.
At the end of the day I decided that all the extra work doesn’t buy you anything and doesn’t make life easier. Both ways require a knowledge of enough Elixir and Phoenix to setup a channel and query data ( a bit less if you’re using Rethink as it will push new changes down).
On the meteor client side all you need to do is:
// we'll use a local collection to store incoming data
Chats = new Mongo.Collection(null);
channel.join()
// prime minimongo with last 20 chats on join
.receive("ok", resp => {
resp.initialChats.forEach(doc => {
Chats.insert(doc);
});
})
/// ...
channel.on("new_msg", doc => {
// upsert because messages we sent will result in duplicates if we insert
Chats.upsert(doc._id, doc);
})
The amount you’d have to learn is minimal and it’s far easier than trying to write your own reddis pubsub to scale livequery. Like I said, no free lunch
Thanks for the details. I still don’t see why not “cross that bridge when needed.” Like you said the server end of Meteor is often fairly straight forward. It’s great that Phoenix is there and it would be even better if there was a simple easy-breezy way to scaffold a Meteor-Phoenix project where you write your publications and methods on Phoenix and subscribe in Meteor using the same /Client /Server folder structure and then when you “Foobar Deploy” it does what it needs to do.
I don’t mind learning Erlang at all, I’m sure its a fine language.
Is there a sample project with Phoenix as the backend and Meteor on the front with instructions on how to get the Phoenix part up and running on Mac? I’d give that a try for sure!
[quote=“Babak, post:273, topic:13519”]
Thanks for the details. I still don’t see why not “cross that bridge when needed.”[/quote]
Most meteor projects will never get to that bridge. If they do, it’s not a realistic expectation to monetize those few hundred users and hire ‘geniuses’ to rewrite your app from scratch in the few short days before performance complaints on twitter kill it…
Two backends simultaneously is somewhat silly. Choose one, use occam’s razor on the other. Skinnygeek had to try it as his project was too far along.
If you chose node because of skillset limits, use one of the myriad of npm libraries that integrate redis/postgres/rethink pub/sub feeding reactivity through Redux/React on the client.
That path will let you at least scale horizontally into the thousands before it explodes as you need to share state between nodes through socket.io. This limitation was discussed in the video Skinny mentioned.
For brand new apps where scalability is a requirement, Phoenix on the backend and Redux/React on the frontend is the path of least resistance/time/money/complexity.
A lot of the time waiting until you get there is just fine. I just chose wrong. Twice. For most people the efficiency/cost will creep up faster than no service. That’s an easier path to have as migrating would only cost more $ and not losing users. I guess my main point was to plan ahead and leave enough buffer for the unexpected (that was my problem).
and it would be even better if there was a simple easy-breezy way to scaffold a Meteor-Phoenix project where you write your publications and methods on Phoenix and subscribe in Meteor using the same /Client /Server folder structure and then when you “Foobar Deploy” it does what it needs to do.
You can almost do this. Meteor doesn’t have any scaffolding generation but it’s very fast in Phoenix (with passing tests!). Once you learn how it works you can wire the two together in less a couple of mins. Deploying is as simple as a heroku push… it’s actually easier than Meteor to deploy (on Heroku, live code patching with Exrm is a bit tricky at first).
Is there a sample project with Phoenix as the backend and Meteor on the front with instructions on how to get the Phoenix part up and running on Mac? I’d give that a try for sure!
If you use the meteor_elixir repo above you could pull that down and run mix deps.get to pull in packages and then mix phoenix.server to get it running. However, since Elixir is not installed yet that can be installed here. It also required mongo to be running in the background mongod. Once it’s running then you can meteor run and the chatroom should be working .
However, it may be easier to run through the up and running guide first as this will get a hello world app running (albeit with Postgres by default).
Why do you sound so condescending? Do you have a human communication ‘limit’ or what? Re-read your comment and see if it sounds convincing, if it does, stop talking to me.
Reddit itself has pretty much crashed a few times, people still use it. Twitter too. I’ve had Reddit crash my server before. No big deal, just scaled up and that was that.
Facebook was written with PHP because it was convenient. Github could have been written with in C++, but it was written with Rails. Instagram started with Django. So what?
Being lazy can be a programmer’s virtue, this is one reason why getting start with Meteor was wisely made easy. I’m not going to go jump through hoops to learn Phoenix especially with people like you trying to promote it.
Maybe you guys should streamline your presentation and get the elevator pitch in order and stop compensating for that marketing ‘limit’ by attacking people who actually warm up to the idea of trying your Meteor forum spam product.
).
.