Meteor in Production - A Scaling Success Story

I understand. It’s only that as soon as I hear MUP for a production app I sort of take a step back.

And, out of curiosity, managing the cluster of DO machines, with High Availabilty and the rest? Load balancing? Kadira’s solutions, while very clever hacks, were just that, hacks.

2K in monthly costs is expensive, no doubt.

Well now we’re on AWS. But Digitalocean worked just fine too.

We use nginx for load balancing. And there isn’t really a solution apart
from Kadira (and galaxy uses it), so not sure what you were asking there.

What are your issues with mup?

What do other solutions have that are better?

MUP is fine in it’s current guise, it went unsupported for a while (afaik) after arunoda left the community but now it’s well supported again. There are a lot of commits and contributors.

I think it depends a lot on your app and use-case as to what is the best deployment method. However I still stand by Galaxy as a good solution for a lot of people @elie. I’m in the same boat as you regarding cost but there’s no denying the ease of setting up an app on Galaxy.

I needed to go and have a look at what the current state of MUP is. I must agree with @korus90 that in this incarnation it looks better. However, the reverse proxy still seems to be experimental.

I mean, if you are already using a solid load balancing and reverse proxy solution, such as Nginx, what’s MUP adding besides Docker, that PM2 or Phusion Passenger aren’t?

And yes, I stand by what I said about Kadira’s stuff being advanced hacks. Cluster and early MUP are examples of that.

1 Like

So you don’t use kadira? What do you use? That’s what galaxy does.

It seems like you havent used mup from the comments. But I can tell you
it’s production ready and has been for years. There was a time it wasnt
being maintained but it is at full force again.

What it gives you is one line mup deploy. Restarting server upon failure.
Users up everything for you. Not too much it doesn’t give you nowadays.
Although some of the bleeding edge features are still in beta.

If you’re referring to the diagnostics app that used to be on kadira.io, and which Meteor is using for optics, yes, we did while it was run by Kadira. But that’s something that has been built for production from the outset. We’d use that again if we needed to, but we did not have any issues with speed. Yet.

I only referred to Kadira’s stuff for deployment and management of Meteor apps.

My original question was because I still have to understand how MUP can help, out of the box, with deploying in two or more availability zones, at the same time with managing HA load balancers (that is, with your 20 machines, you would need 6 Nginx LB instances or more - at least 2 for each zone, plus 2 public ones). My feeling was that, if you rely only on MUP, that doesn’t sound like a mission critical app.

In my opinion the problem is a bit more complex than just spinning up 20 $5 instances and put a list of servers in the MUP config file. Even spinning up servers requires a certain level of automation for which you will be using an API from your cloud provider.

All that stuff requires maintenance, and as such it incurs costs.

I think you need to bring in more information to make the comparison with the OP’s experience fairer.

PS. I have only respect for people dedicating their time to Open Source projects, and it’s great to see projects like MUP taking up new life. This is not a criticism of MUP, but I’m afraid nowadays you can’t say that reverse proxy / load balancing / SSL are ‘bleeding edge’. At least with respect to load balancing, tools like PM2 had it baked in from a very early stage.

Thanks for the post @avariodev and the great plugin. Personally, I use Galaxy for customer-facing services and then DO for backend cron jobs. Has worked really well to offload crunching to cheaper resources, but ensure our users have the great, consistent experience delivered via Galaxy (knock on wood).

@illustreets: re: MUP I’ll say it’s gotten much, much stronger over the past few months since the new maintainers took it over.

One question since we’re talking Meteor: I love using it for the developer experience, regardless of the real-time. If you didn’t need real-time, but wanted a similar quick, integrated developer experience, where would you go?

It looks like it, and it’s very heartening to see that. We used it long ago, when Galaxy wasn’t even around. It was buggy as hell, and we switched to Ansible + PM2 + Nginx. Rock solid, without a single issue in tens, maybe hundreds, of repeat deployments.

1 Like

I’d actually still use Meteor if it was a node app. Turn off reactivity. With the upcoming Meteor 1.6 and mongoDB 3.6 updates there’s a lot to be said for Meteor as a general all-purpose node framework right now.

1 Like

Even if you don’t use MUP, all of that is a one time setup isn’t it? And if you are using docker I’d definitely recommend mup simply due to less work involved. I also use pm2 but without docker. With docker it doesn’t make sense to use pm2 (which is there to restart processes, something docker does natively).

Even with mission critical apps, what does going with Galaxy get you besides less admin and a load balancer? It doesn’t do autoscaling or have mongo. Obviously it makes things easier and is the official solution, but it does seem so expensive, compared to even all in one solutions like nodechef.

Galaxy makes sense if your app is at a scale where you have multiple AZs in AWS. But if you are at that point, you likely have devops and any of the alternate solutions are not that hard.

MUP was designed to get people off the ground. I still use it (we only have ~5 servers) and there’s still a lot of value in it.

I’m not saying there isn’t value in using MUP. Having browsed the repo today it seems like a great solution for people wanting to run a meteor app on their own server.

The advantage I see for Galaxy is the ease of anyone getting an app running really quickly. As I’m sure you’ve noticed on these forums, not everyone is a full-time developer with lots of experience. Some people are just playing around or have an idea where they want to see if it works in practice. I think Galaxy offers a good solution to those people.

I also think if your app doesn’t require a ton of resources or fills a specific niche (and therefore doesn’t get much traffic) then Galaxy would also be a good option. I agree with @elie that when an app is quite large or generates a lot of traffic then Galaxy might be too expensive though.

I use pm2 because I’ve found it to be very reliable and I was using it while MUP was flaky at best. It now looks like a more solid option but overall I prefer to roll my own solution for my own specific needs.

It’s called oplog tailing not trailing.
https://www.mongodb.com/blog/post/tailing-mongodb-oplog-sharded-clusters

Good post and real-life study. Thanks for sharing! I hope we’ll see auto-scaling built-in with galaxy soon…

Wow, thanks for the correction, I feel a bit silly about that :laughing:.

Glad to hear the post was worth reading.

FWIW http://nodechef.com/ offers a Kadira host, we’ve been using it now. Works fine.

1 Like

We don’t have instances in multiple zones. All our instances reside in the
same zone with nginx pointing to 20 different ips.

And to be honest we’ve never seen things go wrong from not being in enough
zones. I don’t think this is something Galaxy provides either.

They have HA if you’re on a higher tier, but not across zones.

The irony is that some of the folks who built successful revenue generating businesses on top of Meteor and are currently spinning many instances without giving back a dollar to MDG will be screaming Meteor is abandoned when there’re no funds to hire and retain the very smart, specialized and skilled developers required to manage such a complex, large and specialized Open Source project, we’ve seen that happen less than two years ago.

But I hope threads like this will contribute to making Galaxy better, I’m sure also those folks have legitimate reasons to be off Galaxy. In my mind, if someone built successful Meteor project and contemplating hosting somewhere else, then Galaxy has room for improvements.

Again thanks for sharing that great story with Meteor and Galaxy :slight_smile:

3 Likes

I agree with what you say but Galaxy isn’t appropriate for a heavy workload application with more than 100 users (in my opinion). It’s not cost-effective enough.

But as I’ve said a few times above it definitely has its place. I would support MDG and put my apps on Galaxy but I can see it costing far more to do so which from a business point of view isn’t a good move.

2 Likes

It seems that the two main complaints about Galaxy are the pricing and the auto-scaling feature, and I think those could be easily improved. With regards to pricing I think they could also do better for businesses that is just starting up by matching DO pricing.

I hope some folks from MDG Galaxy are monitoring those threads, in theory MDG should retain most of Meteor projects at any stage since they know their tool and their audience more then anyone else.

2 Likes