Meteor Galaxy Auto-Scaling Package

announce
1

I’ve released the first version of avariodev:galaxy-autoscale which allows a Meteor app to read current stats from Meteor Galaxy and adjust the number of running containers up or down based on loading.

Since there is no API to use, it relies on phantomjs and webdriverio to connect to Meteor Galaxy, which is far from an ideal solution, but AFAIK it’s the only one available currently.

I’m running it in production and it’s been great so far, my app has been going up and down between 1-4 containers based on the number of connections.

Putting in your project is as simple as adding the package and including this in a server only file:

import { GalaxyAutoScale } from 'meteor/avariodev:galaxy-autoscale';

Meteor.startup(() => {
  GalaxyAutoScale.config({
    appName: Meteor.settings.galaxy.appName,
    username: Meteor.settings.galaxy.username,
    password: Meteor.settings.galaxy.password,
    scalingRules: {
      containersMin: 1,
      containersMax: 3,
      connectionsPerContainerMax: 80,
      connectionsPerContainerMin: 40,
    },
  });

  GalaxyAutoScale.addSyncedCronJob();

  GalaxyAutoScale.start();
});

Screenshot of auto-scaler in action:

auto-scale logs
Auto-Scale Logs1758×550

I currently support a course tutoring site, and I was getting frustrated that the day before an exam my traffic would jump 300% and the only way I would find out is a text message from my client that the site was running slowly. And all of that so I could pull up my phone, click a link, and hit a single button. It was so easy that I figured a script could do it, so why not write one?

Since this package is based on interacting with the Galaxy UI, it’s rather brittle, but if there is enough interest in using this I will add some alert functionality so at least some notifications can go out if things go wrong.

License is MIT. Enjoy!

31 Likes
2

ha, nice solution. Such a shame Galaxy is unwilling to create a simple API to add/remove containers and for performance metrics. Then these kinds of auto-scaling solutions would be a snap to create.

bump:

4 Likes
3

This is really cool. I currently have myself pegged at two x quad cores. Most of the time they’re only using ~0.5 ECU, but occasionally pop up to 2 ECU for ~5 minutes, then drop down (I think it has to do with synced cron scripts that I’m running). Sorry if I’m missing this, but is this package able to determine if ECU is maxed out and add another container based on that?

4

I think MDG really needs to implement automatic scaling along with 2 factor-authentication into galaxy sooner then later, the demand can’t be more clear then that, and customers are very vocal about it.

Thanks for sharing the package, interesting automation :slight_smile:

8 Likes
5

The package determines your current percentage of ECU used to available, but I currently don’t use this is a scaling factor since it changes so often that a snapshot of it doesn’t make a great datapoint for scaling decisions.

This package won’t really help in your case for a few reasons:

  • The ‘refresh’ rate is currently set at 15 minutes, though that is configurable.

  • Spinning up a new container probably won’t help much with responsiveness if you try to do it within a 5 minute period that existing containers are CPU pegged, considering Galaxy runs with sticky sessions and adding a new container will help with future connections.

  • There is a non-trivial CPU and memory penalty when the script runs, since it has to spin up a phantomjs program to interact with the Galaxy management page. You don’t want to be doing this continuously unless you set aside a separate server to do this.

6

Bottom line that MDG is a software company and not an infrastructure company. They only run Galaxy because the VC’s want to see a ROI. But it’s clear that to me that this isn’t going in the right direction. As cool, easy to use and stable Meteor is, and I welcome the continuous development of it, I don’t think that long-term Galaxy is the right provider for us.

Not only cost wise but like others we are very US centric from our customer profile and that means that their current policy of not offering auto-scaling is only favorable to them, makes them expensive.

So thanks @avariodev for coming up with this package, it’s a bit aluminum foil & duct tape and it would be great if MDG would at least offer some easy API to avoid the PhantomJS. BTW, you might want to try Nightmarejs package instead, we switched over.

1 Like
7

BTW, you might want to try Nightmarejs package instead, we switched over.

I recently moved from PhantomJS to Puppeteer, it’s an API to control a Chrome headless browser built by the Google Chrome team. I definitely recommend it! https://github.com/GoogleChrome/puppeteer

8

Thanks for the tips, I went with PhantomJS since I see it used everywhere, but it was actually a big PITA to get installed and working properly in all cases. There was some inconsistent error when Meteor tried to install it as part of its package system using Npm.depends, so I had switch to adding it as a peer dependency.

Next time around I’ll definitely take a closer look at some of the newer frameworks. I like how all the examples for Puppeteer use async/await, that’s how I structured my crawling code :smile:. So much nicer than callbacks!

9

Do you really need to use PhantomJS? Just curiosity, because Galaxy is a Meteor app, and as a Meteor app, it has methods.

For example, they have this method:

  • apps/updateContainerCount

And they have publications too.
So, wouldn’t be possible to connect remotely to this meteor app and do what you have to do (add containers, etc)?

10

When you factor time spent with dev ops vs Galaxy I think it may be worth it.

They keep adding some interesting features, specially for professional tier.

As far as I know they are working in Alerts “now”.

If you want an Azure type of solution, I think Galaxy will never match it. But even for professional deployments it’s a great tool.

PS: It has options of servers in Europe and Australia too. Not sure if I understood your “US centric” thing.

1 Like
11

I thought about connecting directly into the Meteor, I wasn’t sure how authentication would work, and the initial rough solution using PhantomJS only took a few hours to get up and running.

I agree there are much better ways to do this, but I didn’t really want to invest a ton of time into making a polished band-aid when hopefully MDG will provide something that makes all of this work pointless.

If it turns out this band-aid ends up getting used more than it really should, then I’ll definitely look into a less brittle solution.

2 Likes
12

Will there be any official/MDG tool for auto-scaling? Does anybody know? Is there anything on the roadmap?

1 Like
13

I’m a bit puzzled by the lack of presence of the official galaxy representation in this thread and the Meteor forums in general. You would think they would be more active here since it’s them main source of revenue for MDG and most of their potential clients are hanging around these threads.

Maybe I’m missing something here, but I’m really puzzled by this, I think it’s better for Meteor and the MDG business that they’ve a close eye on this community.

1 Like
14

Dear all,

please keep in mind that MDG has to make money, also VCs have had an interest like this when they invested.
I wonder why you have an issue with this. Meteor is free and should be in the future. So I am proud to pay for Galaxy and even ordered the Pro package without having a need for it. :raising_hand_man:

If I have issues with server load, I have customers - that’s the point. :nerd_face:
Please ask yourself, where you business would be if you had no Meteor.

I hope Meteor has a great future and I would even pay more to make that sure. :smiley:

Don’t get me wrong, I understand you - I just wanted to point out the business point of view.

Cheers

2 Likes
15

That is great, and I share your sentiment and goal in making MDG even more profitable and Meteor better. All I’m saying that for that goal to happen, I personally think that they need to pay closer attention to threads like this.

1 Like
16

Amen to this. After all, this is the official forum, right?

1 Like
17

This thread was also featured on the regular forum update email, so you’d think that some of the key Meteor players like @sashko would’ve seen it and passed along to the Galaxy team.

On topic, thanks for writing this up @avariodev! The Meteor/Galaxy ecosystem definitely needs something like this.
Autoscaling is the sort of “Set it and forget it” infrastructure functionality that originally drew me to Galaxy.

18

I’ve been asking for auto-scaling to their support for one year. Last time (few months ago) they said they were focusing in Alerts and no ETA for auto-scaling nor 2-factor auth.

19

For enterprise class applications not having 2-factor authorization, alerts, and auto-scaling is really a big problem. We may be forced to either go back to Heroku or invest more in our dev ops and build our own system on AWS. I’ve communicated this to MDG multiple times with no real feedback except, nope!

2 Likes
20

Random plug, I’ve added a super simple alert function for my scraper plugin.

I added it mainly for my own deployment, but I tried to make to code somewhat general so people can easily customize if desired.