Deploying to AWS with Meteor 1.3.5 still using MUP

aadams · July 30, 2016, 8:49pm

I’ve been using arunoda’s Mup for deployment to AWS EC2 instance for a while now. I updated to Meteor 1.2.2 and Mup still worked, all I needed to do was bump the nodeVersion to 0.10.40.

Just today I updated to Meteor 1.3.5.1 and tried to deploy using the same mup settings file, but it didn’t work. I only thing I did to the settings file was bump it again to 0.10.43.

Setup works fine:

{
  // Server authentication info
  "servers": [
    {
      "host": "subdomain.mysite.com",
      "username": "ubuntu",
      // or pem file (ssh based authentication)
      "pem": "/Users/myname/Meteors/mypem.pem"
    }
  ],

  // this only works with mupx, not this mup
  "buildOptions": {
    // build with the debug mode on
    // You can deploy in debug mode by passing --debug.
    // This will leave your source code readable by your favorite in-browser debugger, just like it is in local development mode
    // :: Think Meteor Toys!
    "debug": true,

    // executable used to build the meteor project
    // you can set a local repo path if needed
    "executable": "meteor"
  },

  // Install MongoDB in the server, does not destroy local MongoDB on future setup
  "setupMongo": true,

  // WARNING: Node.js is required! Only skip if you already have Node.js installed on server.
  "setupNode": true,

  // WARNING: If nodeVersion omitted will setup 0.10.33 by default. Do not use v, only version number.
  "nodeVersion": "0.10.43", 

  // Install PhantomJS in the server
  "setupPhantom": false,

  // Application name (No spaces)
  "appName": "subdomain-mysite",

  // Location of app (local directory)
  "app": "/Users/myname/Meteors/clients-workstation",

  // Configure environment
  "env": {
    "ROOT_URL": "http://subdomain.mysite.com" // change to https
  },

  /*"ssl": {
    "pem": "./ssl.pem"
  },*/

  // Meteor Up checks if the app comes online just after the deployment
  // before mup checks that, it will wait for no. of seconds configured below
  "deployCheckWaitTime": 60
}

But on mup deploy I get errors:

Uploading bundle
Uploading bundle: SUCCESS
Setting up Environment Variables
Setting up Environment Variables: SUCCESS
Invoking deployment process
x Invoking deployment process: FAILED

-----------------------------------STDERR-----------------------------------
 please make sure you have setup your
build environment--
Windows + OS X instructions here: https://github.com/nodejs/node-gyp
Ubuntu users please run: `sudo apt-get install g++`
Alpine users please run: `sudo apk add python make g++`
sh: 1: nodejs: not found

npm ERR! fibers@1.0.13 install: `node build.js || nodejs build.js`
npm ERR! Exit status 127
npm ERR!
npm ERR! Failed at the fibers@1.0.13 install script.
npm ERR! This is most likely a problem with the fibers package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node build.js || nodejs build.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls fibers
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 3.13.0-36-generic
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "install"
npm ERR! cwd /opt/formspoint-clients/tmp/bundle/programs/server
npm ERR! node -v v0.10.43
npm ERR! npm -v 1.4.29
npm ERR! code ELIFECYCLE
npm ERR! not ok code 0
-----------------------------------STDOUT-----------------------------------
 > ./bcrypt: npm install due to binary npm modules
bindings@1.2.1 node_modules/bindings

nan@2.2.1 node_modules/nan

nodeunit@0.9.1 node_modules/nodeunit
└── tap@0.7.1 (inherits@2.0.1, buffer-equal@0.0.2, slide@1.1.6, deep-equal@1.0.1, yamlish@0.0.7, nopt@3.0.6, mkdirp@0.5.1, difflet@0.2.6, glob@4.5.3, runforcover@0.0.2)
make: Entering directory `/opt/formspoint-clients/tmp/bundle/programs/server/npm/node_modules/meteor/npm-bcrypt/node_modules/bcrypt/build'
  CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o
  CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o
  CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt_node.o
  SOLINK_MODULE(target) Release/obj.target/bcrypt_lib.node
  COPY Release/bcrypt_lib.node
make: Leaving directory `/opt/subdomain-mysite/tmp/bundle/programs/server/npm/node_modules/meteor/npm-bcrypt/node_modules/bcrypt/build'

> fibers@1.0.13 install /opt/subdomain-mysite/tmp/bundle/programs/server/node_modules/fibers
> node build.js || nodejs build.js

----------------------------------------------------------------------------

I tried reinstalling bcrypt, npm and nothing works.

The mup logs -f looks like so:

at require (module.js:380:17)
at bindings (/opt/my-application/app/programs/server/npm/node_modules/meteor/npm-bcrypt/node_modules/bcrypt/node_modules/bindings/bindings.js:76:44)
at Object. (/opt/my-application/app/programs/server/npm/node_modules/meteor/npm-bcrypt/node_modules/bcrypt/bcrypt.js:3:35)
at Module._compile (module.js:456:26)
at Object.Module._extensions…js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
error: Forever detected script exited with code: 8

I then tried the M4v3R fork

npm remove -g mup
git clone https://github.com/M4v3R/meteor-up
cd meteor-up
npm install -g
npm link

Same results.

What happened to the Meteor deployment story? What is everyone using these days to deploy? Where is everyone deploying their applications to?

Any help is appreciated.

gsabran · July 30, 2016, 10:09pm

Are you using kadirahq or arunoda’s version of mup?

I’ve also had issues with bcrypt and meteor a number of times (it was mainly when using bcrypt from within my app, which sounds different than what you’re getting.) I think that installing with meteor npm install instead of npm install solved my issues, but not sure they’re the same problems…

vigorwebsolutions · July 30, 2016, 10:15pm

I know it’s largely based on anecdotal/not very sound advice, but before pulling my hair out, I always up the deployCheckWaitTime.

Also, I’ve run into situations where a 120 wait deployed while a 90 threw errors, so I usually set it to like 600, walk away and come back, and if the deployment has still failed, I can at least rule that out.

aadams · July 30, 2016, 11:19pm

I’m on arunoda’s version: https://github.com/arunoda/meteor-up

Thanks for your help @gsabran.

aadams · July 30, 2016, 11:20pm

Thanks @vigorwebsolutions I’ll give that a try.

aadams · July 30, 2016, 11:25pm

Will you walk me through what this would look like?

aadams · July 30, 2016, 11:54pm

I just tried something, I rolled back to Meteor 1.2.2, changed the node version in mup settings to 0.10.40, did a mup setup/mup deploy and the deployment works fine.

Switch to Meteor 1.3.5.1 and node 0.10.43 and it has issues. What is one to do for deployments these days? This shouldn’t be that hard right? Something like this shouldn’t be an obstacle to upgrading software.

aadams · July 31, 2016, 12:17am

There are so many forks of arunoda’s mup, it’s hard to find one that might have a real fix for 1.3.

I found: https://github.com/thomasspiesser/meteor-up and https://github.com/JohnBernardsson/meteor-up both have updates for 1.3 apparently.

Then there’s https://github.com/kadirahq/meteor-up and https://github.com/M4v3R/meteor-up. Where to start…

knana · July 31, 2016, 1:12am

@aadams if you are open to trying other solutions, you could certainly consider NodeChef; the easiest way to deploy Meteor 1.3 and 1.4 apps

aadams · July 31, 2016, 1:24am

I’m interested, but I have a requirement, where I need to have pdftk installed on the local box. It’s a standard install, where I ssh into the EC2 instance and do a sudo apt-get install pdftk. I’ll eventually want SSL support too. I need to be able to have a relatively static IP that I’ll point a domain of my choosing to. Also, I use MongoChef for database access (ssh into it) as well.

If all this is possible, and you can provide step by step instructions, and it works of course, I’m open to giving it a try for a month or so as a trial run.

perumalkuk · July 31, 2016, 1:32am

I am using MUP with 1.3.4.1 and it works. You will find in the forum discussion about it. I used one of the forks of MUP which has architecture flag.
My suggestion is to look at the npm logs on the server under the server directory in your app. You will get clues.I think it is to do with the version of node-gyp used by MUP. Check your version of node-gyp on server ( mine is 3.4.0). I think MUP uses a old version.

aadams · July 31, 2016, 1:44am

thank you.

Do you mind sharing which fork you used?

you mean under /opt/my-app-name/app/server ? What are they named?

So this is what needs to be changed in the Mup source?

knana · July 31, 2016, 1:46am

There is no ssh access on NodeChef but we can certainly install pdftk in your app container. For MongoDB access, we provide a connection string so you can access your database with MongoChef. And the free trial period is for 7 days.

perumalkuk · July 31, 2016, 2:03am

please refer to Meteor 1.3 deploy Error with MUP

I used fork https://github.com/M4v3R/meteor-up Please make sure you follow the steps for reinstall if you are using old MUP.

When MUP is showing the errors, it is using the logs stored on the server. Just look at those logs (not sure about the path …but it should be under /opt/yourapp)… specially lines above the one shown through MUP. I think you have to delete old node-gyp in your home directory, on server, to which MUP refers during installation.Post that it runs happily.

ramez · July 31, 2016, 4:02am

@aadams and anyone else using MUP and MUPX, we walked away from MUP and couldn’t be happier. Depending on a what I believe to be a hacky package to deploy your Meteor instance is risky, and also inefficient.

Here is what we did:

Shell build script that compiles Meteor into the compressed tar, then pushes onto our servers with ssh, then calls local compile and update script
The local update script restarts pm2 (running n - 1 instances of Meteor, where n is the number of cores) – see # 6 below
We use tengine (instead of nginx) which includes session-based load balancing out of the box
We need mongo on the same maching, so we reserve a core for it (hence the n - 1) – but you can use a hosted DB service if you wanted to
We use icinga2 (you could use Nagios) to check that the instances are up and that the URL is responding
When restarting the n - 1 instances, we wait 5s between each instance restart to give the chance to the load balancer to transfer traffic over to another instance so we maintain VHA

You should REALLY look into using Galaxy. Takes away so much of your pain (we wanted to, but our real-time app needed Mongo close by)

vigorwebsolutions · July 31, 2016, 4:14am

If it helps:

Local Box:
mupx 1.5.3 (no fork)

Server:
Ubuntu 14.04
Node 0.10.43

Meteor 1.3.4.1

babrahams · July 31, 2016, 1:46pm

My most successful, long-term deployment strategy has been to use a slightly hacked* version of https://github.com/julien-c/meteoric.sh to deploy iterations of an app on a Debian box starting three years ago when Meteor was at 0.5.7. All I have to do is type meteoric deploy and the four app instances are updated and running about a minute. I have haproxy or nginx load balancing (I’m using this setup for a couple of apps) and also have a 1 member replica set running on the same box. (I don’t use the meteoric setup command – I set up node, mongo, etc., manually.) This is still working with 1.3.5.1, and should continue to work with 1.4 once I’ve updated node, mongo, etc.

* so I can set environment variables in the deploy config file

I’ve had other apps that were running on DO and other on-premise Ubuntu boxes using mup and mupx, but haven’t had much success with either of those since Meteor 1.2. In some cases, I still mupx deploy apps just because it’s an easy one-liner to get a (not working) build on the production server. Then I run a few commands on the server to clean up the build (i.e. remove and install bcrypt and fibers) and start the app instances. It’s a bit annoying, but it works.

In the end, I had to give up on mup and mupx – I don’t think I’ve actually been able to successfully mup deploy or mupx deploy anything since 1.2.

aadams · July 31, 2016, 5:49pm

Thanks @perumalkuk

I saw this post, and tried a few things within, but maybe I missed something – I’ll try again.

I tried this fork. But I’ll clone it and try again. [quote=“perumalkuk, post:14, topic:27306”]
When MUP is showing the errors, it is using the logs stored on the server. Just look at those logs (not sure about the path …but it should be under /opt/yourapp)… specially lines above the one shown through MUP.
[/quote]

I never looked at the logs on the server, just the error output. I’ll have a look at them.

Thanks for this advice – I’ll give that a try.

aadams · July 31, 2016, 5:55pm

@ramez this is a great idea! I’ve always wondered why I never heard of anyone doing this before…

I’m using AWS EC2 and have ssh access. Do you think I could use a script similar to yours to deploy there?

So far, I haven’t had the need for load balancing. But I’ll look into this tech as I grow for sure.

This is exactly what I’m doing – I have mongo on the same EC2 instance.

I’m in the same boat as you, we need mongo close.

Do you have a sample to help me and others get started with this approach?

ramez · July 31, 2016, 6:45pm

Hey @aadams, will start sanitizing the scripts, add some explanations and pushing onto GitHub repo - ETA tomorrow - Monday