Continuous Deployment on Gitlab

help
1

I am currently trying to set up continuous deployment on Gitlab so that whenever I merge into develop, it will automatically bundle and deploy to my dev server using mup. I am able to run mup from my local machine and deploy to it, but I’ve been having difficulty getting it to run on the CI/CD.

The first problem I ran into was that there seems to be a bug with CoreOS where the tar command for installing meteor with curl https://install.meteor.com/ | sh fails.

So I decided to install meteor via the repo:

  git clone git://github.com/meteor/meteor.git ~/meteor
  cd ~/meteor
  # 1.4.3.1
  git checkout 954333b8136835de5933b46aa08400f0f73538ed
  git submodule foreach 'git fetch && git fetch --tags'
  git submodule update --init --recursive
  cd -

So, that works, but then when I try to run mup, I get unable to install fibers. So I decided to try to install fibers on the CI:

    # Try to install fibers
    - apt-get update
    - apt-get install --yes g++ build-essential

    # Install mup
    - npm install -g mup

    # Install meteor if it does not exist
    - export PATH="$HOME/meteor:$PATH"
    - bash ./.deploy/install-meteor.sh
    - meteor npm install

Now when the CI runs, it gets to meteor npm install, and fails with:


> fibers@1.0.15 install /builds/memosa/memosa/node_modules/fibers
> node build.js || nodejs build.js

internal/child_process.js:313
    throw errnoException(err, 'spawn');
    ^

Error: spawn EACCES
    at exports._errnoException (util.js:1026:11)
    at ChildProcess.spawn (internal/child_process.js:313:11)
    at Object.exports.spawn (child_process.js:385:9)
    at build (/builds/memosa/memosa/node_modules/fibers/build.js:57:5)
    at Object.<anonymous> (/builds/memosa/memosa/node_modules/fibers/build.js:46:3)
    at Module._compile (module.js:573:32)
    at Object.Module._extensions..js (module.js:582:10)
    at Module.load (module.js:490:32)
    at tryModuleLoad (module.js:449:12)
    at Function.Module._load (module.js:441:3)
internal/child_process.js:313
    throw errnoException(err, 'spawn');
    ^

At this point, I’ve run out of ideas. Any thoughts or configuration help would be much appreciated!

Thanks!

2

Related: CI and CD using Gitlab as version control and Gitlab CI for continuous build,deployment and test

Unfortunately, the response seems to be a really hacky/good workaround to the main problem of installing Meteor on Gitlab’s CI/CD runner.

3

It looks like this worked for me:

https://acrofever.com/blog/meteor-1-4-testing-pipeline-gitlab-ci.html

I ended up using the image: christiankiely/meteord:test to get it set up.

4

Scratch that, I get the following error when trying access the site:

Error: Cannot find module '../'
    at Function.Module._resolveFilename (module.js:325:15)
    at Function.Module._load (module.js:276:25)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/bundle/bundle/programs/server/npm/node_modules/bcrypt/node_modules/.bin/node-pre-gyp:15:20)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)
5

Any ideas? Nope-Gyp successfully installs from the source on the ci server.

I’m going to take a different approach in the meantime and have the ci trigger the server to build the app itself. I’m having problems on that end as well though. Seems like it might be running out of memory when trying to build?

6

we got it running with this image (made a friend of mine): https://hub.docker.com/r/blurri/meteor-node/

we build the tar with meteor build and push it to the staging-server where it gets unpacked und node restarted

this is the gitlab-ci.yml (work in progress)


image: blurri/meteor-node
variables:
    METEOR_ALLOW_SUPERUSER: "true"
stages:
  - test
  - build
  - deploy
test:
  stage: test
  script: ./test.sh
build:
  stage: build
  script: ./build-staging.sh
  artifacts:
    paths:
    - build
  only:
  - develop
deploy_staging:
  stage: deploy
  script:
    - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
    - eval $(ssh-agent -s)
    - ssh-add <(echo "$SSH_PRIVATE_KEY")
    - mkdir -p ~/.ssh
    - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
    - ./deploy-staging.sh
  environment:
    name: staging
  only:
  - develop

build-staging.sh ist just a small script

cd app
meteor npm install
meteor build --server https://velomittwoch-staging.panter.biz ../build
cd ..

(app is the folder within the git project where the meteor project is)

deploy-staging.sh is also just a small script which pushes the build and executes a remote command:

scp build/app.tar.gz app@foo.example.com:
sshapp@foo.example.com /home/app/extract_and_restart.sh

problem is that the gitlab server needs a private key or password to push it to the remote server. we use a private variable for that and this odd script (under deploy_staging)

maybe this helps!

1 Like
7

Nice, I ended up getting it to work with your docker image. I’ll post the script shortly, just to make sure it works without all the extra cruft I had on it, but basically, I was able to get my Meteor app to get bundled correctly (albeit I had to explicitly add bcrypt to finally get it to deploy without any missing dependencies). So bundled and deployed to a Digital Ocean droplet with Mup on Gitlab CI!

Thanks!

8

Here’s what I ended up with:

image: blurri/meteor-node

variables:
    METEOR_ALLOW_SUPERUSER: "true"

services:
  - mongo

cache:
  paths:
    - "node_modules"
    - "~/.npm"
    - "~/.meteor"
    - "~/meteor"
    - ".yarn"

stages:
  - test
  - deploy

# Deploy to Dev Servers
deploy_dev:
  only:
    - develop
  stage: deploy
  before_script:
    - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
    - eval $(ssh-agent -s)
    - ssh-add <(echo "$SSH_PRIVATE_KEY")
    - mkdir -p ~/.ssh
    - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

    - apt-get update
    - apt-get install build-essential
    - npm install -g node-gyp
    - npm install --production
  script:
    - cd ./.deploy-dev
    - ../node_modules/.bin/mup setup
    - ../node_modules/.bin/mup deploy
9

did caching work for you? My builds where very slow because it reinstalled node_modules on every stage.

I also had recently a problem with docker. on every second build i get:

Running with gitlab-ci-multi-runner 1.10.4 (b32125f)
Using Docker executor with image blurri/meteor-node …
ERROR: Preparation failed: cannot connect to Docker endpoint

1 Like
10

Caching seems to work between builds, but not for stages. Artifacts are supposed to help between stages, but I opted to just have the deploy stage reinstall every time so I wouldn’t have to worry about it. Not ideal, but I’m sure I can improve this in the future.

A simple repo structure with the .gitlab-ci.yml that I am using.

I’m not getting any docker issues yet?

1 Like
11

@idmontie: Its been a while since your post. Are you still using Gitlab? If yes, could you help me out at The BIGASS self-hosted CONTINUOUS INTEGRATION comparison (Jenkins vs. GitLab CI vs. Buildbot vs. Drone vs. Concourse vs. …)?