Continious Integration with Meteor

svenskunganka · November 3, 2015, 5:34pm

Hey fellow Meteorites!

I’ve been tasked with writing a post of my CI workflow with Meteor from the guys over here.

Introduction

CI (Continious Integration) is something all developers should strive to utilize, as it does not only make your life as a developer a whole lot easier and predictable, but also takes away some of the huge headaches with having to deal with deployments. It does take some time to tune your CI towards a specific type of application but when that’s done it’s a breeze to develop applications of that type.

In this walkthrough, we will be using Docker to run all our apps in, including all the stages of the CI. We will also use Drone CI which is a Continious Integration platform written in Golang that utilizes Docker extensively so that works hand-in-hand with our production deployment. I have a private Git called Gogs which is a super lightweight, GitHub-like remote written in Golang as well which I will walk you through setting up with Drone. This Git remote is completely optional and you can use GitHub or GitLab if you want and I will include Drone setup instructions for those remotes too. Keep in mind though, that GitLab compared to Gogs is very slow and resource intensive. Gogs requires no more than 15MB of RAM and can literally run on a Raspberry Pi. My installation hosts 35 repos and currently uses 14MB of RAM.
Drone is also very lightweight compared to other CI platforms.

In my companys’ production environment we run multiple servers each running CoreOS as operating system. We will not use that in this walkthrough as that’s a completely different story as you have to keep lots of things in mind regarding MongoDB where its WiredTiger storage engine works best with XFS filesystem and SSD RAID arrays and the installation is a bit tricky where you have to use an iPXE script. Things like that are completely out of the scope for this walkthrough so we’re simply going to be running off of a single Ubuntu 14.04 machine.

CoreOS is really useful though, as it allows for some really badass high availability and scaling without having to do much lifting apart from setting up the initial state, so I definitely suggest that you check it out. I might edit this post in the future to add a CoreOS version if there’s demand for it.

Let’s get on with it!

Setup

Server

Make sure you have a clean installation of Ubuntu!
First thing we have to do is install docker on our machine. But before we do that we need AUFS support

sudo apt-get install linux-image-extra-$(uname -r)

now you’ve gotta reboot with reboot.
When your server’s back up, you can go ahead and install Docker following this guide.

After that, make sure Docker is running AUFS storage driver with:
docker info | grep 'Storage Driver:'

From this point on, don’t be root user - create a new user if required. In this walkthrough we will be the user app. Our working directory will be the home of the app user, /home/app.

Nginx

We will be using an Nginx installation that proxies requests to applications and can also serve as a load balancer if you add more nodes to your Meteor application. Nginx is also a lot faster on serving static files than Node is - primarily due to it utilizing the sendfile(2) syscall.

First we need to create some directories, remember our working directory is /home/app.

mkdir -p conf/nginx log/nginx data/nginx

Let’s pull Nginx docker container and start it up so we can get the default configuration.

docker pull nginx:latest
docker run --name nginx -v /home/app/data/nginx:/usr/share/nginx/html -v /home/app/conf/nginx:/etc/nginx -v /home/app/log/nginx:/var/log/nginx -d -p 80:80 -p 443:443 --restart=always nginx
rm conf/nginx/conf.d/default

Now we should have the default Nginx configuration files in /home/app/conf/nginx and all logs in /home/app/log/nginx.

The --name option gives container a unique identifier which is really important, you’ll see later on.
The -v options stands for volume and mounts a directory from the host to the container. So the syntax goes -v /host/directory:/container/directory.
The -d option runs the container in detached mode (in the background).
The -p option maps a port from the host to the container.
The --restart=always option simply restarts the container if it crashes.

We will do more configuration of this later on, but for now we’re just going to leave it at that.

Gogs (Optional)

If you wish to install Gogs, we can do so super easy with Docker:

docker pull gogs/gogs
docker run --name=gogs -p 10022:22 -d -v /home/app/data/gogs:/data --link nginx:nginx --restart=always gogs/gogs

Now we want to create a Nginx server block for our gogs installation:

nano conf/nginx/cond.d/git.domain.tld

and paste this inside (customize to your setup):

upstream gogs {
  server gogs:3000;
}

server {
  listen 80;
  server_name git.domain.tld;

  access_log /var/log/nginx/git.domain.tld.access.log main;
  error_log /var/log/nginx/git.domain.tld.error.log warn;

  location / {
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_pass http://gogs;
  }
}

Now we want to test that the nginx configuration works docker exec nginx nginx -t if everything goes smoothly, you can reload with docker exec nginx nginx -s reload. Now if you go to http://git.domain.tld you should get to the installation process of Gogs.

Use SQLite3 as database engine.
Set domain to git.domain.tld.
Set SSH port to 10022.
Set Application URL to http://git.domain.tld/
Disable user registration and force login screen if you want a completely private git remote.

Create a user account directly in the installation process to make things simpler for you. When that’s done you have a fully working Gogs installation. Now you should add your local dev machine’s SSH key to your user account.

I suggest keeping a look at gogits/gogs GitHub repository for updates and when there’s one, run:

docker pull gogs/gogs
docker stop gogs
docker rm gogs
docker run --name=gogs -p 10022:22 -d -v /home/app/data/gogs:/data --link nginx:nginx --restart=always gogs/gogs
docker exec nginx nginx -s reload

The reload is to tell nginx that we’ve restarted our Gogs application since its IP has changed and you have to do that each time you start/restart a container that is linked to the nginx container.

Docker Private Registry (Optional)

You definitely don’t need this if you’re building open source applications - you could use Docker Hub instead, but you probably want it if you’re building closed-source applications. Here’s a quirk though, you need an SSL certificate to host a Docker Private Registry. You cannot put CloudFlare in front of the registry as they have a upload limit of 50MB so the pushes are not going to work. You could try Let’s Encrypt but at the time of writing, their certs are not available.

Grab your certificate and key and put them inside /home/app/certs and run:

mkdir -p conf/registry/auth data/registry
cp -R certs conf/registry/certs
cp -R certs conf/nginx/certs

Make sure you change domain.crt and domain.key and run:

docker run -d --restart=always --name registry -v /home/app/conf/registry/auth:/auth -e "REGISTRY_AUTH=htpasswd" -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e "REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd" -v /home/app/conf/registry/certs:/certs -e "REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt" -e "REGISTRY_HTTP_TLS_KEY=/certs/domain.key" -v /home/app/data/registry:/var/lib/registry --link nginx:nginx registry:2

You will also need to create a user for the registry which you will use to authenticate with. Replace testuser and testpassword.
docker run --entrypoint htpasswd registry:2 -Bbn testuser testpassword > auth/htpasswd

We need to generate a dhparam.pem file with:
openssl dhparam -out conf/nginx/dhparam.pem 4096

Lastly, we want to create a server block for our Nginx proxy:
nano conf/nginx/conf.d/registry.domain.tld
paste and modify:

upstream registry {
  server registry:5000;
}

server {
  listen 80;
  server_name registry.domain.tld;
  return 301 https://$host$request_uri;
}

server {
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;  
  ssl_prefer_server_ciphers On;
  ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4;
  ssl_session_cache shared:SSL:50m;
  ssl_stapling on;
  ssl_stapling_verify on;
  ssl_dhparam /etc/nginx/dhparam.pem;

  ssl_certificate /etc/nginx/certs/domain.crt;
  ssl_certificate_key /etc/nginx/certs/domain.key;
  ssl_trusted_certificate /etc/nginx/certs/trusted.crt;

  add_header X-Frame-Options SAMEORIGIN;
  add_header X-Content-Type-Options nosniff;
  add_header X-XSS-Protection "1; mode=block";
  add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";

  listen 443 ssl default deferred;
  server_name registry.domain.tld;

  access_log /var/log/nginx/registry.domain.tld.access.log main;
  error_log /var/log/nginx/registry.domain.tld.error.log warn;

  client_max_body_size 2G;

  location / {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_pass https://registry;
  }
}

The ssl_trusted_certificate is optional. You can comment it out if you don’t have a trusted CA cert available - you also have to comment out ssl_stapling and ssl_stapling_verify if you do.

Now reload Nginx again docker exec nginx nginx -s reload.

You should now be able to route to https://registry.domain.tld and if you get a 404 page not found plain text error, everything is working as it should.

Drone

Here comes the fun part!

Create a conf and data directory: mkdir -p conf/drone data/drone
Create a configuration file: nano conf/drone/dronerc and insert the following for Gogs:

REMOTE_DRIVER=gogs
REMOTE_CONFIG=http://git.domain.tld

For GitHub, see Drone GitHub
For GitLab, see Drone GitLab

Run docker pull drone/drone:latest followed by:

docker run -d --name drone -v /home/app/data/drone:/var/lib/drone -v /var/run/docker.sock:/var/run/docker.sock --env-file=/home/app/conf/drone/dronerc --restart=always --link nginx:nginx drone/drone:latest

Create an Nginx server block for it nano conf/nginx/conf.d/drone.domain.tld:

upstream drone {
  server drone:8000;
}

server {
  listen 80;
  server_name drone.domain.tld;

  access_log /var/log/nginx/drone.domain.tld.access.log  main;
  error_log /var/log/nginx/drone.domain.tld.error.log warn;

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Host $http_host;
    proxy_set_header Origin "";

    proxy_pass http://drone;
    proxy_redirect off;
    chunked_transfer_encoding off;
    proxy_http_version        1.1;
    proxy_buffering           off;
  }
}

Reload nginx docker exec nginx nginx -s reload and head over to http://drone.domain.tld. You should now be able to login with the same username and password that you use to login with Gogs.

MongoDB

I’m not gonna get into too much detail about this. It’s fairly simple.

Create data directory: mkdir -p data/mongo
Get container: docker pull mongo:latest
Run it: docker run -d --name mongo -v /home/app/data/mongo:/data/db --restart=always mongo:latest
Optionally add --storageEngine wiredTiger if you want to use the WiredTiger storage engine.

CI Integration

Now we’re going to start integrate Meteor with our Git, CI and Docker Registry. Make sure you’ve pushed your project to your Git remote.

Drone

Log in and find your project and activate it. Go to Settings and activate the Tag Hook. Lastly

Now go to the Secrets tab and paste this:
modify username and password with the credentials you used to setup your registry
change ip to the ip of the server you want to deploy to

environment:
  - REGISTRY_USER=username
  - REGISTRY_PASSWORD=password
  - SERVER_IP=127.0.0.1
  - SERVER_USER=app

you should recieve an encrypted string back which you have to remember for the next step.

Git

Go to your local repository and create 3 files: touch Dockerfile .drone.yml .drone.sec.

Paste the encrypted string you got from Drone into the .drone.sec
Paste this into your .drone.yml and modify it accordingly:

build:
  image: svenskunganka/meteor:latest
  commands:
    - meteor build .
    - tar -xzf *.tar.gz
    - mv bundle dist
    - (cd dist/programs/server && npm install)
    - rm *.tar.gz
  when:
    event: tag

publish:
  docker:
    registry: registry.domain.tld
    username: $$REGISTRY_USER
    password: $$REGISTRY_PASSWORD
    email: email@example.com
    repo: user/repo
    tag: latest
  when:
    event: tag

deploy:
  ssh:
    when: 
      event: tag
    host: $$SERVER_IP
    user: $$SERVER_USER
    port: 22
    commands:
      - "docker login -u '$$REGISTRY_USER' -p '$$REGISTRY_PASSWORD' -e 'email@example.com' registry.domain.tld"
      - "docker pull registry.domain.tld/user/repo:latest"
      - "docker ps -a | grep 'registry.domain.tld/user/repo:latest' | awk '{print $1}' | xargs --no-run-if-empty docker stop"
      - "docker ps -a | grep 'registry.domain.tld/user/repo:latest' | awk '{print $1}' | xargs --no-run-if-empty docker rm"
      - "docker run --name 'appName' -d --restart=always -e 'ROOT_URL=http://domain.tld' -e 'MONGO_URL=mongodb://mongo/test' -e 'NODE_ENV=production' -e 'PORT=3000' --link nginx:nginx --link mongo:mongo registry.domain.tld/user/repo:latest"

Paste this to your Dockerfile:

FROM node:0.10
RUN adduser --system --disabled-password --home /home/app app
ADD dist /home/app
USER app
EXPOSE 3000
CMD ["node", "/home/app/main.js"]

Commit your changes git add --all && git commit -m "add CI integration" && git push origin master
Now create a tag with git tag -a v1.0.0 -m "Version 1.0.0" && git push origin v1.0.0 and the build, publis and deploy should fire. You also need to create a nginx server block but I am super tired and will update tomorrow. I probably missed some things here and there and will fill in that as I find it.

Cheers!