Docker: team up to create a reference docker image for meteor

mitar · March 1, 2017, 3:27am

Why everyone is ignoring my Docker image.

michaelcole · March 1, 2017, 4:18am

Hey, there’s alot here. I went through this when setting up my Meteor apps in Kubernetes on AWS and then moved to Google Cloud. I was using DigitalOcean with Mup, but had to leave it behind. After a bunch of work, here’s what I ended up with. TLDR; Meteor is a node app - just use a node Docker image.

Here’s exactly what I did. Sorry for the long post, but OP asked

I use this Docker image for my node.js apps including Meteor:

Docker file:

FROM buildpack-deps:jessie

# See https://github.com/nodejs/docker-node/blob/master/4.5/slim/Dockerfile
# This copy changes NODE_VERSION

ENV NODE_VERSION 4.6.2
ENV NPM_CONFIG_LOGLEVEL info

WORKDIR /var/app
COPY loadEnvRunNpm.sh .
RUN chmod 755 loadEnvRunNpm.sh

# gpg keys listed at https://github.com/nodejs/node
RUN set -ex \
  && for key in \
    9554F04D7259F04124DE6B476D5A82AC7E37093B \
    94AE36675C464D64BAFA68DD7434390BDBE9B9C5 \
    0034A06D9D9B0064CE8ADF6BF1747F4AD2306D93 \
    FD3A5288F042B6850C66B31F09FE44734EB7990E \
    71DCFD284A79C3B38668286BC97EC7A07EDE3FC1 \
    DD8F2338BAE7501E3DD5AC78C273792F7D83545D \
    B9AE9905FFD7803F25714661B63B535A4C206CA9 \
    C4F0DFFF4E8C1A8236409D08E73BC641CC11F4C8 \
  ; do \
    gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; \
  done

RUN buildDeps='xz-utils' \
  && set -x \
  && apt-get update && apt-get install -y $buildDeps --no-install-recommends \
  && rm -rf /var/lib/apt/lists/* \
  && curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.xz" \
  && curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc" \
  && gpg --batch --decrypt --output SHASUMS256.txt SHASUMS256.txt.asc \
  && grep " node-v$NODE_VERSION-linux-x64.tar.xz\$" SHASUMS256.txt | sha256sum -c - \
  && tar -xJf "node-v$NODE_VERSION-linux-x64.tar.xz" -C /usr/local --strip-components=1 \
  && rm "node-v$NODE_VERSION-linux-x64.tar.xz" SHASUMS256.txt.asc SHASUMS256.txt \
  && apt-get purge -y --auto-remove $buildDeps \
  && ln -s /usr/local/bin/node /usr/local/bin/nodejs

ENTRYPOINT [ "bash", "loadEnvRunNpm.sh" ]
CMD ["start"]

loadEnvRunNpm.sh let’s you attach a volume with env.sh to set environment variables. Put this shell script in the same folder as the Docker image.

#!/bin/bash

if [ -f "/etc/env-volume/env.sh" ];  then
  echo "Environment volume found"
  source /etc/env-volume/env.sh
else
  echo "WARNING: Environment volume not found"
fi
npm ${1:-start}

Here’s a quick build.sh that can go with the above two files:

#!/bin/bash
IMG=node:4.6.2
LOCAL=awesome/$IMG

# build locally
docker build -t $LOCAL .    || exit
docker push $LOCAL

So far, this is all node.js. My Meteor app has a Dockerfile (just like my other node apps):

FROM awesome/node:4.6.2

# Install our app
RUN mkdir -p /var/app
WORKDIR /var/app

COPY bundle .

# install dependencies
RUN (cd programs/server && npm install -q )

# Copy package.json for NPM start
COPY package.json /var/app

# Expose ports and run
ENV PORT 80
EXPOSE 80

# This will be passed as parameter to NPM
CMD ["start"]

My Meteor app folder has a Dockerfile in the root, and an $APP folder with the Meteor app. In the root I have a package.sh file that most importantly does this:

# build locally
rm -rf $BUILDDIR
pushd .
cd $APP

# Npm install
[ ! -d "node_modules" ] && npm install
# Meteor build
meteor build \
  --directory ../$BUILDDIR \
  --architecture os.linux.x86_64 \
  --server-only || exit 1
popd

# Docker it
VERSION=$(node -p -e "require('./package.json').version")-$(git rev-parse --short HEAD)
tar -cz Dockerfile package.json -C $BUILDDIR bundle | \
  docker build -t $REPO:$VERSION - || exit 1
docker push $REPO:$VERSION || exit 1

VERSION ends up being a human version from package.json plus a git hash, so I can track images sanely (I have dev/stage/prod in git, but use the hash in docker). tar sends everything into docker for building, then get’s pushed up. The - in docker build is frustratingly important.

So we’re going to switch to Kubernetes here, but it’s just YAML that defines what get’s sent to the docker command line. I trimmed a bunch of stuff here - the important part is defining the volume and adding the two files env.sh and settings.json. This way I can keep separate env.sh and settings.json for different environments (stage/prod).

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: dashboard-deployment
spec:
  replicas: 1
  template:
    spec:
      containers:
      - image: awesome/dashboard
        name: dashboard
        ports:
        - containerPort: 80
          name: http-port
        volumeMounts:
        - mountPath: /etc/env-volume
          name: config
          readOnly: true
      volumes:
      - name: config
        secret:
          defaultMode: 420
          items:
          - key: dashboard.sh
            mode: 256
            path: env.sh
          - key: settings.json
            mode: 256
            path: settings.json
          secretName: env-secret

The important part is loadEnvRunNpm.sh sources env.sh with all the secrets, and settings.json. If you see it complain WARNING: Environment volume not found in the logs, it’s not finding the config. This volume keeps your prod secrets out of your docker image. Kubernetes has been great for managing that - I can’t recommend Kubernetes enough:

export MONGO_URL="shhhh"
export MONGO_OPLOG_URL="shhhh"
export ROOT_URL=https://dashboard.awesome
export METEOR_SETTINGS=$(cat /etc/env-volume/settings.json)

So, this isn’t going to copy/paste work for you, but it may help you get where you want to go if the other options don’t appeal to you. This probably won’t work for a CI build.

lpgeiger · March 1, 2017, 8:20pm

@mitar, can you tell me a bit more about the image? I created a spreadsheet with some features of each implementation. Can you comment on those features? Thanks again.

mitar · March 1, 2017, 11:00pm

You just use it as a base image and it works. No special features, but it is just made that it works. It waits for example for database to be ready before trying to connect, it restarts Meteor automatically, it also makes a bundle. And it is maintained.

aadams · March 2, 2017, 4:53pm

I have a nigx server I use for SSL and I have an executable I need access to on the file system, along with adding/removing files and directories on the file system (or someplace outside of my application) Can this all be set up using your Docker image easily @mitar?

mitar · March 2, 2017, 6:27pm

For nginx server and SSL I use this image which I put in front of the Meteor image.

To access additional files, you just mount an extra volume into the container and then Meteor app can access it. If you look at an example here, I mount /storage in such way: --volume "${METEOR_STORAGE}:/storage".

You can also extend the init script (so every time a Meteor runs, it runs also this extra script, which is helpful to for example fix ownership of files you mounted in).

And if you need extra things to be installed into the image, you can add such script which will then be picked up by Docker image automatically at the build time.

aadams · March 2, 2017, 9:30pm

Wow that’s great @mitar, but I think I’m going to need an extra course on Docker to figure this all out!

mitar · March 3, 2017, 4:28am

I am open to pull requests against README with somebody making some examples of how to do various things.

But in general, for now, just add a Dockerfile to your Meteor repository and you can FROM tozd/meteor and this is it.

lpgeiger · March 8, 2017, 9:01pm

@abernix Do you have any thoughts about moving forward on official documentation for meteor-docker?

lpgeiger · March 9, 2017, 8:41pm

Has anyone had success building AND deploying on Docker Cloud with any Docker Image?
I’ve tested many and had no consistent success.
Anyone in production on Docker Cloud?

markoshust · March 10, 2017, 12:39pm

I’ve used this docker image on hundreds of builds. Works great. There isn’t much magic needed for meteor. Check out the builddeploy script for local build automation.

hoser23331 · June 25, 2017, 4:56pm

hi,

do you maybe have a manual on how to run this? I tried to create the 1 line dockerfile and execute the build command, but what should I do next?

Do I first have to build my windows based meteor project into a bundle? this is not documented?

markoshust · June 25, 2017, 5:32pm

See the builddeploy script for usage

hoser23331 · July 1, 2017, 4:31pm

hi, this TODZ image gives this error with meteor 1.5

PS: it does not mentoin it should work on 1.5… (only 1.4)

hoser23331 · July 1, 2017, 4:42pm

Hi, to be honest I have no clue where to begin… (yes I have seen a lot of docker videos, so if there is a manual I can follow it)

hoser23331 · July 1, 2017, 5:07pm

#miniMongo

I can run the container like this docker run -it -e ROOT_URL=http://localhost -e MONGO_URL=localhost -p 80:3000 -e METEOR_SETTINGS="$(cat settings.json)" e159f5090bfd now people only just need to know how to get a simple mongoDB running for meteor.

Do you know how I can run a Mongo image that I can connect my new meteor container?

I was hoping, that there would be a Mongo image with a predefined database to which I can just connect?

For example what is unclear for me is this. I understand what a mongo_url is, and if I order one at compose I get an URL to a specific database. But if I run a mongo image, I have no pointer to a default database, and I don’t see a ‘use database’ command in the docker files.

mitar · July 1, 2017, 10:06pm

Thanks. Can you report this to its GitHub issue tracker?

hoser23331 · July 2, 2017, 7:09am

So maybe good to know: I use docker for windows. and via CMDER I can execute commands.

I downgraded my project (master still at 1.5) to meteor 1.4.4 and also here I get the EISDIR error both with TODZ as well as the meteor launchpad images.

Error: EISDIR: illegal operation on a directory, unlink '/opt/meteor/src/.meteor/local/dev_bundle'

EDIT: EISDIR: illegal operation on a directory · Issue #8224 · meteor/meteor · GitHub could be the issue. (I use CMDER and not CMD or powershell)

EDIT: with powershell I get another error


/root/.meteor/packages/meteor-tool/.1.5.0.utbu0o++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/meteor-promise/promise_server.js:218
      throw error;
      ^

Error: EINVAL: invalid argument, readlink '/tmp/mt-1hv15b3/mt-os.linux.x86_64/dev_bundle/lib/node_modules/syntax-error/node_modules/.bin/acorn'
    at Error (native)
    at Object.fs.readlinkSync (fs.js:857:18)
    at Object.realpathSync (fs.js:1457:25)
    at Object.wrapper [as realpath] (/tools/fs/files.js:1586:35)

EDIT: this also does not work I got it working by manually removing all directories and links where the name started with "dev_bundle" inside the "/.meteor/local" directory. from Error while starting Meteor App: illegal operation on a directory, unlink ".../dev_bundle" - Stack Overflow

hoser23331 · July 2, 2017, 9:06am

project ricochet

Also not succesful

/bin/sh: 1: cd: can't cd to /home/meteorapp/meteorapp/app

martinezko · July 6, 2017, 9:20pm

Still interesting thread!

I already test som new docker meteor build in ubuntu based image from phusion/baseimage-docker … actually it is only beta test. I will bu lucky if some body can help test this image:

Thanks!