Acceptance testing on Circle CI 2.0


#1

Folks -

has anyone had luck setting up acceptance tests on circle ci’s 2.0 environment using docker images?

Regards
Addy


#2

not yet. i started to migrate my 1.0 workflow and noticed how much more complex it seems. would be great to have a working example for Meteor to work from ideally.


#3

yeah I’m going through this now and think it would be great to have an official Meteor image akin to the node and other language docker images they provide


#4

would you be willing to post an example of your configuration?

that would probably be really useful to those who are tackling this soon and much very much appreciated be me!


#5

Alright I’ve got CircleCI 2.0 setup running unit tests and functional tests for pull requests.

I ended up using one of CircleCI’s pre-build Docker images, specifically the circleci/openjdk:8-jdk-browsers which comes with:

  • Java 8 (which is needed for Selenium tests)
  • web browsers accessible in headless mode

Then in my Circle config.yml job config I do the following:

  • set some environment variables needed for Mongo
  • install build-essentials, necessary for building some npm packages
  • install Meteor
  • run code linting
  • run unit tests
  • spin up a dev server with the meteor command and run functional tests against it with chimp using this script
  • upload unit test results to a job artifact

The whole ordeal takes about 5 minutes right now, you could pull out the functional tests step and move it to another job in the workflow if you wanted to speed up builds.

Here is my config.yml

version: 2
jobs:
  build:
    working_directory: ~/app
    docker:
      - image: circleci/openjdk:8-jdk-browsers
        environment:
          # lang settings required for Meteor's Mongo
          LANG: C.UTF-8
          LANGUAGE: C.UTF-8
          LC_ALL: C.UTF-8
          LC_NUMERIC: en_US.UTF-8
          METEOR_BIN_TMP_DIR: /home/circleci/build-temp/
          METEOR_BIN_TMP_FILE: meteor-bin-temp
    steps:
      - checkout
      - restore_cache:
          key: build-temp-{{ checksum ".meteor/release" }}-{{ checksum ".circleci/config.yml" }}
      - restore_cache:
          key: meteor-release-{{ checksum ".meteor/release" }}-{{ checksum ".circleci/config.yml" }}
      - restore_cache:
          key: meteor-packages-{{ checksum ".meteor/versions" }}-{{ checksum ".circleci/config.yml" }}
      - restore_cache:
          key: npm-packages-{{ checksum "package.json" }}-{{ checksum ".circleci/config.yml" }}
      - run:
          name: install build essentials
          command: sudo apt-get install -y build-essential
      - run:
          name: restore cached meteor bin
          command: |
            if [ -e ~/build-temp/meteor-bin ]
            then
                echo "Cached Meteor bin found, restoring it"
                sudo cp ~/build-temp/meteor-bin /usr/local/bin/meteor
            else
                echo "No cached Meteor bin found."
            fi
      - run:
          name: install Meteor
          command: |
            # only install meteor if bin isn't found
            command -v meteor >/dev/null 2>&1 || curl https://install.meteor.com | /bin/sh
      - run:
          name: check versions
          command: |
            echo "Meteor version:"
            # this forces Meteor to download whatever release your project is using
            meteor --version
            which meteor
            echo "Meteor node version:"
            meteor node -v
            echo "Meteor npm version:"
            meteor npm -v
            echo "Java version:"
            java -version
      - run: 
          name: install npm packages
          command: meteor npm i
      - run:
          name: code linting
          command: meteor npm run lint
      - run:
          name: unit tests
          command: meteor npm run test:unit
      - run:
          name: start app and run e2e tests
          command: meteor npm run test:e2e-standalone
          no_output_timeout: 5m
      - run:
          name: copy meteor bin to build cache
          command: |
            mkdir -p ~/build-temp
            cp /usr/local/bin/meteor ~/build-temp/meteor-bin
      - save_cache:
          key: build-temp-{{ checksum ".meteor/release" }}-{{ checksum ".circleci/config.yml" }}
          paths:
            - ~/build-temp
      - save_cache:
          key: meteor-release-{{ checksum ".meteor/release" }}-{{ checksum ".circleci/config.yml" }}
          paths:
            - ~/.meteor
      - save_cache:
          key: meteor-packages-{{ checksum ".meteor/versions" }}-{{ checksum ".circleci/config.yml" }}
          paths:
            - .meteor/
      - save_cache:
          key: npm-packages-{{ checksum "package.json" }}-{{ checksum ".circleci/config.yml" }}
          paths:
            - ./node_modules/
            - ~/.npm/
      - store_artifacts:
          path: ./tests/coverage/unit-tests/
          destination: unit-tests

Here is a build from my CircleCI job using this setup.

edit: changed the config to cache all of Meteor to speed it up


Recommended testing packages (mainly headless UI testing)
New Meteor app - lessons learned
#7

I was finally about to build an image that can be used with the circleci 2.0

this is my dockerfile:

FROM node:latest

# Install ps command
RUN apt-get update && \
  apt-get -y upgrade && \
  apt-get install -y --no-install-recommends \
    build-essential \
    curl \
    locales \
    procps \
    ca-certificates \
    bzip2 \
    libfontconfig \
    git && \
  apt-get remove -y man && \
  rm -rf /var/lib/apt/lists/*

RUN set -x  \
  && mkdir /tmp/phantomjs \
  && curl -L https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 \
        | tar -xj --strip-components=1 -C /tmp/phantomjs \
  && mv /tmp/phantomjs/bin/phantomjs /usr/local/bin \
  && apt-get clean \
  && rm -rf /tmp/* /var/lib/apt/lists/*

RUN localedef -i en_US -f UTF-8 en_US.UTF-8

RUN groupadd -r meteor && useradd -ms /bin/bash -g meteor meteor
USER meteor
WORKDIR /home/meteor

RUN curl https://install.meteor.com/ | sh

WORKDIR /home/meteor/app
ENV HOME /home/meteor
ENV PATH $HOME/.meteor:$PATH

EXPOSE 3000
EXPOSE 4000

CMD ["meteor","npm","start"]

and here is my circleci config

version: 2
jobs:
  build:
    working_directory: ~/app
    docker:
      - image: imagename
        auth:
          username: port80labs  # can specify string literal values
          password: $DOCKERHUB_PASSWORD
    steps:
      - checkout
      - restore_cache:
          name : Restore NPM Cache
          key: npm-cache-{{ checksum "package.json" }}
      - restore_cache:
          name: Restore Meteor Package Cache
          key: packages-cache-{{ checksum ".meteor/versions" }}
      - run:
          name: Create Settings File
          command: echo $METEOR_SETTINGS > settings.json
      - run:
          name: Install npm dependencies
          command: meteor npm install
      - save_cache:
          name: Save NPM Cache
          key: npm-cache-{{ checksum "package.json" }}
          paths:
            - 'node_modules'
      - run:
          name: Run Test
          command: meteor npm test
      - save_cache:
          key: packages-cache-{{ checksum ".meteor/versions" }}
          paths:
            - './.meteor/local/build'
            - './.meteor/local/bundler-cache'
            - './.meteor/local/isopacks'
            - './.meteor/local/plugin-cache'

hope this helps someone


#8

Very helpful. Thanks, @so1oonnet!