Continuous Integration with Meteor, CircleCI, and Gagarin

@SkinnyGeek1010 I agree with you. The only thing which is keeping us from using “normal tools” for unit testing is the lack of modules. Gagarin’s main feature is to manage a sandboxed meteor app environment, which shouldn’t even be needed if one’s doing real unit testing, not “integration-unit” testing.


Once Meteor 1.3 has modules, the biggest next steps in testing would be (IMO):

  1. A great integration test runner
  2. A Meteor testing toolbox for stubbing and mocking all parts of the Meteor core framework and recommended packages

So, just did some preliminary tests, and it looks like Gagarin and Nightwatch are going to work great together. I just added Gagarin integration to StarryNight, and have Nightwatch and Gagarin both using the same ChromeDriver installation. It’s doing just fine picking up tests from the package directories also.

starrynight run-tests --framework gagarin --verbose
starrynight run-tests --framework gagarin --webdriver http://localhost:9515
starrynight run-tests --framework gagarin --path /packages/*/tests/gagarin/*.js

We should eventually be able to either a) replace the Mocha instance that Nightwatch can access with Gagarin, so that Nightwatch has access to Gagarin’s integration functionality; or b) add a Nightwatch dependency to Gagarin to access the Nightwatch UI testing API in Gagarin. Either way, I’m pretty sure we should eventually be able to get something like the following syntax:

var nightwatch = require('nightwatch');

describe('Items List Page', function() {

  var client = nightwatch.initClient({
    silent : true

  var browser = client.api();

  before(function () {
    return client.execute(function () {
      Items.insert({_id: 'someFakeId', name: "Foo"});

  it("should be available on server", function () {
    return server.wait(1000, 'until data is propagated to the server', function () {
      return Items.findOne({_id: 'someFakeId'});
    }).then(function (value) {
  it('should display the item in a list', function (done) {
      .waitForElementVisible('body', 5000)
      .assert.title('Items List Page')
      .waitForElementVisible('body #itemsList', 1000)
      .assert.visible('body #itemsList .listItem:nth-child(1)')
      .assert.containsText('body #itemsList .listItem:nth-child(1) h2', 'Foo')


Also, since Gagarin shares the same Client/Editor/TestEnvironment isomorphism that Nightwatch is using, it's going to be pretty trivial to add Gagarin support to the StarryNight-Helper package for Atom.  Should have Atom integration finished by the end of the weekend.

Gagarin running in Atom (using the starrynight-helper package).

Atom’s package manager is a bit buggy, and my account is currently out of sync, waiting for a DB admin to fix things; so for the time being, anybody interested in tinkering with this will need to do the following to install:

git clone ~/.atom/packages/starrynight-helper

@awatson1978 Nice! I am really impressed :slight_smile:


Likewise! Gagarin has been a giant Christmas present for our team, and I’m tentatively going to say that we’ll be done with the FDA testing story once we get it fully integrated. Nightwatch Validation tests and Gagarin Verification tests. I think that may be a winner.

So, over the weekend, I set up a repository that includes all of the packages from the Clinical Meteor track, and got Gagarin to scan all of the packages in it! Only three of the packages currently have Verification test coverage, but initial work seems to indicate that this is going to work perfectly for package and distro maintainers.

The glob pattern is a really great approach, btw.

Updated the StarryNight/Nightwatch architecture diagrams to include Gagarin and Circle CI.


The idea to use glob for gathering test cases is thanks to @smeijer BTW :wink:

@samcorcos Just wanted to point out that gagarin@0.4.12 release is out, including the “platforms file fix” we’ve been discussing.