How to speed up acceptance (end-to-end) tests?

testing
1

Hi guys,

my chimp-based acceptance tests now run around 40 minutes…
in my current workflow this really slows down from time to time,
p.e. when I wait 40 minutes just to be “allowed” to do the git-commit.

I read somewhere that chimp is planning to support parallel-tests.
Is there any progress on this (parallel tests)?

@sam: are there any updates on this?

Is there anything else you recon for a better/faster acceptance test workflow?

2

OK, for the future readers, this is what I did:

I split my tests into 3 parts, simply by assigning “@pass1”, “@pass2” & “@pass3” tags. Then I made use of the --watchTags=“” parameter and created 3 meteor-instances plus 3 chimp-instances running on different ports. It works great!!! I just need to spend some time on really equally splitting all tests upon those 3 instances… this is something that could be automated in chimp.

The commands I use like:
chimp --ddp=http://localhost:3200 --mocha --path=tests --watchTags="@pass1" --watch
and
meteor test --full-app --driver-package tmeasday:acceptance-test-driver --port 3200;

In my terminal it looks like this: (I actually have 1 extra chimp-instance for tests marked with “@watch” - those are the tests I am working on at the moment)

1440×886 76.1 KB

The only thing I am missing is the option to run those filtered-tests without the --watch instance. Now all tests restart whenever code changes.

@sam @Sanjo : I have 2 feature suggestions for chimp:

  1. It would be great if chimp would support splitting up tests by default. Allow the user to configure the “number of parallel meteor instances”… then internally note how long each test takes to pass and split the tests up equally upon the chimp-instances.
  2. Can you make the --watchTags="@*" filter work even with “watch”-mode itself disabled? This would be really handy, so that we can keep on developing without the tests restarting?
3

You can use tags to split your tests for sure. The other option you have is to use directory based splitting.

  1. You can definitely add a feature request in GitHub. We’d love to make that happen soon, but we just don’t have the time to add such a feature at the moment.

  2. You don’t need to use watchTags, you can just use tags for non watch tags :slight_smile: See the cucumber.js docs