Super-fast, 3rd-Generation Reactive Testing with Chimp

Updates:
MDG highly likely to use Chimp in Meteor guide
Podcast reviewing Chimp

We recently released a new version of Chimp that has deeper integration with Meteor to allow you to do next-generation testing. Here are some highlights:

Reactive Testing
If you change your application files, Chimp will detect the hot-code deployment and reruns in an instant. As soon as you save changes to a spec, Chimp runs in an instant. You can also add adhoc directories to watch. All of this is happening without Velocity.

Synchronous Testing
No callback hells or promise headaches, we took the original WebdriverIO and made it synchronous! So you can write easy-to-read tests. We also wrapped the node DDP library and made that synchronous. This means you can do a server call, then a client call, then a server call then an assertion from an opto-decoupled test.

Contextual Objects In Your Specs
Within the context of your steps, you get a client object. This is a pre-configured WebdriverIO instance that you can use to control the front-end using ANY browser and you get a server object is a pre-connected DDP instance. This allows you to test Meteor methods and publications or to use data fixtures. You also get a request object that allows you to test REST api’s or to use data fixtures over HTTP

Remote Code Execution
WebdriverIO already comes with a method called client.execute, which runs code on the client. We added a server.execute method that does exactly the same thing as the client. This means you can employ some seriously advanced test-driving techniques, like Modeling by Example for instance.

Use Mocha Or Cucumber (Jasmine soon)
Choose the testing framework what you’re most comfortable with and enjoy the same benefits listed above in all of them.

Head over to the our blog to see the full announcement, or head straight to the docs site.

GitHub repo is here, give us a star if you like it! :wink:

23 Likes

Sounds like…

VELOCITY IS BACK!

5 Likes

Congrats @sam !

Using Chimp has been pure bliss… i’ve been used to beating my head against the wall using other selenium based products and I’ve always spent more time getting my failing tests to pass (when I know they should pass).

Chimp just seems to work :smiley:

I’ve been using Capybara and Rspec for a long time and Chimp is now actually faster with productivity and also when running. I can’t wait for the Jasmine support! (Mocha is nice too but all my unit tests are in Jasmine).

Is there a chance that in the future Chimp will be able to split features up per core and run 4 browsers at once?

Also will mocha support coffeescript via it’s internal transpiling? I’m not super concerned about this but it would be make for some seriously readable acceptance tests! Example: (using some capybara helpers with mocha)

require "capy-js"

describe "Create comment", ->
  before ->
    visit "/posts"

  it "should create a blog post", ->
    fill_in "Title", "Chimp is Awesome!"
    fill_in "Description", "How now brown cow"
    click_button "Create Post"
    expect(browser).to_have_content "Chimp is Awesome!"
    expect(browser).to_have_content "Billy Bob", within: ".author"
1 Like

Haha! Not quite, more like Xolv.io is back!

Velocity relies on the MDG to progress where as Chimp relies only on us. We have written it as a framework agnostic NPM package and we sprinkled some Meteor toys on top :wink:

3 Likes

Thanks @SkinnyGeek1010!

Glad it’s working for you and we’ll add Jasmine soon, promise.

This is exactly why we wrote Whirlwind! It can run multiple anything (not just Chimp) against anything (Not just Meteor)! We’ve used it to bring a build on CI from 1.5 hours to 12 minutes :wink:

Under the hood we’re using plain old Mocha so I don’t see why not. I guess we just need a mechanism to allow people to control Mocha at a raw level through Chimp so advanced users such as yourself can get more freedom.

1 Like

Congrats guys!

Though I would love to see how you got it working on CI services, I haven’t had much luck yet. But on my local machine it’s been much easier to work with and nicer to use, it actually made it much easier to figure out why one of my tests were failing as well.

Thanks @firrae

To get it working on CI, you need to write a script that will start a Meteor app then launch Chimp. We do need a sample project out there that with a circle/travis config that you can copy and roll with. I’ll get onto that and post back here.

Glad to see it’s working better for you locally!

1 Like

That’s cool, thanks. All I needed was the prod to the idea of starting it with a script.

I also assume that Whirlwind works properly with CircleCI’s parallelism so there’s nothing I would need extra on that is there? I took a glance and it seems like something useful to set up while I have vacation rather than as I wait for tests to build in the future.

1 Like

You might be able to do it with:

meteor run &
cd tests
chimp

The only thing is that it may run chimp too soon (before meteor starts) so you might need to throw in a sleep before running chimp.

re whirlwind, yes definitely.

I’m working a script right now to make your lives easier.

@SkinnyGeek1010 all you need is the Started proxy like to appear then the DDP connection will connect and wait

1 Like

@sam btw https://circleci.com/docs/background-process
to not throw away all your hair trying with &

1 Like

We are passing tests on CircleCI with

- cd app && meteor --settings .deploy/build/settings.json:
    background: true
- sleep 10
- cd app/tests/cucumber && chimp --ddp=http://localhost:3000 --browser=chrome --jsonOutput=cucumber_output.json

good luck fellow testers

2 Likes

Nice going! I’d love to see support for AVA, would be awesome as I’m working completely in ES6 at the moment.

2 Likes

Would love to see some documentation articles aimed specifically towards:

2 Likes

Those are great suggestions. Let me find some time in the next few days and I will do exactly that.

We actually use Chimp with React on our app and we go below the UI by doing things like

// this is in the test context (Mocha/Cucumber)
client.execute(widgets.myComponent.doSomething);

which is referring to a widget library that we have that looks something like this:

widgets.myComponent = {
  doSomething : function() {
    // this is the client context so we can bypass going through the UI and
    // access the components directly to get them to doSomething
  }
}

This is seriously awesome. Sounds like a major improvement…

So awesome in fact that I don’t mind re-writing all my cucumber tests for the 3rd or 4th time in the last 6 months :wink:

4 Likes

Thanks Hex!

For anyone just reading this thread, here’s an important update:

MDG highly likely to use Chimp in Meteor guide

And don’t take our word for the Chimp quality, check out this Meteor Strike podcast on Meteor testing

You don’t have to, it’s fully backwards compatible with xolvio:cucumber

2 Likes

hey @sam

I’m looking at various acceptance test frameworks and like the looks of Chimp, but I’ve noticed that in your docs you state that the widgets API doesn’t user fibers yet for synchronous calls and should be updated soon. Looking at the chimp-widgets GitHub issues, it looks like it may be done. Is this just a case of outdated docs? If so do you have an example of updated docs that include the synchronous widgets API?

thanks

@efrancis hello

Widgets has been removed from Chimp since Webdriver.io has native PageObject support now, which are awesome! They are all fiberized

I need to update the docs and remove the widgets reference