All great responses. I’ll chip in my 2c
Doing BDD is a good way to ensure your team(s) can scale as it can help you communicate effectively, save time on rework and it involves everyone from designers to developers.
There are two main problems people face when doing BDD. They are:
People don’t know the difference between specs and tests. Typically this ends up with developers getting annoyed that free text is not refactorable and they abandon ship!
There are so many moving parts in doing end-to-end automation. A developer typically has to start a selenium server, connect some webdriver bindings, manage sessions, setup some test data, wait for the right conditions to happen, and so on. When something goes wrong they find the whole experience brittle and they abandon ship!
xolvio:cucumber (which uses our chimpjs under the hood) aims to solve problem 2. It deals with all the problematic and mundane aspect of setup and wiring, and allows you to get on with the interesting automation parts. It’s been a rocky road trying understand it all and through deliberation and constant user feedback, we’ve improving and steadily achieving this goal and is now better than ever. Here’s what it does today:
@dev tag as you mentioned has been a huge win for performance
- You have a button in the reporter than runs the whole suite when you want
- Using Chrome as the default browser has made tests a ton more stable (phantom is now for the brave)
- Full CI support with automatic screenshot grabbing on failures
- We have created a fully tested app that runs on CI with awesome real feedback. Check out Letterpress
- Chimp has very good process management so the perf issues related to hanging processes have been dealt with
- The browser sessions are reused now using a Session manager so you get ultra quick response when you save a file
- We have made a ton of Meteor stubs packages to ease the automation for things like email, Github/Twitter OAuth and even Stripe
- We promisified everything so that you don’t have nested callback hell. Still, this area is not 100% as we like it (See fibers below)
- We have created mechanisms and patterns for setting up and managing test data
On the roadmap we have:
- Fiber based calls to Webdriver and server DDP from within steps so you can do synchronous style coding like Meteor
- We have an experimental parallel testing mode that currently requires big hardware, but we’re have a streamlined version planned
- Automatically generating boilerplate code for undefined steps
- Any other problems we find whilst we’re using it / hearing from others
We are also working on solving problem 1 with Simian.io to help people write better specs and collaborate with others. xolvio:cucumber will soon send results to Simian so you can see and share progress of your feature completion.
As you can see, we’re in love with this problem doing BDD, writing good specs and automating the world and we’re not stopping any time soon. The more than join this party and give feedback, the better this party will get.
It may not be everyone’s cup of tea, but if it is then we are sure serving the best out there!