Which testing frameworks are popular today - end 2018?

[simonsimcity]

I have the strong impression that the current chapter for testing in the meteor guide (https://github.com/meteor/guide/blob/master/content/testing.md) does not really reflect how the community here does testing.

The current testing chapter mentions some outdated frameworks and leaves out mentioning a lot which I found by a research - https://github.com/meteor/guide/issues/888 and https://github.com/meteor/guide/issues/672 are two examples.

The main testing framework currently seems to be mocha. The docs there need to be improved https://github.com/meteortesting/meteor-mocha/issues/80 but there’s also https://github.com/cult-of-coders/meteor-mocha which is a fork of https://github.com/practicalmeteor/meteor-mocha, making it usable for Meteor 1.6.x and newer, making it possible to show the results of both server and client tests in the browser. @diaconutheodor would be nice if you would join the conversation.

On the other side I also found some people playing around with Jest (https://blog.meteor.com/real-world-unit-tests-with-meteor-and-jest-3d557e84e84a, https://www.npmjs.com/package/jest-meteor-stubs) which I mainly started looking into because they support snapshot testing (https://jestjs.io/docs/en/snapshot-testing). It could be that this also is more suited for creating unit-tests, I don’t know. @rdickert, I also want you to join, because you wrote the article.

What do you use? Which is (as of your opinion) the better choice to take and should be included (removed, replaced, updated, kept) in the guide?

When I see this thread settling, I’ll create a PR to the repository of the guide.

This thread is meant as a fork of Testing guide article - providing a place to discuss which of the frameworks the testing-guide should include.

• Mocha (meteortesting)
• Mocha (cult-of-coders)
• Mocha (practicalmeteor)
• Mocha (dispatch)
• Mocha (other)
• Jasmine (sanjo)
• Jest
• Tinytest
• Other (please explain)

[simonsimcity]

Just to show off how we do testing:

We currently do all our testing (unit and functional) using meteortesting:mocha in combination with the npm packages chai, sinon and the meteor package johanbrook:publication-collector.

Another bunch of tests is run by creating a special version of the meteor application, containing xolvio:backdoor, where we do end-to-end tests using chimp and cucumber by remotely controlling a browser. These tests could easily be written by one of our clients who wants to write his own tests against our application.

[ziinc]

Mocha+puppeteer+xolvio/backdoor seems like a contender for replacing chimp/chimpy for ui testing, especially since chimp/chimpy doesn’t seem to be maintained anymore

[arggh]

Heard a lot of good things about Cypress, but haven’t dived in yet.

[diaconutheodor]

I also recommend this: https://github.com/hubroedu/meteor-mocha this is a fork of cult-of-coders:mocha, with coffeescript stripped and a sexier design!

[simonsimcity]

@moberegger, @cloudspider, @sabativi, @gywem1989, @bduff9, @primerlabs could you please describe how you set up Jest? Are you only writing unit-tests by mocking away all of Meteors libraries or are you also doing integration tests where you test your code in conjunction with the full Meteor framework? If yes, how have you set it up?

I want to get this info to improve the testing chapter - since you represent a fourth of the voters and I don’t have a good concept of how you’d write integration tests with Jest for Meteor.

@zodern, @veered it would also be nice to hear 2-3 sentences from you, since you voted for “other”. Even if you only tell that it’s something internal or outdated would already be enough information to me.

[veered]

We’ve been using Chimp for testing

[stolinski]

Jest & React Testing Library

[moberegger]

Are you only writing unit-tests by mocking away all of Meteors libraries

That’s what I’m doing, yeah. I wouldn’t gleam too much from that, though, because I have been primarily doing it to learn TDD and better testing practices in general. Jest just makes it easy for me. Mocking the Meteor stuff is necessary to get anything working in that environment.

Using Meteor’s out of the box testing wouldn’t teach me much about DI, pure functions, and things like that, because it facilitates all the side effects for you. That feels better suited for larger integration tests (or E2E?) that I’m just not interested (… ie. capable :() in right now. Baby steps, I guess.

[csiszi]

Anyone know good resources on how to learn testing? I found these so far:

• real world unit testing with meteor and jest - seems easy, but mocking every meteor lib and package looks time consuming
• Kent C. Dodds’s testingjavascript - fresh, covers a lot of topic (basics, jest, cypress) but not meteor specific

[awatson1978]

That guide page has always been political and weirdly biased.

We’re still using Nightwatch for E2E testing after close to 4 years. Have used it to refactor core business logic and UI workflows from PHP to Meteor 0.5, then Backbone to Blaze to React. It’s been a rockstar for us, and has integrated with Meteor fantastically via it’s detailed config file, which we store as .meteor/nightwatch.json.

Also using:

• mocha:meteortesting for Atmosphere package integration and unit tests.
• Jest for UI tests.
• Coveralls and for code coverage in NPM packages.
• Newman for API, load, and stress testing.

Nightwatch v1.0 ditches the old Java Selenium Hub, and uses direct Chromedriver functionality, so it’s now 100% V8 interpretor, instead of just having a javascript API. Nightwatch also got a big boost last year when Walmart created an extention for use with TestArmada.io.

Along those lines, Webdriver.io is the new hotness in the E2E testing community. It was the first framework to ditch the Java based Selenium hub for pure V8, and is now rolling out features like video recording of test runs.

For the E2E section of the testing article, take a look at the NPM numbers, which I think speak for themselves.

Nightwatch - 154,605 weekly downloads
WebDriver - 62,615 weekly downloads
Chimp - 1,793 weekly downloads

[satya]

We’ve been doing extensive testing with Cypress the past 3 months, and unfortunately it doesn’t live up to its promise that it’s not flaky. There are notable differences between the CI and visual version, as well as running Cypress in Docker. It’s very cool, but our overall feeling is that it is not that mature yet.

[drollins]

I practice TDD - use Blaze and React for unit and integration testing.

Cultofcoders:mocha
Sinon
Enzyme

I like the fact that I can very quickly debug my tests in the browser as needed.

E2E testing I still need to implement. Looking at Nightwatch and Chimp. Should I b looking at anything else?