📡 TRANSMISSION #9: Testing Deep-Dive with Tom Coleman

[benstr]

Thanks for your patience! Transmission #9 audio has been released. Enjoy!

#9 Test Mode in 1.3 and what happened to Velocity | Transmission

Test Mode in 1.3 and what happened to Velocity

[ 01:20 ] - Why not provide a MDG test mode earlier?
[ 07:15 ] - How to re-brand Meteor as “professional grade”?
[ 16:30 ] - What happened with Velocity?
[ 24:24 ] - Thank you all test package maintainers!
[ 28:00 ] - Overview of Test Mode
[ 34:00 ] - Full-App mode
[ 41:50 ] - Difference between standard and full-app modes
[ 49:25 ] - Chimp will be in the guide for E2E
[ 49:56 ] - TinyTest R.I.P
[ 51:30 ] - What are Driver-Packages?
[ 57:00 ] - What can the community do?
[ 61:55 ] - Was 1.3 too big of a release?
[ 64:35 ] - Conclusion

[sashko]

Yo guys, this week the topic is… test mode!

So chill, dude. Can’t wait to see the video!

[sashko]

We’ve got the video! Lots of great info there about testing and also about the strategy to position Meteor as a tool for professional developers:

[abhiaiyer]

@sam was everything correct in what they said about velocity?

[ramez]

@sashko, Man … you blow me away! When do you sleep? Forum, webcasts, coding … wow!
Super impressive.

[sashko]

It’s what happens when you land a super awesome job where you get paid to do all of these things! Super pumped about that.

[sam]

@abhiaiyer yep, it was all spot on.

Thank you for the thanks guys, appreciate that

I would sum up the new testing modes as this:

###The new Meteor test commands
These are both integration testing modes. They provide the Meteor context to the tests and the system under test (SUT).

meteor test: does not automatically load all dependencies. Your tests load the deps as needed.

meteor test --full-app: automatically loads all dependencies. Your tests will have a fully configured app to run against.

In both modes, you can achieve unit testing if you isolate correctly which would be very fast, the downside here is that you have to wait for Meteor to startup and resolve dependencies.

###Using NPM testing tools
You can do pure unit testing by not loading Meteor at all by using test doubles. These would run at lightning speed. If you want to do this easily, you need to apply the separation of concerns principles. This option is for advanced testers as it requires a deeper level of design-level understanding of code.

@tmeasday For mocking / stubbing / etc you might want to check out this awesome framework instead of Sinon - http://blog.testdouble.com/posts/2016-03-13-testdouble-vs-sinon.html - it requires less testing knowledge to use

[tmeasday]

Thanks Sam, I did read that article!

I did toss up recommending TD over Sinon, but I think it’s a bit “fresh” at this point. Maybe in the next iteration (it’s not like we use Sinon too much in the guide, so it’ll be easy to replace—or mentally substitute for readers).

“Pure unit testing” will be great when we get there! AFAICT (and correct me if I’ve got this wrong) but running regular Meteor app code outside of the Meteor context will break as things stand because import from 'meteor/X' messes up the module replacement that things like Jest, testdouble.js and mockery use.

So unless you are testing pure utility functions, which isn’t that useful unless you write your app in a very test friendly way (is this what you mean by “deeper level of design-level understanding of code”?), it kind of falls over. Perhaps it would work for the front end of a React app though? (the “pure components”).

If someone has a chance to experiment more with this, I’d love to include something about it in the Guide.

The good news on this is that our strategy is definitely taking us in the direction of true node imports everywhere. This means that in the not-to-distant future Meteor apps will be testable in this way without having to contort your code at all!

[sam]

You’re right that testdouble is still new. It’s the right direction for sure as it simplifies a lot of issues with creating test doubles that trip up new users.

We already do pure unit testing at all levels in our best practices repo!

You can see here. The spec is importing modules directly into Cucumber domain-level units without Meteor at all.

The spec is doing the same thing here with pure React components as you mentioned.

And it’s doing the same thing here with the pure domain tests.

It’s working for Meteor specific code like publications as you can see here.

There are two things it’s doing though to enable the above. It’s still relying on global Meteor objects, so there aren’t any import from meteor/X calls yet. And it’s using referential paths for imports which is a little cumbersome.

The key is to minimize the amount of coupling to Meteor. There will of course be a few tests that need the Meteor context. Typically these are tests that focus on configuration. For example, using packages Astronomy where it’s just impractical to create test doubles. Interestingly this happens because such packages are deeply coupled. If Astronomy was built with testing in mind it would probably be easier!

Finally, we even wired up Webpack to run the tests directly within wallaby and bypass Meteor’s testing mode all together! The upshot of this is that we get realtime test results in the IDE using Walaby.js with 0 overhead. It’s pretty rad actually. I’ll make a video soon and post it up.

[awatson1978]

5/5 stars for testdouble.js

Thank you for giving it consideration and bringing it to Tom’s attention; as it’s in keeping with decades of Verification & Validation work by the FDA and NASA in Python and C#, and has a clean and simple API that won’t cause misunderstands in the workplace.

Starrynight would be happy to adopt it as a preferred library and promote it; and I’ll reach out to Nightwatch and Gagarin projects to see their status on mocking/stubbing and getting it supported by them as well.

This is an excellent recommendation.