Hey!
So as part of the guide, we’ve been doing a lot of investigating into testing, planning a testing article, and writing some tests for the todos app. Coincidentally, as we know from Sam’s post, Xolv.io are discontinuing work on Velocity, the official testing framework.
It’s a good point to step back and look at what the Velocity project is trying to achieve and what the framework can do to make it easier. Especially given the upcoming ES2015 module support and how that’ll change things.
I’ve been trying to figure out what the smallest possible change to the framework that would allow proper application testing and unit testing of modules[1] would look like. Then we can look at whether to include that as part of 1.3 or soon after (hopefully!).
Here’s what I have:
Meteor Application Testing
It seems like package/app authors could do all they needed if we supplied three things:
-
meteor test
, which bundles the app as usual, except with the contents of anytest/
directory included. -
Meteor.isTest
, which is true in “test mode” (meteor test
andmeteor test-packages
) - A
testOnly
flag for packages[2], which meant they are only included when running in “test mode”
What this would allow
A test package developer could create a testOnly
package mocha
which, when included:
- Exports the various symbols needed to define a mocha test.
- Adds client code to show the output of a mocha test run, and calls a publication to get the result
- Adds a publication which runs
mocha
when called
(This is basically what practicalmeteor:mocha
does already, it’s just right now the only way to get it to work is to use the undocumented --driver-package
, which only works for test-packages
)
A app developer could depend on mocha
in the app, and add tests to test/
folders which register tests via the standard mocha
API. Then they could simply run meteor test
to get a proper test over them.
A “singleton” package (like flow router) could inspect the Meteor.isTest
flag and not autostart in this mode.
Nice to haves, not needed for a first version
- It’d be great if compiled build output was shared between
meteor run
,meteor test
andmeteor test-packages
. But it’s not the case that it is already (betweenrun
andtest-packages
), so it’s hardly a deal breaker (i.e. this is strictly better than what there was before). - There are some weird command-line options could be rationalized and not made velocity-specific:
-
meteor run —test
(which runs the app as usual, but points phantom at it, talking to velocity) -
meteor test-packages —velocity
(which sort of does the same thing, but fortest-packages
)
-
- Although a “run once” API for a test reporter package would be sensible, combined with some kind of phantom script to run client side tests, I don’t see why the reporter package can’t handle the job of printing test results and
process.exit()
-ing (rather than depending on Velocity). That’s much more flexible. - Rather than
testOnly
ordebugOnly
, what would be more sensible and consistent with what other build tools do would be to allow users to decide which environment they wish to use a given package in directly (likedevDependencies
inpackage.json
for instance).
Prior Art
-
@Sanjo has done a lot of work in this direction here already. Potentially most of the work is already done.
-
Also, his other PR does some work towards sharing compilation between different run modes, although it seems to me that a lot of the challenges of Velocity revolved around this sticking point, and as I noted above, it seems a “nice to have”, not a hard requirement.
[1] Which are not packages, and thus don’t have a package.js
to describe their tests.
[2] Or alternatively, modified .meteor/packages
and package.js
to allow adding packages in “test mode”. This would probably be strictly better, but would probably mean changing debugOnly
in the same way, so might be more complicated.
So consider this a RFC – do you think this change, along with things that can happen in package-space, would allow application and module testing? Is there anything missing?
Keep in mind we are trying to make sure this happens, which means keeping it as small as possible! (Competing priorities and all that).
Looking forward to comments…