Previously we’ve discussed a simple design for testing Meteor application code; today we’d like to share a first implementation for feedback.
As part of the work leading up to Meteor 1.3, @avital has built a new command in the tool, meteor test-app. This runs your app in a special mode (see below) and adds a test driver package (currently a Mocha driver built off Ronen Babayoff’s package although in the future it will be fairly simple to make other driver package using different test frameworks) to run the tests and report results.
The command runs your app in “integration test” mode, which means that your app runs as usual with a few differences:
Test files that usually ignored by the tool are included in the build (files with .tests. in the name)
The app runs with a separate test database which is reset on each run (so you can run tests in parallel with your app)
Meteor.isTest is set to true
You can read a little more about our plans for this feature in the early draft of the guide article, including a unit test mode, to be implemented soon.
To try it out, run Meteor from source and check out the app-tests branch. From your app folder (once you’ve defined some Mocha tests), run path/to/meteor test-app.
You can take a look at some example tests in the testing branch of the todos example app.
We’d love you to give it a try and if you have existing Mocha tests written against your app, try them out with this new system. We’d really appreciate any feedback at this early stage.
=> Errors prevented startup:
While building package practicalmeteor:mocha:
error: File not found: runTests.js
=> Your application has errors. Waiting for file change.
when I do meteor test-app with the the app-testing branch of meteor. I wonder where is the modified version of practicalmeteor:mocha published? The last release was 14 days ago.
Now the tests run. But after the first run only the first test result is shown.
Other from that I was mainly interested in the speed, but because it runs from a Meteor checkout it rebuilds a lot more on each restart then necessary. So I cannot really see the real speed right now. (I wonder why local packages are always rebuilt.)
For result reporting the stuff that we have built in velocity:core might still be quiet useful (e.g. report schema) and the reporters. I will probably keep using the reporting part of Velocity in Meteor 1.3 for sanjo:jasmine. If necessary I can split it into a separate package.
How would I run the tests on CI? Is this already implemented?
assert = chai.assert
describe 'Main View Model', ->
vm = undefined
beforeEach ->
vm = Template.hello.createViewModel()
it 'has default properties', ->
assert.equal vm.name(), ''
describe 'canEnter', ->
it 'returns false when name is empty', ->
vm.name ''
assert.isFalse vm.canEnter()
it 'returns true when name is not empty', ->
vm.name 'Alan'
assert.isTrue vm.canEnter()
Then I saw her face, now I’m a believer
Not a trace, of doubt in my mind
Does this mean removing all the collections or the content of these collections? We have some tests that require an index and they tend to fail if the index needs to be rebuild every time (race condition). That happens when collections (or the whole database) are thrown away. We got it working by emptying the collections instead of removing them.
Does this command run all local package tests as well?
You’ll be able to choose which test runner you use, and you’ll be able to create one that prints to stdout and exits with a success or failure error code.
This means that every time you run meteor test-app, it starts with a new database. The database isn’t automatically cleared every time you run your app. You can write your tests with a first step before all others that initializes the database as you need (eg with indices)
I think @jamiter means if a project is composed of packages, would you be able to run the tests for all packages in the project.
Related… haven’t tried it but, would you be able to run the tests for a package within the package folder? Right now with practicalmeteor:mocha you have to add the package to an app and then run the tests from the app folder, not the package folder. Maybe I’m using it wrong…
You run package tests with the same command you always did – meteor test-packages. The new meteor test-app command runs tests in app files, not packages.
As Meteor progresses, there will be less need for packages and apps will transition to using modules.
The one thing that you’ll definitely need packages for for a while is build plugins.
After I see all test results initially, I just have changed a line in a tests file (e.g. add a console.log). Then the tests results only show one test result. I used the todos app: https://github.com/meteor/todos/blob/testing/.
So will the implementation of these two command be very different? In other words: If we move the packages and their tests in to the normal app structure (like the /imports directory, instead of /packages) will they still work? After fixing imports statements of course.
Maybe my real question is about that transition: can we easily migrate from a package structure to an normal app structure. We used the packages mostly for load order, dependency management and separate testing. But with Meteor 1.3 this will be solved with import/export statements. Will most of the test still be useable, even when they were written in Jasmine?
I’m very glad to see this happening because we have been migrating from one test framework to another. First velocity using cucumber and mocha. Then chimp for cucumber, because velocity was dropped. Had to use jasmine for package tests, because mocha was not working with CI. Now that official support is coming, mocha seems to be the standard again.
I’m introducing a new term: Continuous Migration. Can’t wait to see some stability here and official support.
Is chimp still recommended for end-to-end tests in Meteor 1.3?
Conceptually, yes. I mean obviously you need to make sure that you are importing the same way that your package.js was doing things and your test driver needs to support the new system, but I don’t think the way you write the tests themselves will change at all.
