Coverage on meteor

packages
8

I’m working on a fork of spacejam that can execute all the test-packages of a meteor app and then export the lcov file:


For the moment, the report contains the coverage of all files prototyped (~ 150 files), I need to clean that and figure it out what is required by coveralls.io to accept a coverage report.
Right now, you can add my package to your meteor app and looks the web report at localhost:3000/coverage , tell me if it’s ok for you.

2 Likes
9

Tried… failed :frowning:

The behaviour I was expecting…

meteor add lmieulet:meteor-coverage
meteor test --driver-package practicalmeteor:mocha --port 3100
http://localhost:3100/ to see the test run
http://localhost:3100/coverage to see coverage ... no luck

Other behaviour I was expecting

meteor add lmieulet:meteor-coverage
meteor 
http://localhost:3000/ to see the application
http://localhost:3000/coverage to see coverage ... no luck UI router redirects to the login screen
logged in
http://localhost:3000/coverage to see coverage ... no luck UI router redirects to not found page
1 Like
10

My package is not enabled unless the environment variable COVERAGE=1 is set up
On Unix, you can test with the following :

COVERAGE=1
COVERAGE_VERBOSE=1
COVERAGE_APP_FOLDER=/path/to/your/meteor/app/ meteor

Windows:

set COVERAGE=1
…
meteor

Then, you can look the report: http://localhost:3000/coverage
The last things you can try is to send the client coverage to the server. Open the console tab of your browser and execute that line:

Meteor.sendCoverage(function(stats,err) {console.log(stats,err);});

Now on the report http://localhost:3000/coverage you see the client coverage.

And I just tested with a basic application (meteor create app) and it works as expected with meteor add practicalmeteor:mocha && meteor test --driver-package practicalmeteor:mocha. It outputs me the following global stat of lines executed:

30.57% Statements 9024/29520
16.41% Branches 2991/18227
25.36% Functions 1373/5415

1 Like
11

Cool, got it working. Though I only see the server side code being covered and the PATH is kind of off (SET/ENV/COVERAGE_APP_FOLDER/OR/READ/README/) :wink:

It’s a good start.

I think for non-production we should be okay to have /coverage enabled and have it turned off when Meteor.isProduction much like how I did it http://www.trajano.net/2016/04/minor-performance-boost-for-angularjs-and-meteor/

I did do the Meteor.sendCoverage(function(stats,err) {console.log(stats,err);}); and now I see the the client coverage. Too many things there though. I think a reasonable default would be just anything that is in packages, .meteor, meteor home and node_modules should be removed from the list.

As far as COVERAGE_APP_FOLDER=/path/to/your/meteor/app/ perhaps you can do it by relative path much like how I did it here http://www.trajano.net/2016/04/publishing-meteor-application-api-documentation/ I just put a console.log(process.cwd()) to find out where I am then just traversed from there.

1 Like
12

Cool, got it working. Though I only see the server side code being covered and the PATH is kind of off (SET/ENV/COVERAGE_APP_FOLDER/OR/READ/README/) :wink:

Do you have setup COVERAGE_APP_FOLDER ? It looks like the package cannot find process.env.COVERAGE_APP_FOLDER :confused:

I think for non-production we should be okay to have /coverage enabled and have it turned off when Meteor.isProduction much like how I did it http://www.trajano.net/2016/04/minor-performance-boost-for-angularjs-and-meteor/1

This package is debug only:

The debugOnly property tells Meteor’s build process to exempt a package from your production code.

So it’s even better than testing Meteor.isProduction !

I did do the Meteor.sendCoverage(function(stats,err) {console.log(stats,err);}); and now I see the the client coverage. Too many things there though. I think a reasonable default would be just anything that is in packages, .meteor, meteor home and node_modules should be removed from the list.

Work in progress !

As far as COVERAGE_APP_FOLDER=/path/to/your/meteor/app/ perhaps you can do it by relative path much like how I did it here http://www.trajano.net/2016/04/publishing-meteor-application-api-documentation/1 I just put a console.log(process.cwd()) to find out where I am then just traversed from there.

This cannot be reproduceable every where on every OS, sometimes Meteor stores the build outside of the main project directory (I have no reproducible environment to provide, but I observed it several times). There are several usage of the COVERAGE_APP_FOLDER inside the package, and right now I cannot say to you if it’s possible to eliminate this dependency.

With the fork of spacejam I’ve created, this environment configuration is created automaticaly, so you just ask spacejam to test your application and print the coverage in a file.

1 Like
13

Changes 0.5.0

  • edit the Travis CI conf file
    • Using a fork of spacejam, it runs tests, saves the coverage inside a lcov.info file and using node-coveralls it sends the coverage to coveralls.
  • add the coverage img to the repo ! Coverage Status
    I think this is the first meteor package that has one :slight_smile:
    Note: the server code of this package is not covered at all because on runtime the server reads and executes the meteor-coverage server js and after that it starts to instrument every single js that will be read.
  • fix file paths when meteor test-packages inside a package (necessary for covering itself)
  • clean & other fix…

Next

Check if any meteor package that uses Tinytest and Travis can run their tests on travis and export coverage.
Support for meteor app coverage.

3 Likes
14

Changes 0.7.1

  • remove underscore dependency: I tryed to instrument a real package and had severe issue with this dependency. The simple fix was to remove all reference to underscore.
  • works well with test-packages
  • add eslint

So here is real example of meteor package coverage:


Coverage Status

Here is an example of .travis.yml:

language: node_js

node_js:
  - 0.10.40

cache:
  directories:
    - $HOME/.meteor
    - $HOME/.npm

before_cache:
  - rm -f $HOME/.meteor/log/*.log

before_install:
  # Download Meteor
  - PATH=$PATH:$HOME/.meteor
  - if [ ! -e $HOME/.meteor/meteor ]; then curl https://install.meteor.com | sh; fi

  # Install spacejam
  - npm install -g https://github.com/serut/spacejam/tarball/master
  - npm install -g https://github.com/nickmerwin/node-coveralls/tarball/master
  - npm install -g eslint

script:
    # Check clean code
  - eslint .
    # Run packages tests
  - spacejam test-packages ./ --coverage && cat lcov.info | coveralls

You can try it with your package, but keep in mind that this .travis.yml file is not the final version.

Next

Do the same thing for meteor test & meteor test --full-app

1 Like
15

16% coverage? not too impressed… j/k

I see you use spacejam in your travis, but you’re doing test-packages, can it work with the main app?

16

Right now spacejam doesn’t support the meteor test command, but the pull request is there : https://github.com/practicalmeteor/spacejam/pull/51.

17

oh suX0r

r0x0r. Hopefully it gets merged soon.

1 Like
18

Fantastic work, @serutan. I added the coverage package to the Meteor Chef Base, and it picked up the .test[s].js files with no problems.

1257×743 98.2 KB

3 Likes
19

Changes 0.7.2

  • Handle the server coverage of this package when testing so now it’s covers 100% of the meteor-coverage files (client and server side)

I create a sample repository to show how to test using spacejam and this package: https://github.com/serut/meteor-coverage-app-exemple
It supports:

  • Test package using tinytest
  • Test package using mocha
  • Test app using mocha
  • Test full-app using mocha

I made several pull request to spacejam: https://github.com/practicalmeteor/spacejam in order to merge my changes into the main repository and let developpers embrace coverage soon and easily.

3 Likes
20

do you know how spacejam relates to meteor test?

21

spacejam test
==> meteor test --driver-package=test-in-console --port 4096

spacejam test-packages
==> meteor test-packages --driver-package=test-in-console --port 4096

spacejam [] --driver-package=practicalmeteor:mocha-console-runner
==> meteor [] --driver-package=practicalmeteor:mocha-console-runner --port 4096

If you want further information about spacejam, run it with the option --loglevel debug. I will print a lot of informations.

2 Likes
22

Changes 0.8.0

  • Fix a severe issue with configuration file - it was not working at all
  • Add a new entry to the configuration file to ignore files from report using the key others
  • Add tests to the package
  • Add simplicity to configuration file, if you do not define an entry the default one will be used

Now you can define that every files in a tests folder are ignored (issue https://github.com/serut/meteor-coverage/issues/3#issuecomment-227007143), which is exactly what the package needed to exclude from coveralls the coverage of tests files

The coverage report is also compatible with codecov.io :
https://codecov.io/gh/serut/meteor-coverage
codecov

23

Not working right now on meteor 1.3.4

serut/spacejam + practicalmeteor:mocha-console-runner is not working anymore on travis, but a fork of https://github.com/DispatchMe/meteor-mocha-phantomjs can simplify everything : the test runner exports the coverage report when the job is done and tests ok

I’ve done that before, so the future meteor-mocha-phantomjs-and-coverage will need to implement this kind of logic : https://gist.github.com/serut/7872af9d2beac8c12a417b6c91171fba

24

How do I run it to generate report? I tried it few times but the only result I get is the empty ~11% coverage report. README is missing the information how to record the tests coverage. It says only how to generate the report.

25

To answer your question, I will need further informations:

  • How can you say you have an empty coverage if you have 11% of coverage (on the /coverage url ?)
  • To record the coverage of your app - we say instrument in the istanbul.js world, you just need to add my package as a dependency. The working exemple can help you to add such dependency (https://github.com/serut/meteor-coverage-app-exemple)
  • You want to save the coverage on a file or display it on your console ? In that case you need to use a test runner, but
  • CI integration is not working right now, so there are no working way to do that
  • Anyway, run your test, send your client coverage to the server (if you have) and open the report (/coverage) and let me know exactly what you want to do
26

By ‘empty coverage’ I meant the tests are not run and it’s showing executed lines of code during the app load.
So now I found out that I need to run coverage options during the tests, not after which I was trying before.
Now I’m trying to figure out how to merge coverage results from each test level (unit and integration). I know how to export it to file but not sure how to merge the results, because they are overwritten after each export. It is possible only via Coveralls.io, if I understood it correctly?

27

All right ! You need to execute your application with all the coverage configuration if you want your project to be instrumented and to get the coverage report

Merge coverage results is a big issue right now (pull request are welcome by the way). It can be done using our API but right now no test runner has this feature. If I want to do it by myself, I will do:
// run unit tests
Meteor.exportCoverage(‘coverage’, function(err) {console.log(err)});
// stop the server
// run integration tests server and before tests
Meteor.importCoverage(function(err) {console.log(err)})
// run tests - here the question is : does importCoverage merge or overwrite the current coverage report ?
Meteor.exportCoverage(‘lcov’, function(err) {console.log(err)});
// send coverage to coveralls
cat lcov.info | coveralls

I don’t know if you can send 2 coverage reports to coveralls and if they merge it. It is a good question.

1 Like