One thing that’s a plus… if you’re using ES6 modules (webpack for now, core later) the outlook is really nice!
You can use standard JavaScript testing tools to unit test your code. It’s super fast too because you don’t go through the build step… i’m getting sub 1 second pass/fail specs with Karma + Jasmine in Chrome.
Wrapping your data layer with something like a repository pattern makes things much easier to unit test. Most Meteor bits that i’ve ran into can be stubbed with a simple Jasmine stub (opting for exportable functions instead of anonymous funcs helps too).
End to end tests don’t require velocity either so those don’t have to go through Velocity either. I used to use Rspec + Capybara for this but i’m looking into Chimp now as an alternative for cucumber.
The remaining bits could be tested with integration tests via Velocity but IMHO if you have high coverage with unit and end-to-end than your overall confidence should be high. Even if you have a few integration tests for publications you can still run Velocity in CI mode as a sort of smoke test instead of watching continuously.
@sam is there a date (timeframe) you can share for the release of the testing manual?
I remember pre-paying for a testing book about (probably more than) a year ago and receiving an email about that book getting folded into the testing manual. But no more news ever since
Thank you Xolv.io for all that you have done for meteor testing.
Now, i’m a litlle worried by this announcement.
In July 2015, MDG announced their plans to "to move Velocity into core and simplify and streamline it."
Today Xolv.io is announcing that they are “handing back the responsibility of the “Official Testing Framework” to the MDG.”
I may be wrong on this, but for me, since this is a Xolv.io only announcement, it means that Xolv.io is tired of waiting for MDG to begin work on testing in Meteor.
In mars of 2014, before 1.0 which was to be released without testing support, i said that MDG seemed to have no real interest in testing. Someone from MDG responded that they were “commited” to testing and that a more relevant question would be “when” they would get to this topic. One year and a half later, i still stand by my comment.
You should have access to it online if you purchased the Testing Meteor book I was working on. If you didn’t get that access, feel free to contact @sam directly to handle that.
And I am sure they still stand by their comment too. I think this is a time/people issue more than anything. The real tragedy here is that you had a group of community people interested in doing something and MDG did a poor job of helping them achieve this goal…
Couldn’t agree more with both of the statements. Wrapping and introducing indirection is the best approach for decoupling for any framework, with the caveat that the indirection could be misconfigured and the unit / end-to-end tests would still pass.
Way I see it, E2E tests ensure the value to the user is being met. Integration tests make sure you wired up and configured your components together. Unit tests validate the logic.
First off, thanks so much for leading the charge in the community on testing and Meteor. I think you and the others in the Velocity team have done a great job of picking up the considerable slack.
Also, thanks to you and Jonas for helping and advising Sashko and myself with the guide and especially with the testing article. I’m still trying to get a complete picture of everything that’s out there, and it’s been very helpful to hear from you guys, as you are clearly at the forefront of the space.
Something that we think will come out of the guide is a good picture of what the landscape looks like today (with testing and a lot of other areas of Meteor). Of course with the advent of ES2015 modules, and the potential for webpack integration, that landscape is in flux for testing in particular. But I hope and expect this will lead to some obvious low-hanging fruit that can be picked off both by the community and by us at MDG.
With that in mind, it’s great to hear these war stories about Velocity, and I’d love to hear more about some of the lessons learned, and your thoughts on the best ways forward. For instance, I’d love your comments on this ticket.
I think the lack of consideration for testing and modularity in the current version of Meteor has been a huge problem. Any serious software engineering requires these things so that a team of people can work together on a project without worrying that each change might break something. Refactoring becomes nearly impossible without the safety harness of good tests.
In my mind it’s a side effect of the way Meteor’s development has historically emphasized a low-boilerplate and easy to get started with design, but at huge expense in terms of encouraging good software engineering practices.
I believe the Meteor Guide is a big step in the right direction - the intent is to describe all of the things one would need to do to build a “serious” Meteor app, and I believe that it can unshackle Meteor from the world of 1-hour long getting started tutorials and remind people to think about app organization, maintenance, and reliability. Perhaps the first version won’t have a perfect testing story, but at least there will be some obvious gaps that could drive development of a fix.
Another great step will be the introduction of a modern JavaScript module system in Meteor 1.3. Hopefully soon after we can rewrite large parts of Meteor to be less global and magical, so that you can stub out modules or some of their imports or exports. I hope this will also enable the Meteor community to use more off-the-shelf JavaScript testing tools that know more about how to deal with these modules - for example, Jest or Istanbul.
I will surely comment. My intention with this thread was to provide the feedback in a retrospective format so that the MDG can run with. We should have a hangout to give you a good download of the lessons learnt - I think that will help you a lot with creating the testing app story. We have a lot of demos and use-cases we can share.
[quote=“sashko, post:15, topic:13117”]
Another great step will be the introduction of a modern JavaScript module system in Meteor 1.3. Hopefully soon after we can rewrite large parts of Meteor to be less global and magical, so that you can stub out modules or some of their imports or exports. [/quote]
Great to hear about 1.3 bringing modularity, we’ve definitely been waiting for it and it will be an enabler for unit testing. All positive things.
I just want to highlight that supporting test environments is still necessary. Not only in the interim to cover the case of testing of not-yet-decoupled code, but also for integration testing in general as noted in my comment above:
Currently the only way to do integration testing is to use a Velocity compatible integration testing framework, and this will continue to be the case after the modularity.
In the Java world for example, I can run a build that only includes my integration-test using Maven profiles. The same is possible in many other build systems like msbuild, ant, nant, you name it!
For E2E testing, test environments are not essential but they are still highly desirable. I would want to include packages that provide me back-door access in an e2e testing environment but not to production. Interestingly, it’s easy to do that in TinyTest / Velocity package testing using the onTest api.
What I’m really saying here is that the build system is another accomplice to the difficulties in testing. The current Velocity tracked release of Meteor fixes this by allowing mirrors to compile code inside the tests directory for a given framework, and by outputting the compiled files to another directory. Merging this into the core is probably the quickest win to getting integration testing support, but I think the long term solution should be to upgrade the build system, or to use an industry standard one.
Anyway, excited to see the modularity refactor. If this includes a revamp of the build system, then hallelujah and in both case, I will soon be happier than now
First of all thanks for all your great efforts! I just started to learn velocity and jasmine, so I’m not sure where should I ask my questions. I already wrote bunch of unit and integration tests and now, when velocity docs are shut down - what should I do next? Should I start looking at some another framework? If currently I should stick with velocity - is there any chance to get all docs and examples from https://velocity.readme.io , cause they were very helpful.
And btw, is there any ETA when MDG will start support their own testing framework?
We continue to maintain sanjo:jasmine and xolvio:cucumber, because we use it. The documentation is still available under new URLs. The new locations are linked here: https://velocity.readme.io/.
I can’t tell you what the MDG will support and when, but my advice is to continue writing tests using Jasmine until the MDG/community come up with a better solution.
As discussed above, the main gap of Meteor testing that Velocity plugs, is the ability to do app-level unit / integration testing, and as Sanjo said, we will continue to use our own Velocity-based testing frameworks in the meantime and share with the community. Just be mindful that it’s not the “official testing framework”.
The good thing is that there are two main testing tools in the JS community: Mocha and Jasmine. I can only imagine one of those will be picked in the long run, and to convert your tests from Jasmine to Mocha is very easy, it’s just a little syntax.
On the plus side, app-level end-to-end testing is completely within our control since it does not require any changes to the Meteor core. We are focusing on this with our Chimp npm package that already works without Velocity. You can run it against either your main app, or start your own parallel app (mirror) from the command line, then start Chimp this way:
chimp --watch --ddp=http://localhost:3000
Chimp downloads Selenium for you, initializes a webdriver connection, initializes a DDP connection, fiberizes everything so you have no callbacks, no promises, adds a watch mode to any features you tag with @watch. All this means you can just write tests in your favorite testing language. Currently it works with Mocha & Cucumber and we’ll be adding Jasmine. I’ll make a fuller announcement + tutorial soon.
I think the Meteor community deserves more attention from the MDG. They are not the most communicative group I’ve ever seen.
They must say something about this. It’s really a chaos trying to test with Meteor, no official docs, nothing. If you compare this to RoR or Django…
