Awesome! Thank you, @sashko
1 Like
Chimp works really really well for end to end testing, they even have some meteor specific hooks you can use.
1 Like
As does StarryNight, Nightwatch, and Gagarin.
Do you have references where I can read about some HR departments which are uncomfortable with testing frameworks that use these terms and/or names? I tried a bit of googling and couldn’t find anything.
4 Likes
I think it’s a slippery slope to put a lot of weight on how things sound. StarryNight sounds like a child’s tale, Nightwatch a cartoon. Cucumber is a vegetable. Considering spying inappropriate terminology seems super subjective.
Having said that it seems more than fair that the author of Nightwatch deserves an answer on why Chimp would be officially recommended.
It also seems a Guide recommendation carries very little weight if it goes against the officially MDG blessed Velocity.
2 Likes
Looks like Velocity is no longer being maintained. One of the main maintainers is now running Chimp as their main testing project:
I would be interested in hearing about the advantages of other tools over Chimp.
As of 1.3, I believe the Meteor Guide is going to be the official record of “MDG recommended”. We didn’t have any mechanism of recording or updating that before.
We (Xolv.io) are keeping Velocity alive until the MDG solution for testing takes full effect. You can already see on this issue that the MDG are actually working on an app-testing solution now. @funkyeah Velocity is no longer the officially blessed testing solution, but this new app-testing mode will be as official as it gets. Once the new app-testing is ready, and depending on the exact final implementation, we may invest time to port the sanjo:jasmine package to use it instead of the Velocity mirrors.
If you take Mirrors away from Velocity, then all that’s left is a Meteor package that contains a reports collection so it becomes a reactive test reporter. Maybe that’s Velocity’s destiny, who knows.
For end-to-end testing, we’ve been HEAVILY investing in Chimp. You can use either Mocha or Cucumber, depending on your preference. We’ve been focusing entirely on the developer experience and are seeing some great traction over the past few months so it’s very encouraging.
@sam, I’m interested in learning more about your prioritization of Meteor testing vs. wider JavaScript-based webapp testing. Do you have plans to promote this solution to the wider JS community somehow?
This situation irritates me more than I have words for, but I’m withdrawing my remarks to avoid a flame war. Suffice it to say that I’ll be releasing an update to StarryNight this weekend, and posting detailed responses to various discussion threads on the guide and elsewhere.
7 Likes
That sounds great. Also, I think @tmeasday is the primary author for the upcoming testing guide, so I could be getting some things wrong.
Absolutely, we’ll be doing some marketing for it soon as it’s inline with our top secret business plans (which will soon no longer be top secret). We’re creating a couple of sample projects with a Meteor React app, and probably a MEAN stack app to allow developers to learn by way of example.
It will demonstrate how to write subcutaneous tests that bypass the UI and talk directly to the domain using server.execute, or client.execute. This is a great way to tease out the domain model of an app without thinking about the UI at all (DDD with testing basically). Once the domain works, then you attach a UI to it like a joystick. The coolest thing is that you can use an @ui tag on a few happy-path scenarios, and write some UI testing code with some polymorphism to use WebdriverIO instead of controlling the domain, and thereby sticking the testing pyramid. Neat eh?
NOTE: Chimp is, and always will be fully open source 
5 Likes
Typical. This kind of language is what drives women away from the tech industry.
4 Likes
@awatson1978 Just Chill out and ignore this! it’s not like everyone will be using Cucumber or Chimp …
I already hate them loool since they suck and would never use them ever 
Respect!
1 Like
There is such a thing as context, I’m not exactly sure how someone could interpret those words as being offensive based purely on the words themselves. “sticking”, seriously?
7 Likes
subcutaneous means situated or applied under the skin.
and so on!
and you should never assume something won’t affect someone because you think so!
1 Like
Thanks @thanson for bringing this topic up.
I am also looking for a good testing solution for my apps. I tried Velocity twice, but always had issues with it. Also, it was quite slow.
So I welcome that there are alternatives like Chimp and Starrynight, but to be honest, I am a little “lost in hyperspace”, since I wasn’t able to invest much time so far to invest in research regarding all of those different frameworks. Hence, I would especially welcome any recommendation and practical tips in the upcoming Meteor guide!
As far as I understood the products so far, Chimp wants to be a simple yet powerful one-stop solution to get you easy up and running. It supports Cucumber and Mocha, but no Jasmine so far. If I read the docs correctly (just scanned them roughly), Mocha seems to be an alternative to Jasmine, so this would not be a bummer for me, as I have to start from scratch anyways. Is my assumption correct?
My primary first goal would be to sprinkle some tests here and there, to allow for regression testing of my model classes. So I don’t really need the full complexity of mimicking end-users in a web-browser (yet). This is what StarryNight seems to target at, using a Selenium-based approach. Plus, as far as I can tell from the docs, StarryNight wants to be way more than “just” a testing framework, so I got the impression it’s a bit too “overweight” / sophisticated for my purposes. I need a simple solution, I’m simple-minded
3 Likes
Yep, you got it. One thing to add is that Chimp also handles web-browser testing using Selenium, in fact - that’s how client interactions are handled. And if you prefer Phantom, you can just use --browser=phantomjs
Thanks for your reply.
After digging a (little) bit deeper into the chimp and Mocha docs, I’m a bit confused, TBH. I expected that I could operate directly on Meteor objects, like I knew it from Velocity and the sanjo:jasmine package. But from what I saw in the docs, the Mocha integration in Chimp seems to primarily target end-user-like testing in the browser, by mimicking end-user interactions. I saw that there’s a way to call functions on the Meteor server side via server.execute; but that seems to be much more complicated than just calling the Meteor methods directly. Will this be possible with the upcoming Jasmine integration, or is this a side-effect of not being so tightly integrated with Meteor as Velocity was? Maybe that’s a dumb question, but all that testing stuff is quite new to me… :-/
You did not mention Velocity in your reply. Is Velocity still the “Official Testing Framework”, as announced here:
http://xolv.io/blog-posts/2015/11/8/xkybpw2cdvg6ytncjbh7hmvu5f9jbw
and Chimp is “just” your recommendation for a certain type of testing (i.e. ‘end-to-end-testinng’)?
Or did MDG change their plans in the meantime? Sorry, but I am a bit confused, because the OP asked about the future of Velocity in particular.
My answer has multiple parts, and it goes like this:
- Starting with Meteor 1.3 the Meteor guide will be the record of “official recommendations”. So if Velocity doesn’t end up in there it’s safe to say it won’t be the official testing framework anymore - and when we release that content we’ll pair it with a guide for what people who are currently using Velocity should do.
- The testing guide is not done yet, and @tmeasday is working on it, so he would be the best person to ask.
- Velocity has run into some problems, and MDG currently doesn’t have the bandwidth to maintain the set of features Velocity in particular has committed to.
I believe our plan is to build/recommend just enough for Meteor 1.3 that it covers all of the requirements for testing an app - unit tests, integration tests, and end-to-end tests for every part of a relatively normal Meteor app.
On a personal level, here’s my thought: Velocity had very ambitious goals like connecting a variety of different test runners and reporters, reactivity for all parts of the system, merging the results of all kinds of tests into one output, and more. I think if it had more limited goals it would have been easier to build and maintain. That’s why I think Sam’s new project Chimp is great - it looks like it is focusing on getting just one part of the testing story right, and in a way that is minimally Meteor-specific.