Testing : General Setup Question on OSX

testing
1

I’m trying to get some measure of BDD running for my project. Technical debt and all that. Ideally, what I would like is a setup that gives me as close to the equivalent of rSpec/Cucumber in Rails. On OS X.

Each time I do this, I run into some snag that simply makes me give up. I know that’s bad behavior, but I’ve invested quite a bit of time into getting something running and I’m coming up empty.

Here’s what I’d like to accomplish:

  1. I don’t really need the keen browser overlay that Velocity gives me – in fact, it gets in the way of the login stuff I put at the top right of the screen.

  2. I want to be able to write Given/When/Then tests but haven’t come up with the proper installation-fu to get Cuke running. I keep winding up with incompatibilities. Is there a known good set of steps to install this? I’ve tried xolvio:meteor-cucumber and it seems promising but I can’t get past the first step of getting the deliberately failing test example to build. I get type errors with World and then with mirror. The example code on Github seems to have the same issues. I installed according to this on Github.

  3. I’ve tried mike:mocha and sanjo:jasmine for unit testing and found tests designed to fail often pass and tests that should pass sometimes fail. Jasmine seems more full-featured, but mocha with sinon seem like they almost exactly approximate it. Again, it seems difficult to trust tests that are not going red when they should and then transitioning to green when fixed.

I’ve done a ton of testing with rSpec and Cucumber and just want to carry this over into my Meteor development, but strangely, can’t get the test harness working.

Am I being too vague here or is anyone else experiencing issues getting test tools installed? What to do…?

2

I can help with this. Can you tell me your exact issues?

FYI Velocity is currently undergoing some major rework and the new version out this week will no longer show false positives/negatives in jasmine/mocha.

3

@sam – thanks! I will certainly have issues as I go, but my most immediate one is the one of false positives. A secondary one is that Velocity seems to rely on a manifest of test files, but they are not automatically cleaned out when the file is renamed or disappears. So, the file-not-found thing results.

The second thing is that the pulsing blue concentric circle thing is cool, but not a great progress indicator. If there could be some kind of console logging that showed where in the testing process we are, that would be terrific.

Third thing: Completely headless automatic CI. I’d love tests to run automagically in a console window every time I change a file. I prefer not to run them in browsers until I absolutely have to. Am I wanting too much here?

Finally E2E. The best shot at this right now seems to be Cucumber (and I’m veeerry comfortable with Cuke). It suffers from a number of the same issues I’ve outlined above. But even more, it appears to be migrating from using World to encapsulate everything to using this. But I’m not even able to get the most basic (the example) step to pass.

I will grant you that Ruby and Rails are somewhat opinionated. Rails more so than Ruby. But to DHH’s credit, he made sure that Test::Unit files were created by the Rails generators and that you could run a working set of tests right out of the box. I don’t feel Meteor/Velocity is at that point, yet the apps people are creating can be vastly more complex in some ways than a straightforward Rails app.

I suggest injecting some opinion into Velocity and saying: “If you want to do unit tests, use XYZ, which you can install completely by {some set of steps}”. Actually, better if tests are installed as base packages.

Further, I suggest E2E be embraced. There’s a divide between Nightwatch and Cucumber. Neither seems to work flawlessly and the failure causes are often obscure. This suggests some writing. Articles would be wonderful. Everything out there now is more than a bit dated.

I’m not sure I’ve narrowed things down any for you, but these are my thoughts. I should say that I’m really impressed with what’s there now. I’m just not getting the install right.

Clarification:

I keep winding up with incompatibilities

This refers to what is apparently a shift from the World encapsulation to stuffing things in this. Basically, the package, rc8, does not generate an example that fails as advertised in the README.

4

Hey, thanks for reporting. I’ve just released 0.6.0 and fixed the issue you were having. Could you try again and let me know please?

5

Sorry. I’ve been completely out of Internet range for over a week. There are still places where the Internet is hard to get to. I’ve seen that there is a healthy discussion of these exact issues and that you’ve addressed this so I plan to dedicate some time to installing Cucumber again and then Jasmine. I’m hoping that stepwise approach gets me a broad coverage (Cuke) and functional/unit (Jasmine) set of test suites. Baby steps.