You can go nuts reading tutorials and books showing the most useless examples. Much like the classic OOP example using a car example to describe classes. It gives you the concept, but otherwise useless.
A Gherkin-snippet I found in an article:
When I create 5 widgets
And I delete one widget
Then I should see that one widget disappear
And I should see that “4” widgets remain
I find this overly complex testing unless you’re specifically targeting animations and user experience.
When making a Meteor-app you can test for everything from access to z-index(A to Z… get it? ).
But writing those tests take far more time than they’re worth. Where do you draw the line and why?
In a client/consultant setting I totally see the whole cucumber/gherkin solution. Agree on the tests, make them pass. Job done.
When you make your own app or you work continuously for a company with an app, the picture is slightly different.
Sure, you can write tests to ensure the next guy doesn’t mess things up.
You can go bananas with Unit, E2E, Integration testing.
Or you can just completely avoid it and use manual testing.
I’m completely baffled by either my inability to find it, or the lack of resources as to what one should test in a node app instead of just how.
I’d like to know what and why. The internet is however very helpful as for the how!
Does anyone have any good resources? Either completely general, or directed at webapps/node.
Funny, right now I’m writing a small testing library for Angular 2 to let me test Angular components like I test ViewModel ones.
The gist of it is that my confidence level is very high if I have 1) unit tests for each property and method of the component, and 2) tests that make sure the bindings/directives are properly set in the markup. If I have that then the end to end tests are just to confirm the application is working as a whole (a few happy and unhappy paths here and there). No need to go through all permutations of the screens.
‘Angular 2’ ??? did I get on the wrong train? I got on the ViewModel ‘react’ train recently … should I be planning a transfer?
Thanks for all your work!
At my company we’re creating a new application using Angular 2 (it’s “Enterprise Approved”). I think the ViewModel way of testing components is very good so I’m trying to test Angular 2 components the same way.
My next project will definitely be with ViewModel for React and I’m giving it all the love I have.
The problem with unit tests is that they’re tied to an implementation. They’re great if you know you’re going to stick with one framework or technology stack for the life of the application; but swap out your view layer or database? Uh oh, things fall apart.
That’s why we always start with a critical-path walkthrough of the app. Call it acceptance testing or validation testing or end-to-end testing or whatever. Just as long as it’s Selenium based, involves a 'bot simulating a user, and isn’t tied to the underlying implementation. For example… our weblog CRUD walkthrough has been walked from PHP Lithium to Spark to Blaze and most recently to React. Because our tests aren’t tied to a specific implementation, they’ve been able to handle major architectural changes (including Apache to Node in one instance!).
Each user type and role should have a critical-path walkthrough. It should be scripted to check for the underlying business value; and the parts that business stakeholders have agreed shouldn’t change. If the stakeholders can’t agree to what shouldn’t be changed, then they don’t understand what their product is.
Accounts login and user onramping should have its own walkthrough. Amazing at how people can’t agree to some of those basics and fundamentals, and keep reinventing the wheel with the accounts system. And invariably it turns out more complicated than they expect, something blocks the user onramp path, and the app fails to go into production because people can’t log in, change their passwords, etc. If you write it correctly, you can then reuse the script for load testing and stress testing.
After that, boundary testing inputs is always a good idea. Best done with unit tests, not acceptance tests.
And then writing unit tests for your business logic. This step can go first if you’ve got a clear idea of an algorithm or business logic you want to write. But there shouldn’t be any UI code or third-party dependencies involved. If it’s not completely self-contained, don’t write unit tests yet.
We like to spot check the app with common user profiles. Since we do a lot of healthcare apps; it’s common to have ‘the allergy patient profile’ or the ‘pregnancy profile’ or the ‘geriatric profile’ etc. We’ll test users can submit/complete the correct froms for each user profile, and that results are calculated as expected. It’s a form of boundary testing and integration testing, which works pretty well.
I disagree to some extent that unit tests are automatically tied to an implementation. With the right setup you can blackbox-test it. Pass input, check output. Though it does become harder to make it a true unit test.
Then again it often makes sense to tie it to the implementation as much as possible. A Blaze UI is designed quite differently from a React UI for instance.
Passing something through selenium, wouldn’t that require you to “hold on” to stuff like classnames and id throughout implementations? If not, I’d love to see an example of how you avoid it.
How do you write those tests to account for completely useless users? As a dev I have a sense of how the app should be traversed. Order of things that “just makes sense and would be silly to do otherwise” without thinking about it.
If you are allowed to, I would love to see snippets of examples from actual production code, however I assume an NDA would block you
@greblak
There is a method to the madness, and I have come to discover that the majority if testing is an absolute science.
You start with the premise of “I want to design a system”, and you use specifications to describe how the system will behave. You then automate those specifications to execute against a codebase to ensure the design is working as it should be now, and in the future as other developers change it.
In one way, the tests are a means of communication of the original intent of the system. In another way, they are an antidote to regression.
End-to-end, integration and units are not exclusive. You can use them together to get the perfect balance of test coverage. The combination of BDD + outside in testing is your friend to produce this.
Beware that much of the “BDD” literature out there is actually rubbish! If you ever see a feature file containing any reference to the UI, such as click and what elements appear / disappear (as in your example), then stay clear.
True BDD is about discovering the domain layer, and making it match the real business domain. The litmus test to apply to Gherkin specs is to ask “if it’s 1912, there are no computers and everyone is blind, will the scenarios make sense?” If so, you have a BDD spec.
What most people try to do is to apply it to the UI, because they think it’s nice to use “natural language”, but what they end up doing is creating a pile of tech debt that needs to be thrown away after 6 months. See here as I’ve written in detail about it here
Rule of thumb is:
Use Stories + Scenarios to describe the business domain. (see here)
Use unit tests with UI components (like React + Enzyme + Jasmine/Mocha)
Apply an extreme level modularity using the separation of concerns principle
That sure cleared up a lot around BDD
And should indeed help it survive changes later. Sure, the tests will change with internal changes, but the feature-file doesn’t have to!
I do however feel I often end up writing stupid and unnecessary tests. They may be useful for regression testing, but testing the simplest behavior for a component(For example relatively correct rendering, or checking to see if the onChange or onClick fires the correct function) is slightly redundant.
What I’m currently landing on practically for a Meteor + React app:
Unit test React components for expected behavior for a series of inputs.
Unit test schema-validation with a wide range of inputs
Unit test server methods (However, I don’t really like having to do DB-calls to do this. Any other way?)
E2E / Integration test general expected user behavior with a mix of “just simply dumb” behavior.
Am I missing something? Any of this redundant?
How’s that book of yours coming along by the way? Pre-ordered it a while back. Still waiting
).