This diagram depicts the sets of unit tests that exist.
What's immediately obvious is that tests are either considered useful or useless. There is no common ground. It's obvious that the set of useful unit tests is smaller than the set of useless unit tests, but not by much.
Also you can see that you only need a fraction of the tests to get maximum code coverage, the set in blue. This set is a fraction of the total test set in either set.
What's not obvious is that the set of useful tests are more difficult to write because they require a good understanding of the problem. While the set of useless unit tests is relatively easy to develop because they require an understanding of the code. For most developers, it's far easier and less hassle to focus on the code, as opposed to the problem we are trying to solve.
If you just mandate developers to write unit tests, without much support from the people who specify the problem - inevitably they will end up writing tests from the set on the right. These tests, while you think you want them, are pure waste. Extra lines of code. At worst you have to maintain them and at best have to be aware of...
If you then mandate your developers to target code coverage, they will focus on the subset in blue, in the set on the right. This isn't the worst thing in the world. It means you will write less wasteful tests... thats' not a bad thing. However they are still waste.
If you have developers who are mature and experienced enough to write their tests in the set on the left, then you are really lucky. They are a very rare resource. The more of this set we complete the better. We will have less bugs in the future.
When you set a coverage target when writing great tests, there is a danger you write the least valuable tests. The 80:20 rule tells us that 20% of our code is run 80% of the time by our users. Or that users need just 20% of a component, 80% of their time. Then thinking of our users, we need to have tests to verify this 20% of the code base works, all the time. In other words you get most value ignoring 80% of the code base.
So if time is a limiting factor and you are writing good tests, targeting code coverage isn't a good idea because it means that you are writing the least valuable tests.
No comments:
Post a Comment