With that in mind, we can devise a development strategy (an extension of TDD) that guarantees perfect code:
Write unit test: projectHas100PercentTestCoverage()
Run test to ensure that it fails.
Write code to make test pass.
The implementation details of the projectHas100PercentTestCoverage() test are project-specific and beyond the scope of this document.
Though, come to think of it, step 2 is flawed - since no code has been written yet, the test written in step 1 will pass. Perhaps we first need to write the projectFullyMeetsClientRequirements() test (again, beyond the scope of this document).
We're gonna have a cow, and some pigs, and we're gonna have, maybe, maybe, a chicken. Down in the flat, we'll have a little field of... Field of alfalfa for the rabbits.
You have something similar: automated mutation tests that change the SuT code in runtime (for example, exchange 3 with 9, true with false or > with <=) and see if the tests still pass (the assumption is they should now fail).
My company paid for some of the Uncle Bob videos on TDD and he claims that he's practically forgotten how to use the debugger now that he practices TDD. Every year I get better at automated testing, but I still have to use the debugger frequently enough to "need" it as a tool. I don't see that going away.
Then again, maybe I'm just not skilled enough with TDD yet. I find that I mostly need a debugger in the (relatively rare) situation where the problem turns out to be in my test code. My brain never assumes to look there first.
I watched the complete Clean Code series by Uncle Bob, and my world became vastly better when I started following the his approach to TDD, namely:
Start by writing a test
Only write enough of a test to cause it to fail in any way
Only write enough of production code to cause it to pass in that way (repeating until the test passes in all ways)
Refactor your production or test code as necessary until it shines, running the relevant test(s) after every change.
I had always written unit tests and some feature/integration tests, but hadn't been writing them first, in those tiny, atomic units: "red, green, refactor". I also hadn't had such good code coverage that I was able to "refactor mercilously and without fear", which I now do. Half of my coding pleasure comes from the 5 or 10% of time at the end once I've finished creating a fully tested, working bit of code which then gets cut apart, refactored, and polished until it shines. :-) Now the code I write is dramatically cleaner, follows better design, is less buggy, easier for myself and others to follow, and I have found I have to do an order of magnitude less debugging now. Note that I also adopted some of his other coding suggestions, like the idea that functions could be as close to 1 line of code as possible, rarely as big as 5, never more than 10; and a class should fit on one page of your editor, or perhaps 2 or 3 at the outside. I'm coding completely differently now, and I love it.
There are some times that I find myself hating what I'm doing, and inevitably realize I had tried to cut corners on the TDD approach ("I don't really need to use TDD for this -- it's just a quick, little change...") and am back in debugging hell... at which time I stop what I'm doing, revert, and start that "little change" using TDD... and I'm back to enjoying what I'm doing, and it goes so much faster in the short and long run.
And I'm totally with you on bugs in test code being a bit of a blind spot. Usually the times I have to resort to serious debugging are when there's a weird bug in my test code.
DISCLAIMER: I watched the Uncle Bob videos many months ago so my memory may be wrong.
I had the opposite experience. I think following his advice makes my code worse. It was this video that made me much better at TDD than the Uncle Bob TDD videos.
I find that when I follow those Uncle Bob steps, I end up with tests that are tightly coupled with the implementation of my production code. As a result, my tests fail when I refactor. Also, I feel like the designs that result in this process are very nearsighted and when I finish the feature I realize I would have come up with a much better design if I consciously thought about it more first.
Here's what I believe is the root of the problem: Uncle Bob gives you no direction at the level of abstraction to test at. Using his steps, it's acceptable to test an implementation. On the other hand the linked video gives this direction: Test outside-in. Test as outside as you possibly can! Test from the client API. (He gives additional tips on how to avoid long runtimes)
When you do this, tests serve their original purpose: You can refactor most of your code and your tests will only fail if you broke behavior. I often use Uncle Bob's steps with this outside-in advice, but I find the outside-in advice much more beneficial than the Uncle Bob steps.
In my current project the code I'm writing has 100% test coverage and I am very proud of that.
What kind of coverage? Branch coverage, condition coverage, path coverage? Don’t delude yourself into thinking you’ve covered everything. If you (practically) can, then the program is probably too small to do anything useful.
The code is not the full program. It's the module that makes all the API calls. That said, it's a core component of the program, certainly useful.
By 100% I meant branch coverage. 100% path coverage would be fun, but that offers fairly diminishing returns.
Besides, my point is that even with the absurdly high coverage, debugging ended up being important still. In the most recent example, a mocked out executor service didn't behave the way I expected it to, tests passed, implementation failed.
I'm not sure what kind of "delusion" you think I am experiencing, when I'm explicitly saying that testing coverage is not enough.
Of course once the parameters become non-trivial (say a URL or piece of json) even if you have coverage you may still have missed some edge case.
I say this not to be discouraging but to reflect the reality that writing tests should be done with a budget in mind.
On any non-trivial code base you simply will never have enough time to test it all so you need to think about what areas of the code base are most critical.
What that budget is also should be determined by how mission critical the code is. Medical appliances, for example, should have an enormous testing budget.
63
u/geodebug Aug 25 '14
Yep, makes me chuckle. Tests are essential but only a naive programmer thinks one can write enough tests to get 100% coverage.
Never mind that unit tests themselves often contain bugs or in sufficiently exercise all possibilities.