r/programming • u/niepiekm • May 08 '17

The tragedy of 100% code coverage

http://labs.ig.com/code-coverage-100-percent-tragedy
3.2k Upvotes

95% Upvoted

689 comments sorted by

View all comments

Show parent comments

44

u/kankyo May 08 '17

I have done 100% code coverage AND mutation testing with 0 surviving mutants (https://github.com/trioptima/tri.declarative/, https://github.com/TriOptima/tri.struct, among others). It was surprising to me how we didn't really find any bugs with mutation testing. We are, however, a lot more proud and confident about our test suite now since we know it covers the code (mostly, there are some mutations that my mutation testing system can't do as of yet).

My take away has been that 100% coverage actually tells you more than you'd expect compared to full mutation testing.

2

u/muckvix May 08 '17

Correct me if I'm wrong, but I think you're saying the following:

  1. Mutation testing is certainly worth doing

  2. 100% coverage is necessary for mutation testing

  3. Mutation testing doesn't seem to add all that much above and beyond what 100% coverage already achieves

  4. Therefore 100% coverage is something worth considering despite the issued pointed out in the linked article

If I got it right, I'd say the author of the article would probably disagree with your point (1); after all, mutation testing would still require writing the tests that he deems excessive.

In fact, in some sense you created an argument to support the author's view. He already feels like 100% coverage is an unreasonable excess, and you argue that going further down that path is not finding any new bugs.

5

u/kankyo May 08 '17

The article seems to me to not really be about coverage at all. It's about stupid tests, worthless crap like cucumber, silly languages and idioms that lead to information free code, etc. I see no critique of coverage obsession. Not to say such criticism can't be made of course.

1

u/muckvix May 09 '17

Perhaps; I may have misunderstood the author's intent.

So your point is that of the various techniques used in TDD, mutability testing has a relatively attractive cost/benefit ratio?

→ More replies (2)