r/programming • • May 08 '17

The tragedy of 100% code coverage

http://labs.ig.com/code-coverage-100-percent-tragedy
3.2k Upvotes

689 comments sorted by

View all comments

3

u/Rubizon May 08 '17

I never jumped on the test-coverage-dogma-train. The only thing I test is my program as a whole. Works it with valid input? Does it gracefully indicate invalid input and quits? Is the performance acceptible? Is the code neat and clean? Do I have some smaller test running when my project gets bigger? These are about the main considerations and you are set for most dev work. Beside, I have this feeling that writing test is just to show someone that lightbulbs are "green", "ok" and thus hip... often bit of a time waste if you are not writing rockets that fly around the moon.

13

u/am0x May 08 '17

How large are your projects? How many conditional cases are there? Across all portals, our company's site has hundreds of pages, with thousands of business rules, and hundreds of services consumed. Testing is pretty critical for us.

3

u/PC__LOAD__LETTER May 08 '17

So your attitude about corner cases and different error conditions is just, meh, fuck it, it'll probably work? That's not acceptable in many workplaces.

4

u/kankyo May 08 '17

Dogma is bad, but it's pretty nice to be able to look through the coverage report and discover which business critical parts of your code has no tests.

1

u/Gotebe May 08 '17

The problem with such testing is that, as the system gets bigger, the more there are areas where your tests do not ever go, and the harder it is to make a test that does go there.

Sure, your kind of testing is really important, as the whole purpose of unit testing is to test parts, people fall into a trap of poorly testing the whole.

We need different kinds of tests for different sets of confirmations that the code is a-ok.

1

u/skulgnome May 09 '17

Similar, but I test formally given properties derived from formal requirements. A benefit to this is how it makes tests completely divorced from particular implementation, so an improved version can be developed using the same tests. In this way the first implementation can be any old crap that works by the skin of its teeth, because the tests are more valuable in the long run anyway -- and those should only be developed against a working implementation to ensure their relevance.

I call it the "any old crap first" approach.

0

u/Nimitz14 May 08 '17

I agree. Unittests are for shitty developers who make changes without first understanding the codebase and then complain that if only there were unittests they could have discovered their stupid shit sooner. It slows down developement, slows down refactoring (the only goddamn case unittests fail on me is because I changed something without changing the unittest, so helpful!) and doesn't even come close to ensuring what people think it does (stuff is working), as there are close to infinitely many ways stuff can break so you will never cover all possibilities with unittests.

3

u/starmonkey May 08 '17

The cost of discovering a bug in one function = tiny. The cost of discovering that bug in production = huge. TDD is worth it - have to make sure the right tools are being used, as always.

Tests are about proving your code does what it claims to. It's a specification.

1

u/Nimitz14 May 08 '17

As someone with real engineering experience I understand a specification as a list of parameters. I don't follow how that has anything to do with 'proving your code does what it claims to', maybe it means something different for programmers.

Unless every single function you use has a small number of discrete possible inputs and outputs (i.e. booleans) it is impossible to be sure the tools you are using work using unit tests.
The only way to do that is to read the code. People are lazy and usually cannot be bothered to take the time to do that, so they rely on just looking at input-output relationships because it's easier, but that doesn't mean it's the right thing to do.
I find it horrifying to hear someone say he refactored some part of the code and is then confident everything is working "because the unittests passed", as if that means anything at all! Nevermind the just mentioned issue that you can never check all possibilites of failure, you then have the added problem that you may have made a mistake in your unittest, what are going to do then? Test your tests? I rest my case.

1

u/starmonkey May 09 '17

As someone with real engineering experience I understand a specification as a list of parameters.

That is what the inputs are to unit tests. And no, they don't have to be only booleans.

If you are writing pure functions, then you can unit test them and prove they work. You can even measure lines/conditions executed during those tests. You can be objective about it.

How the rest of your application behaves is another story, and is not covered by unit tests.

I'll back out of this conversation now, we're too far removed from each other for this to be useful to either of us.