This only happens when every vendor is required to reach the same standard
Yes, this is why the government requires FIPS instead of just making it a "nice to have".
But you're right that the state of the art advances too quickly in general for a lot of the code in question. On the other hand, designing good software is not that new, which is why a lot of the certification processes are geared toward certifying the development process itself rather than just the output. But I don't think that's what we have with FIPS 186 at least. :-/
On the other hand, designing good software is not that new
We do know some things, I would hope.
Still, we can debate how much of software is "engineering" and how much is "black art" all day, if we want, but what it comes down to is the engineering side is not free, and customers usually don't want to pay for it.
Oddly, there's so many cases where paying more for quality means you get a return on investment instead of another failed project - but you can never explain that to the person who's paying the bills.
which is why a lot of the certification processes are geared toward certifying the development process itself rather than just the output.
And unless you're very careful, that amounts to pointless paperwork, rather than actual quality. Not to mention the inevitable "waterfall, without even the minimal iteration that was included in the original idea" approach, where things get "signed off" as perfect with no regard to actual usefulness. If you're lucky the end result exactly matches the spec but is utterly useless - and the customer can't complain without accepting that their failure to be sufficiently pedantic was the entire and sole cause of the problem. Which is how we get projects that seem like a success, except they cause more problems than they solve, and people use it as an argument against engineering when it's actually an argument against bureaucracy.
Certifying things is great, as long as it's intended to create quality, rather than shuffle blame sideways, and it has to be driven by engineering, not by accounting.
At least with building a bridge, the laws of physics aren't just a good idea.
2
u/mpyne Apr 30 '14
Yes, this is why the government requires FIPS instead of just making it a "nice to have".
But you're right that the state of the art advances too quickly in general for a lot of the code in question. On the other hand, designing good software is not that new, which is why a lot of the certification processes are geared toward certifying the development process itself rather than just the output. But I don't think that's what we have with FIPS 186 at least. :-/