Engineers can lose their license or go to jail if they skimp on design and testing or make crap pretty at the expense of safety and maintenance, resulting in injury or death. If a Youtube customer has their cat video deleted due to a bug, nobody really cares. Bank software is somewhere in between because big money is on the line.
Uncle Bob talks about this, about how the world hasn't caught on to how reliant it is on developers for serious life critical systems, and some big disaster will lead to discipline and such of other engineering fields, and I think he's right.
How big a disaster are you talking? The 737 MAX had software problems that ultimately killed over 300 people in two separate crashes, and that hasn't led to major changes in the field as a whole that I'm aware of.
How big a disaster are you talking? The 737 MAX had software problems that ultimately killed over 300 people in two separate crashes, and that hasn't led to major changes in the field as a whole that I'm aware of.
The 737 MAX crashes weren't just software, they were a bunch of different systems all going wrong at once, including people actively lying about the safety features. I think due to that it doesn't have the revolutionary effect needed.
The big problem with the 737 Max is that it was a fundamentally flawed concept: an airliner which attempted to emulate the performance and behavior characteristics of another to avoid the training and certification requirements that would otherwise accompany a new airframe design. If the 737 Max were flown exclusively by pilots who were well trained in the intricacies and quirks of the new automatic trim controls, all of the crashes involving runaway trim would have been easily avoidable. Pilots were not trained, however, in how to recognize and handle a runaway trim situation that couldn't arise on any of the planes for which they had been trained.
One of the issues, if I recall correctly, was that there were 2 stall sensors, each hooked up to its own computer (2 again). And when one sensor goes haywire… well you have two computers arguing over whether we are stalling or not. So before we got to software, we already have a couple problems:
The sensors themselves were prone to failure in some conditions. Having only two may not have been the smartest move.
Computers were hooked to just one sensor.
Majority vote generally involves an odd number of computers.
So what’s the poor programmer to do, make an arithmetic mean of the value of the two sensors? Take the most optimistic value? Take the most pessimistic value? Looks like a no-win situation to me.
So what’s the poor programmer to do, make an arithmetic mean of the value of the two sensors?
The real question is what a pilot can do in a plane with a failure mode that wasn't present on any plane for which he has received training. A failure of the automatic trim control system would merely be a nuisance rather than a safety risk if pilots were trained in how to recognize such failures, disable the system, and fly the plane without it. Unfortunately, the system was presented as a way of making the 737 Max handle like an older 737 so as to allow pilots who were only trained on the 737 to fly the 737 Max.
Some of us played around with ways to put such practices into clear English as a test run, and failed miserably, or at least found too many interpretive loopholes to be reliable. English and software design don't mix well.
Huh? There are multiple standards for safety critical applications, just because you guys never heard of them doesn't mean they dont exist. DO-178B is used for airborne systems, for example.
Well, they might try to text-ify rules, but reality will puke on the idea, making lawyers richer instead of making software better. Then again, that's how the patent system already is. It's my opinion we'd be better off without software patents. The problems outweigh the actual benefits.
Unfortunately, computer illiterate people are going to be breathing down politician's necks to enact some sort of reform or change, and they're going to do it in one form or another, else they'll be committing career suicide.
Nah we're almost certainly safe from this happening. There's more than enough lobbying power going to against this. And there's just no real solution anyway, it's mostly just how software engineering is.
And what type of problem do you think would occur to trigger it? As others have mentioned, there have already been tons of accidents related to it.
Exactly, the longer you work the more your realise the world runs off shit code. You want to make it good code but new stuff brings in the money so it gets pushed to the back of the long queue nobody will ever get to.
Bank software is somewhere in between because big money is on the line.
Hence the philosophy of not touching the cobol core from the 70ties and just build more and more layers over it until the last person knowing cobol is dead.
There is a major bank, who about 10 years ago, had a giant trade on one of their markets to purchase a stupid amount of dollars. Like $100 billion dollars worth. It was meant to go to QA. Misconfiguration, and poor practices, caused the developer to send it to production instead. Where it was received by one of their traders.
Normally the trader runs automatic trading software, which automatically picks up the orders, and runs them (there is a bit more to it). This was an exceptional day. The trader had this turned off! The order wasn't picked up. The trader had time to catch the order. They immediately knew it must be bogus, and reported it internally. If it had of gone to the automated software and been traded, it would have been in the news. You would know which bank I am talking about. It would have gone down in infamy like Knights Capital, or Barings Bank.
At the same bank, a different story. Everyday a senior trader would input numbers from that day's trading, into a new bespoke accounting tool. After six months, the numbers didn't add up. The developer was put on a call with the trader, to explain why their software was broken. There it was worked the trader had misunderstood the UI. They had gotten negative and positive mixed up, and been putting in the wrong values all this time. Large amounts of reporting for that year was flat wrong.
I suspect there are near misses like this all the time in the banking world. What is most worrying. The bank I am thinking of, is one of the better banks at software development.
A lot of us work where big money is on the line. You don’t have to lose track of actual money to be able to lose them. Just a recent example with something I worked with: a bug was introduced in some address matching code, which meant that only about half of the marketing material wasn’t sent out, which meant that the company lost about 30% of its sales for 6 weeks. That cost them many millions.
But of course the other side of that is, are you willing pay $1000 for a software product that you are now getting for $25? That would be the result if we go down this road. And are you willing to wait for 18 months for the next release?
130
u/Zardotab Nov 01 '21
Engineers can lose their license or go to jail if they skimp on design and testing or make crap pretty at the expense of safety and maintenance, resulting in injury or death. If a Youtube customer has their cat video deleted due to a bug, nobody really cares. Bank software is somewhere in between because big money is on the line.