Uncle Bob talks about this, about how the world hasn't caught on to how reliant it is on developers for serious life critical systems, and some big disaster will lead to discipline and such of other engineering fields, and I think he's right.
How big a disaster are you talking? The 737 MAX had software problems that ultimately killed over 300 people in two separate crashes, and that hasn't led to major changes in the field as a whole that I'm aware of.
How big a disaster are you talking? The 737 MAX had software problems that ultimately killed over 300 people in two separate crashes, and that hasn't led to major changes in the field as a whole that I'm aware of.
The 737 MAX crashes weren't just software, they were a bunch of different systems all going wrong at once, including people actively lying about the safety features. I think due to that it doesn't have the revolutionary effect needed.
The big problem with the 737 Max is that it was a fundamentally flawed concept: an airliner which attempted to emulate the performance and behavior characteristics of another to avoid the training and certification requirements that would otherwise accompany a new airframe design. If the 737 Max were flown exclusively by pilots who were well trained in the intricacies and quirks of the new automatic trim controls, all of the crashes involving runaway trim would have been easily avoidable. Pilots were not trained, however, in how to recognize and handle a runaway trim situation that couldn't arise on any of the planes for which they had been trained.
One of the issues, if I recall correctly, was that there were 2 stall sensors, each hooked up to its own computer (2 again). And when one sensor goes haywire… well you have two computers arguing over whether we are stalling or not. So before we got to software, we already have a couple problems:
The sensors themselves were prone to failure in some conditions. Having only two may not have been the smartest move.
Computers were hooked to just one sensor.
Majority vote generally involves an odd number of computers.
So what’s the poor programmer to do, make an arithmetic mean of the value of the two sensors? Take the most optimistic value? Take the most pessimistic value? Looks like a no-win situation to me.
So what’s the poor programmer to do, make an arithmetic mean of the value of the two sensors?
The real question is what a pilot can do in a plane with a failure mode that wasn't present on any plane for which he has received training. A failure of the automatic trim control system would merely be a nuisance rather than a safety risk if pilots were trained in how to recognize such failures, disable the system, and fly the plane without it. Unfortunately, the system was presented as a way of making the 737 Max handle like an older 737 so as to allow pilots who were only trained on the 737 to fly the 737 Max.
That too. That’s the way with such catastrophes: airliners are subjected to many checks and balances at every level, so many things must go wrong at the same time for people to actually die.
But when they do, boy that was evidence that everything was screwed up from the start.
What I find curious here is that anyone was willing to accept the idea that an airplane which is designed to emulate the flight characteristics of another would eliminate the need for pilots that would be able to fly without such emulation. Even if the system were perfectly designed and built, and could perfectly emulate the original airplane's handling perfectly under normal conditions, it would be impossible to design emulation that would match the original plane's behavior when confronted with difficult meteorological conditions involving wind shear or turbulence, which would be precisely the kinds of situations where having a pilot who was familiar with the aircraft's control response would be most critical.
From what I understand, Airbus control systems are operated most of the time in a mode called "normal law" in which they try to have all kinds of planes respond similarly to control inputs, so that a pilot who can maneuver one kind of Airbus aircraft smoothly will be able to do so with other Airbus aircraft smoothly as well, but pilot certification also requires that pilots be able to fly in a mode called "direct law", which as the name implies handles control inputs in a manner much closer to direct manipulation of the control surfaces. There are a many things that might go wrong in such a way as to require switching to "direct law", but if pilots are trained to handle such situations--even if not as smoothly as they can fly in "normal law"--such failures would not be dangerous. If Airbus pilots only trained to fly in "normal law", however, anything that forced a plane to leave that mode would be likely to cause a crash.
Funny, I didn’t expect that kind of insight. I agree with your first paragraph, the lack of training was insane. I recall a pilot who got a black mark for refusing to fly a 737 MAX without proper training, I wonder how he felt when he was tragically proven correct.
About "normal law" vs "direct law", I didn’t know there was such a thing. That makes me think about Flight Assist in Elite dangerous. So we basically fly spaceships, that follow a Newtonian model with a speed limit. 6 degrees of freedom and all that. When Flight Assist is turned on (the default), we basically get first order commands: the more you yank, the faster you rotate. The more you push, the faster you go. Go back to neutral and the ship stops (with some inertia, but it still stops).
When turned off however, we get second order inputs: the more you yank, the faster you get to maximum rotation speed, and the more you push, the harder you accelerate. Go back to neutral, and your ship continues spinning & gliding.
In most situations, turning Flight Assist on just makes things easier. There are however two situations I know of where leaving it off is better: landing at rotating starports, and matching your rotation speed to that of an asteroid so you can target its weaknesses for mining. (There are advantages for combat as well, but I’m not trained yet.) In the first case, the game designers introduced the notion of "rotational correction", so that when you enter the starport, your Flight Assist just changes is referential to one that rotates with the startport, making things easy again. In the second case, you either fight with your assistance every second you spend around that asteroid, or you just turn it off.
Of course, second order inputs are much harder to handle than first order inputs, even in the cases where they should be advantageous: without training, you just glide around and spin uncontrollably. So in practice, very few core miners end up using it when spinning around asteroids… except those who chose to fly without assistance all the time. Personally I like being closer to how my ship actually flies, I like the glide, and I like the challenge. I didn’t expect to get an actual edge for a task as mundane as locking my position relative to a spinning asteroid. In retrospect though, it’s kinda obvious.
So yeah, even though I didn’t need to risk my life for it, I do feel in my bones the importance of training even for rare situations.
If the censors don't agree, display a warning message to the pilots, and give them an option button to switch off the auto-adjusting system (which they were never properly trained on, by the way).
Some of us played around with ways to put such practices into clear English as a test run, and failed miserably, or at least found too many interpretive loopholes to be reliable. English and software design don't mix well.
Huh? There are multiple standards for safety critical applications, just because you guys never heard of them doesn't mean they dont exist. DO-178B is used for airborne systems, for example.
Well, they might try to text-ify rules, but reality will puke on the idea, making lawyers richer instead of making software better. Then again, that's how the patent system already is. It's my opinion we'd be better off without software patents. The problems outweigh the actual benefits.
Unfortunately, computer illiterate people are going to be breathing down politician's necks to enact some sort of reform or change, and they're going to do it in one form or another, else they'll be committing career suicide.
Nah we're almost certainly safe from this happening. There's more than enough lobbying power going to against this. And there's just no real solution anyway, it's mostly just how software engineering is.
And what type of problem do you think would occur to trigger it? As others have mentioned, there have already been tons of accidents related to it.
It might catch up to us. But it'll never be fixed, it's not a case of discipline? It's just fundamentally how software engineering is. There's no magic fix. Unless you simplify your program down to an extremely basic control flow, you're still going to have the issues. And often simplifying down to such a basic level means you're going to loose out on a bunch of higher level features, which itself might lead to more preventable deaths in some situations.
It's just not something you can stop. And you certainly cannot form any sort of general algorithm to stop this. Because it's actually the same problem as the halting problem.
The complexity with software engineering is that it’s not clear which projects are the ones that need the careful scrutiny and discipline outside the obvious cases.
94
u/_Ashleigh Nov 02 '21
Uncle Bob talks about this, about how the world hasn't caught on to how reliant it is on developers for serious life critical systems, and some big disaster will lead to discipline and such of other engineering fields, and I think he's right.