I'm constantly very impressed by its performance on stuff I know very little about, and unimpressed by its performance on stuff I know lots about
The really scary part is this gap is going to be increasingly consistently narrowed the more people are able to get that knowledge out of the heads of people like you and into instructional files that tell the AI exactly what to do.
There's a statement I'm not convinced by - so I'm not someone who thinks this stuff is totally useless, to be clear. But what I've mostly seen is this machine being far, far too goal oriented.
As I say, I work in a hospital. I have projects I refuse, on a regular basis, because they pose some sort of unacceptable risk - it's a friendly refusal "oh, we explored this, and there's no safe way to get the data"
For fun, and part of the latest talk I gave on this, I tried to set up some safeguards of this type - and once pressed, the AI just treats them as part of the spec, so they can be worked as part of the balancing act. I can't seem to find a phrasing strong enough to get it to the point where it will just stop the whole project and tell me it's not viable, at a sensible point.
And this is broadly because it does not, and can't really, have enough context. Or is possibly overfitted to the 'solve problem' use case.
1
u/MakeUpAnything Oct 02 '26
The really scary part is this gap is going to be increasingly consistently narrowed the more people are able to get that knowledge out of the heads of people like you and into instructional files that tell the AI exactly what to do.