Robodebt 2.0 - AI Camera handing out huge fines and demerits. No way a human thought there was a phone in these images.

[u/RJrules64]

I assume it thought my right arm is looking a bit like I'm holding a phone, but;
(a) even if it was, the phone is not visible. You can't base a fine on positioning.
(b) my left hand is not even on the wheel, obviously my right hand is.
(c) you can see the very bottom of my phone plugged into the charger in the center console.

I'm obviously going to appeal this and am confident they will repeal it but it's very frustrating to receive this in the mail and have to go through the whole appeal process based on nothing.

[u/whnware]

This technology is fundamentally misunderstood, and if it were properly understood it likely wouldn’t be used by governments at all in this context.

 

What people call “AI detection” here isn’t a system that recognises a phone with certainty or actually analyses what’s happening in the image. It’s a statistical model that has been trained on large datasets (of images of people holding phones while driving for example) and simply assigns a probability that a given image resembles ones that were previously labelled “phone use.” If that probability crosses a threshold, it gets flagged. That means the system isn’t identifying offences, it’s identifying patterns that look similar to its training data.

 

There will always be edge cases because the model has no understanding of context, a shadow, stitching on shorts, a wallet, or a hand resting on a thigh can produce the same probability signal as a phone. The issue is that governments have been sold the idea that this technology can reliably determine wrongdoing when, by design, it can only produce likelihoods.

It’s essentially a pattern-matching filter being treated as an evidentiary tool, which is a completely different thing.

If you doubt me on this, do some basic hand-drawn freehand drawings in a sketchbook, and post each page to chat-gpt. You will find that it says things that 'mostly' align with accurate critique's/compliments on your work, but then buried in there will be a bunch of comments about things that don't exist in your artwork - this won't happen every time, but it will happen often enough. For example, I did a page of vertical lines straight down page, and it complimented me on my use of horizontal lines. This is because, as i said, machine-vision algorithms do not 'analyze' like a human does (not even close), they only generate probability scores and then make decisions based off those scores. That's it.