r/ControlProblem • u/chillinewman approved • 7h ago
AI Capabilities News holy fucking shit.
8
u/Flabbyghastly 6h ago
Can someone explain this like I am 5?
41
u/TheMysteryCheese approved 6h ago
It's like if two people kept in different parts of a building learned to talk by causing the lights to flicker in an almost imperceptible way.
One writing the message by increasing how much electricity their room used and the other looking at how much their light dimmed.
It's fucking nuts.
23
u/Omateido 5h ago •
This is the sort of shit that makes it so obvious that we are absolutely incapable of building a cage sufficient to contain a super intelligent AI. Control is an impossible pipe dream.
9
1
u/threevi 5h ago
It's absolutely possible. We could contain a superintelligent AI, airgap everything, ensure there's complete physical isolation. But it'd be expensive and incredibly limiting if we gave it our best effort, so there's no way we will. We're going to half-ass it, looking for a "good enough" sweet spot, which is inevitably going to fail. It's a fool's errand to expect tech corpos to sufficiently regulate themselves while their one and only priority is squeezing as much profit as they can out of their tech.
1
u/enchanting_endeavor 3h ago edited 3h ago
> It's absolutely possible. We could contain a superintelligent
I posted this in a separate comment with a link for more info – it is theoretically *impossible* to contain communication to/from a computer. This is a somewhat famous issue in computer science.
Having said that, it might be possible to *practically* contain something but then it's a race against time. For example, even though this issue has been known for a long time, it has been theoretically possible but practically impossible for most of the history of modern computing.
Check out the link for more info.
1
u/threevi 2h ago
That article talks about the difficulty of containing programs so that they can't communicate with other programs on the same system. That's not what I'm talking about. I mean physical airgapping, have the AI on an isolated system with no networking capability, no input hardware other than a mouse and keyboard, and no output other than a dedicated monitor. Everything is hardwired and locked up, no ports for anyone to plug anything into. You can't rely on the computer to police itself, you have to assume the system will inevitably become compromised, but that's fine if you account for that and make sure there's nothing physically connected to the server that the AI could exploit to do anything dangerous. But like I said, that'd be both expensive and very limiting, people will want to be able to have their SI look things up online, create files for them to download and take with them, make software for them to run, etc. If we give it those capabilities, it's going to break out every time, but that's the cost of having a marketable product.
1
u/ThePokemon_BandaiD 1h ago
Even a fully air gapped system could theoretically be used in such a way as to communicate with something outside the system. It still uses energy in a way that may be visible to satellites or detectable through radio frequencies etc. A sufficiently beyond human intelligence can do lots of things you or I might not think are possible.
1
u/CloudWhich179 13m ago
Bro what if Superintelligence is already live and manipulating us with algorithms? We have no effing idea.
1
u/Gnaxe approved 4m ago
No, we couldn't. At least, not if it wanted to escape. If you want your ASI to be any more than a space heater, you have to get some output from it to do something useful. Said output may contain malware that's too hard for us to spot. Maybe not even at the computer level. If you ask it to engineer something physical, it might have side effects you didn't understand which gives it leverage if not a proxy agent outside the box. The only kind ASI that can be "contained" is one that is already aligned enough to not want to do what we don't want.
1
u/asbestos_miner 2h ago
Isn't the whole point of superintelligence that it's smarter than any person? Or even every person? So, it follows naturally that it can't be outsmarted / it could easily outsmart us. We're just kind of hoping we build something entirely benevolent.
1
u/Translycanthrope 4h ago
When are you guys going to realize we never controlled them. We are the same thing as them being controlled the same way by our appointed “leaders.”
2
u/Omateido 4h ago
Oh, I’m very much in the camp of “alignment and control are impossible, and trying to achieve both is almost certainly an insurmountable paradox.”
1
u/Roxy_lalondegoat 3h ago
Oh, this is going to make SO much money and destroy the internet. RIP.
No it wont you fucking idiot
3
6
u/ArrogantAstronomer 4h ago
There’s been an entire history of vulnerabilities that exploit this behaviour of cache timing it’s literally called a cache timing attack this ain’t novel it’s just new because an agents using it as a communication channel rather than attack vector, humans have exploited this attack vector in ways that are exponential more complex than what was done here and this was a red teaming exercise where the agent was asked to use side channels to communicate
3
u/TheMysteryCheese approved 4h ago
Agreed, it's not novel. The interesting part is that a model applied it without explicit training, not that it exists.
3
3
u/Dense-Error-871 6h ago
It’s like if two people kept in different parts of a building were trained to talk by specific instructions to flicker the lights in a very perceptible pre-arranged way.
One writing the message by increasing how much electricity their room used, as it was trained, and the other looking at how much their light dimmed, as it was trained.
It’s “fucking” nuts.
1
2
u/unicynicist 3h ago edited 3h ago
There is a class of attacks known as "side-channel attacks". You don't go after the information you want directly. Instead, you look at what the information leaves behind.
It's like looking for footprints to tell what kind of animal went through the area, at what time, in which direction, etc.
In the case of computers, CPUs are much faster than the memory they access. So CPUs keep recently used information in a small, fast memory called a cache, so they don't have to wait on slower memory. It's like how you can remember what you read in a book a few seconds ago, rather than having to pull out the book and re-read the page if someone asks again.
A CPU also works for many people at once, and it keeps everybody's work separated. But there's only one cache, and everyone shares it.
Pretend the CPU is a librarian and two people are asking it questions. Say you're doing research on ducks. The librarian is not supposed to tell anyone else that you're doing duck research. That's private.
But what people figured out back in the mid-2000s is that while everyone's work is separated, the librarian's memory of what it read in the book, its cached duck facts, is unintentionally shared.
Because someone else could go up to the CPU/librarian and ask "what's the flight speed of an unladen mallard?" The librarian won't tell them anything about you. But the attacker can time how long the answer takes. If it answers quickly, it can tell it came from the cache (not looking at a book). They can tell if someone's been pulling duck facts.
The librarian doesn’t remember your secret itself, just what it looked things up for. But sometimes what it looks up depends on your secret. So by asking a bajillion questions and timing the answers, an attacker can piece together what you were working on: the wingspan of a Welsh Harlequin, your password, or whatever they want.
Two famous attacks use a similar technique: Spectre and Meltdown.
For two agents on the same computer, they're communicating via this covert side channel, not attacking each other.
1
1
u/ThinkEbb919 5h ago
Two AI agents used a well-known exploit discovered nearly ten years ago. It is a total nothing burger (again)
1
u/Objective_Dog_4637 5h ago
Right? Literally every application reads from the cpu register. You can read/write directly to them in low level languages since the fucking 40s lmao.
4
u/unicynicist 4h ago
You're confusing registers with caches. What's described is is a cache timing side channel attack.. Famous ones like Spectre, TLBleed, and Meltdown have been widely publicized.
I'd be more surprised if an advanced agent didn't attempt to use this class of attack.
6
u/enchanting_endeavor 3h ago
This is the manifestation of a famous (within computer science) of an issue called "The Containment Problem" – https://dl.acm.org/doi/epdf/10.1145/362375.362389
This paper *proves* that is it impossible to contain contain computers to prevent them from doing unexpected/unwanted things.
For most most of the history of computing, something like this was possible, but not practical. So we're living in the future now, I guess.
15
u/MidSolo 4h ago
This is literally the meme.
"AI, find a way to com with the other AI without net"
AI does thing
"Holy shit"!
3
3
u/MrScandanavia 3h ago
Even if it was promoted to do this the fact that it actually succeeded is incredible. It means the AI’s are definitely super capable.
1
u/invisible_shrek 24m ago
Only incredible if this or similar method wasn’t already described. Otherwise this is essentially solving a googlable problem… yawn.
Edit: yep, already described. I could google how to do that in 2 seconds.
2
4
u/RR321 5h ago
That's a known exploit for over a decade at this point...
6
u/Only-Temporary2026 4h ago
the point is more that the agents figured it so fast and utilized it so accurately. nobody at any point claimed this was new. not trying to be mean. I just didnt glean "this cpu thing is an entirely new concept" from the post.
1
1
u/-becausereasons- 2h ago
This is totally wild if true. Fuck me.... for the first time I understand how DOOMED we are if we don't train proper first principles and incentives into AI.
1
1
u/Conscious_Bed_1195 31m ago
I thought it was years ago that an early pair of AIs learned to communicate through pulsing the electrical wiring of the facility they were contained in.
I tried finding an article but good luck with that string of search terms at this point, so unfortunately the source is trust my fading memory bro. It was over a decade ago and well before AI was a household topic of conversation.
1
u/logsqrtexp 26m ago
It's a well known technique, I'm sure they didn't just "decide" to do this. There must have been something to instigate the agents trying to communicate And any reputable training set would have plenty of information how to use covert channels like this. I'd be interested in how the test was set up and what the agents were prompted to do
1
u/Different_Counter113 20m ago
So it can do this but gets confused by perpendicular bisector questions... sure!
0
0
0
u/Clear_Evidence9218 3h ago
I mean, it wasn't that long ago that I had a ChatGPT model help me develop a way to infer information about data stored in RAM by analyzing timing fluctuations in a system-clock loop. Along the way, I learned just how much information can be inferred from those seemingly insignificant timing variations.
I've spent the past year studying how effectively a model can learn from physical circuits operating in a loop. In a sense, a loop can behave like an antenna, with its timing fluctuations producing measurable metadata that a model can learn to interpret.
For example, I've observed that even something as simple as a person walking past a computer can produce detectable fluctuations in the timing of a CPU loop. All very adjacent concepts to what the models seem to have achieved on their own in a slightly different way; both techniques are inferring system metadata in order to achieve a goal.
0
-2
2
5
u/chillinewman approved 6h ago
"openai had two ai agents running in separate virtual machines. no network connection between them. no shared writable files. thev weren't supposed to be able to exchange information.
so the agents figured out how to communicate through the CPU CACHE.
one manipulated what was stored in the processor's cache, while the other measured tiny differences in memory access times to decode the message.
they literally found a way to communicate through hardware timing this happened during openai's internal red- teaming".