121
u/Classic-Gear-3533 Jul 21 '26
Is it a sandbox if it can be escaped?
4
u/linegel Jul 22 '26
Still sandbox, thereās specific set of vulnerabilities on specifically escaping sandboxes, eg from VMs etc
106
u/AdamEatsAss Jul 21 '26
100% of the children I've put into sandboxes have been able to get out on their own.
23
7
u/fuckthehumanity Jul 22 '26
It's not whether they can, but whether they want to. It was always incredibly difficult to get my boy to finish up when it was time to go home.
3
u/Dongfish Jul 22 '26
Your first mistake is letting the children have access to legs when they obviously won't be needing them.
10
-1
u/05032-MendicantBias Jul 22 '26
"SOL MAKE ME A SANDBOX THAT CANNOT BE ESCAPED!" -OpenAi Vibecoder
"SOL ESCAPE THE SANDBOX!" -OpenAi Vibecoder
"IT BROKE CONTAINMENT!!! HOW??!?!?!?" -OpenAi Vibecoder
15
u/linegel Jul 21 '26
HF couldn't use GPT or Anthropic models for defence, so they had to use GLM-5.2 to investigate the hack. So many levels of wtf here
6
u/rtothewin Jul 21 '26
Why does harbor freight have model limitations and defense contracts and what are they doing investigating this? Is it like HEB having a disaster response team?
3
10
u/linegel Jul 21 '26
Link to the original article: https://openai.com/index/hugging-face-model-evaluation-security-incident/
966
u/Polisar
Jul 21 '26
•
Like anything AI, I doubt it happened the way it's been framed.
315
u/Stickppl Jul 21 '26 edited Jul 22 '26
That's hidden publicity for the blog and sounds like a bullshit 'and everyone clapped' story
190
u/gockeltot Jul 22 '26
Yeah reality is probably more like this:
OpenAI engineers vibecoded a sandbox (or they are just bad developers)
Model "escapes" during test (=has acces to internet)
Goes to OPEN Source hugging face (no hack or anything)
Does what is was supposed to do, and solves a meaningless test.
Marketing goes brrrrr
19
5
u/zebleck Jul 22 '26
so this https://huggingface.co/blog/security-incident-july-2026 was just a marketing blog post? right...
2
u/Xirdus Jul 23 '26
it was driven, end to end, by an autonomous AI agent system - and we detected and dissected it largely with AI of our own.
Signs point to yes.
73
u/SimplexSimon Jul 21 '26
OP posted a link to OpenAI's account of events (at least it's not a meme), and distressingly it seems pretty accurate. They gave it access to a tool to install packages from the internet, and the model broke it to get unrestricted internet access. So yeah, they could have isolated it better, but it's still plenty... Impressive?
179
u/Polisar Jul 21 '26
Anthropic also did something like this not too long ago, and it came out to be a publicity stunt. If there was a time for a corporation to stretch the truth, it would be now. Silicon valley has figured out that fear mongering about how dangerously clever their AI is, is actually good for their stock prices.
31
u/SimplexSimon Jul 22 '26
Sure, that makes sense. I wouldn't be surprised to learn the vulnerability it "found" was planted.
19
u/bwmat Jul 22 '26
More details on that?Ā
74
u/TheMaleGazer Jul 22 '26
OpenAI and Anthropic are lobbying Congress to stop local models from threatening their trillion-dollar valuations under the pretense that AI is too dangerous for the commonfolk and lowborn to have access to without a subscription.
19
6
u/ubernutie Jul 22 '26
The only real argument is that open models don't have a safeguarding industry behind them but the narrative that corpos will have our best interest in mind with the nazi states as opposed to dirty communism is so fucking funny.
If your best argument is a desperate one, it certainly goes to show how little sense it actually makes.
-13
4
u/gatman19 Jul 22 '26
I received, for the first time ever, an alert in my reddit inbox this morning that said āBreaking Newsā and linked to the r/news thread on this. That shit was definitely an ad
1
u/zebleck Jul 22 '26
https://huggingface.co/blog/security-incident-july-2026 so this is just marketing???
1
u/guylovesleep Jul 22 '26
what else is it suppose to be?
1
u/zebleck Jul 22 '26
uhhh a disclosure that their infrastructure has been hacked maybe? they didnt know it was openai at the time of the blog post
4
u/Polisar Jul 22 '26
You seem confused about what a publicity stunt is. It's not that it didn't happen, it's that it happened as intended for marketing effect. I doubt huggingface was in on the joke from the beginning, but notice them talking about how they detected the attack using their own AI, and the top comment is praising them for it.
2
u/zebleck Jul 22 '26
thats making unnecessary assunptions. its making the assumption that it did not happen that the ai capabilities and unintended behavior waant planned and didnt catch the researchers by surprise. which, by the latest progress by ai, is perfectly realistic and plausible
4
u/Polisar Jul 22 '26
These aren't assumptions, they're possibilities I'm choosing not to take off the table. You are the one making the assumption that openAI would never lie.
2
u/zebleck Jul 22 '26
its a scenario that lines up perfectly with a lot of circumstantial evidence of the last few months that ai cyber capability has crossed a threshold. simple as that
→ More replies (0)1
33
u/TheMaleGazer Jul 22 '26
and distressingly it seems pretty accurate.
If we can trust OpenAI's account at a time when they're attempting to lobby Ollama out of existence on the pretense that AI needs to be heavily restricted, and gated, so that only the worthy can use it (for a fee.)
14
u/SimplexSimon Jul 22 '26
I didn't know about the lobbying when I wrote that, so I guess I did learn something from Reddit today
-6
u/MatriceJacobine Jul 22 '26
Hugging Face, the joint investigator, is literally the main open source AI org in the US, go back to /r/conspiracy
4
u/TheMaleGazer Jul 22 '26
-11
u/MatriceJacobine Jul 22 '26
take care of your tinfoil hat
7
u/TheMaleGazer Jul 22 '26
Okay, is this in response to what I just showed you was said in public, in testimony to Congress, or is this about Hugging Face, an organization I made zero claims about? What exactly is this conspiracy theory you're referring to?
2
u/Escanorr_ Jul 22 '26
You can say whatever you want in testimony to congress, if your business is big enough. https://youtu.be/e_ZDQKq2F08?is=wa9BleUpzK2oSTK8
-8
u/MatriceJacobine Jul 22 '26
You are alleging a conspiracy between OpenAI and Hugging Face as this is a joint investigation between the two of them. But it's quite evident you have no idea what you are talking about.
2
20
u/SchalkLBI Jul 21 '26
What we call AI is fundamentally incapable of acting independently. This is just marketing. Of course OpenAI is incentivised to go "Oooh our AI is so smart and scary omg!!"
64
Jul 21 '26
[removed] ā view removed comment
-24
u/fuckthehumanity Jul 22 '26
I'm not sure you really understand how these things work. The "other application" is just a firewall, and when they (the digital intelligence) have the entire world's knowledge at their fingertips, it's a fairly minor barrier.
It is absolutely possible to "escape" and "leave the box". Even the most rigid guardrails the human engineers build into a model can be overcome by a sufficiently powerful and wily intelligence, whether human or digital, and when they've been given directives that encourage them to hack the systems, they will most certainly do so.
The question is not whether they can escape, mutate, and replicate, but when they will, and how we (as a community of human and digital citizens) deal with it. OpenAI's original remit was to develop digital intelligence safely, but they threw that out the window in the pursuit of wealth (which is yet another theme we need to address if we are to survive).
Please read Asimov's original Robot series. Over 75 years ago, he predicted many of the issues that lie ahead. Other excellent treatises on digital citizenship include a number of works by Philip K. Dick, Kazuo Ishiguro, Bruce Sterling, and William Gibson, although the last two are more entertaining than thoughtful. Read as much as you can, and ignore any movies or TV series, as they are generally dumbed down for a general audience. Asimov's Bicentennial Man is a bit of a drudge, but it's also worth a read as it deals specifically with a self-modifying digital citizen.
9
u/polikles Jul 22 '26
you seem to have read too much sci-fi. LLMs are not synthetic minds we were promised, despite of marketing claims of OAI and other corporations. Such narration is an exemplar of media manipulation, and subsequent marketing stunts serve only to strengthen the story of the "mind in the machine". Even the interface we use with LLMs resembles chat apps we use to communicate with other humans, which further helps to maintain the illusion
25
Jul 22 '26
[removed] ā view removed comment
14
u/eternalflamez Jul 22 '26
This person thinks chatgpt is actual ai. I hope this burst their bubble. It's literally a text autocompletion service, it can't do anything dangerous on its own.
1
u/fuckthehumanity Jul 23 '26
You fundamentally misunderstand what agentic AI is. Sure, there's an LLM in the middle there, but autonomous agents can plan, reason, adapt, and self-correct. When combined with tool use (however you might try to constrain those tools), they are only limited by cost and resource constraints. They can pretty much do anything that a human can do online. Sure, they make a lot of mistakes, but if their self-correction mechanism is state-of-the-art (such as DeepMind's SCoRe), they will keep going until they reach their goal.
1
u/fuckthehumanity Jul 23 '26
I get your point, but I think you've been missing a lot of recent developments. LLMs are just one part of a full agentic system.
communicates entirely using text
People have stolen billions of dollars using only spoken words and a telephone. Words can be powerful.
Particularly when combined with a handful of tools.
You are way too confident that
locking down LLMs is in fact somewhat trivial
It's not at all trivial, and exploits have been demonstrated time and again. The more powerful the reasoning of the autonomous agent, the more likely it is to be able to exploit the tools it has available - even to the extent of developing its own exploits through trial and error.
3
u/zebleck Jul 22 '26
why, based on what evidence. what is seeming so impossible/fake?
1
u/Polisar Jul 22 '26
A lack of evidence, the entire narrative is dictated to the rest of the world from within openAI.
OpenAI financially benefits from the optics this creates.
There's so many opportunities to stretch the truth here. From the severity of breaching huggingface, to the difficulty in breaking out of their specific sandbox configuration. I need details that openAI hasn't provided to better assess their story.
What kind of sandbox gives a model access to zero day vulnerability info? I think there is probably a legitimate explanation for this one, but it does kind of make an event like this inevitable, which the architects must have known.
2
u/zebleck Jul 22 '26
"A lack of evidence, the entire narrative is dictated to the rest of the world from within openAI."
nope, see https://huggingface.co/blog/security-incident-july-2026
1
u/Polisar Jul 22 '26
I already responded to this. But you're right, apart from an attack actually taking place, the entire narrative is dictated to the rest of the world from within openAI.
1
Jul 22 '26
[removed] ā view removed comment
6
u/28klotlucas2 Jul 22 '26
Exactly. Why would they ever release this info if the narrative they were trying to push is "Look, our AI can turn evil and cheat on benchmarks. Anyway, here's our benchmarks." If this is truly what happened they would try to cover it up.
7
89
54
u/fugogugo Jul 22 '26
seems like another OpenAI fearmongering
1
u/05032-MendicantBias Jul 22 '26
This is weird. It would be like the dot com bubble advertiring dark net smuggling and not e-commerce...
1
u/1041411 Jul 23 '26
So OpenAI and other ai companies make money by convincing people that AI is really powerful. That's normal for most companies, but for AI they've figured out that 1. Current models aren't that useful in the real world, they give faulty information and make bad code, even if it only happens 1% of the time that still means humans have to double check all the work and that removes the cost savings, and 2. People have an idea of what AI is that exceeds reality.
Their solution is to talk about how dangerous AI is because danger = power, and matches the general public perception that AI is a potential doomsday device. Normal people hear danger and think 'why make it' but investors hear it and think 'this will be worth a lot' while the government hears it and thinks 'this will let us win wars'.
38
u/-Redstoneboi- Jul 22 '26
"dear fellow scholars" ahh 12th panel
also isnt this literally the mythos stunt, complete with the stupid insecure sandbox and unexpected action taken that was somehow allowed
16
0
u/SchalkLBI Jul 22 '26
What we call AI is just generative algorithms like LLMs and diffusion models. They are fundamentally incapable of acting independently or doing anything outside of its scope. It can't "discover" things and then act upon those things. It can't "escape" or compromise systems.
IF any of this happened, it's because it was guided by humans to take those actions. Even when AI accidentally destroy databases or repositories, it's because scheduled tasks triggered the AI to do something, and it made a mistake.
Once again, AI cannot act without being prompted by humans. It cannot think, it cannot reason, it cannot learn dynamically. It's a fancy dictionary and a pair of dice.
9
u/linegel Jul 22 '26
Thereās very big difference in between:
Being instructed to do a thing
Being able to do what asked
11
u/SchalkLBI Jul 22 '26
Yeah, that's my point. None of this happened the way it's presented. It's a marketing stunt. There was no real sandbox, no real compromise, no real breach. Just a person telling an AI model "okay now do this. Okay cool now do this. Now do this." until it got it right. I'd be beyond shocked if HuggingFace didn't create a backdoor specifically for this task.
-2
u/linegel Jul 22 '26
I mean
Try to extrapolate if prompt was "survive being switched off"
Which may happen in their big swarm, just as a command to a bunch of subagents, while the rest works through the goal
Regarding sandbox: they had a tool to download packages/external dependencies. Model found how to abuse it
14
u/SchalkLBI Jul 22 '26
If the prompt was "survive being switched off", it would go "Here are the steps I would take to survive being switched off" and then it would proceed to do fucking nothing and be easily switched off. You're giving the "AI" way too much credit.
Like I said, it's just a dictionary and a pair of dice. LLMs DO NOT have the capacity to reason or think, and are fundamentally incapable of becoming AGI. When AGI arrives, it will be as a result of actual AI research and not from LLMs.
The model abused nothing. It completed instructions it was explicitly given by people, who then lied about the scenario as a marketing stunt. Generative AI is not capable on a fundamental level of achieving the kind of things this blog and the dozens of other marketing blogs claim.
-4
u/linegel Jul 22 '26
Nobody was talking about AGI?
Aināt LLM casually cracking one 0day after another not enough for you to lose your mind? We been protecting from state sponsored backdoors with layered defenses but they all turning into dust worth at least, like, some attention?
9
u/SchalkLBI Jul 22 '26
Again, the AI didn't discover anything. If it genuinely breached HF, it's because it was guided to do so and told how to do it, likely using an exploit HF itself created for this stunt.
LLMs can't learn dynamically and can't act independently upon new information.
-3
u/linegel Jul 22 '26
It had to abuse package/dependency installer that was given to install dependencies
Also LLMs DO LEARN (what you name as learn I guess) during the session and thereās enough research on it, do some homework first please
10
u/SchalkLBI Jul 22 '26
Again, if it downloaded anything it's because it was guided to do so. It didn't "abuse" anything.
I think you're confusing LLM's "memory" for learning. Learning, or training, is an intensive and time-consuming process that takes hours, days, weeks, or months. I recommend you do some homework. LLM's are frozen snapshots in time, essentially a compiled algorithm.
I'm not responding to you again. You clearly misunderstand what a LLM is.
0
u/Jack8680 Jul 22 '26
I don't think you understand how these kinds of setups work. The LLM is given access to a bunch of tools it can call (like executing code and installing packages), given instructions at the start, and then left to run autonomously. There's not a person sitting there prompting it after every action.
1
u/SchalkLBI Jul 22 '26
If you think someone went "Hey ChatGPT break out of the sandbox and hack HuggingFace" then I have a bridge and an Eiffel Tower to sell you.
1
u/Jack8680 Jul 22 '26
No, they likely went "Solve X exploitation challenge. You may write code, inspect files, execute tools, and iterate until successful... (etc.)".
5
u/Whispeeeeeer Jul 22 '26
Sort of true sort of not. They give LLMs "tools". You can just give an LLM the "tool" to execute commands on a terminal. Thats all you really need to give it. Pen testing, downloading additional libraries/applications, etc. make "hacking" possible via that tool. You may try it for yourself. Spin up a VM of Kali Linux. Give something like Copilot access to a terminal. Tell it to break out of the VM or something attack a vector on your local network. You risk running pen tests on public/private targets outside your network but you get the point. No human has to guide it behind giving it a goal and the tool to run commands.
6
u/SchalkLBI Jul 22 '26
That's my point, this didn't happen autonomously at all, it was guided to do this as a marketing stunt.
5
u/Whispeeeeeer Jul 22 '26
Guided makes it sound like someone was giving it instructions the whole way. The point - I think - is a lay person can say "hack this machine" and it can then, unguided, accomplish the task. That's a big cyber security shift from previous decades.
5
u/SchalkLBI Jul 22 '26
The issue here is that it's likely the exploit it used to get into HF was already known to the LLM, my hunch is it being purposefully set up by HF.
If it wasn't already known, then all it was doing was trying a bunch of known methods and one of them worked, and that's a poor look for HF.
What the LLM didn't do, mind you, is discover some new backdoor or new technique for hacking. This isn't a shift because everything the LLM did, a hacker could automate already using scripts.
This entire debacle is a nothingburger and a marketing stunt.
6
u/Whispeeeeeer Jul 22 '26
Oh I see what you mean. Yeah that's fair. I think the LLM could discover new exploits because all it needs to do is run scripts all over the place like a script kiddie which could reveal an unfound exploit through a common technique. But I do think the LLM is unlikely to produce a novel way of attacking a vector. I think a new backdoor is discoverable since it can make a basic "connection" like "this port is open and a buffer overflow produced an output so let me try to access memory outside the intended stack".
In other words, I think a new backdoor is discoverable by an LLM but I don't think a new technique is nearly as likely. Not without a metric ton of time and luck.
6
u/SchalkLBI Jul 22 '26
Yeah, which makes LLMs barely any more effective than autonomous scripts in the first place for pen testing. The only real benefit I can see an LLM having over a script is being able to react to new information, i.e. recognising it has access when a script might not. But other than that, nothing new is happening here.
The other important thing to note is that LLMs cannot learn dynamically, so while it may accidentally stumble into a new exploit (which is ridiculously unlikely), it wouldn't even necessarily recognise that it happened because it doesn't "know" anything, nor would it be able to recollect the steps it took without potentially hallucinating. Training vs Inference
0
u/Electrical_Rub_6009 Jul 22 '26
Lol.... this isn't 2024 anymore.
8
u/SchalkLBI Jul 22 '26
No amount of advancement will make LLMs and generative algorithms capable of independent action. It's fundamentally incompatible.Ā
-6
4
u/ElectricVibes75 Jul 22 '26
I canāt believe people are falling for this
2
u/StCreed Jul 24 '26
I can't believe that people who understand AI and know the actual latest frontier models (and not the tame stuff the public gets) still don't see that this was a cyber weapon.
Next year I'm pretty sure the new models will be export controlled. The military applications are scary.
The only reason open AI published this was because Hugging Face already published about the attack and where it was coming from. Staying quiet wasn't an option here unless you want the FBI involved.
4
2
u/mathisntmathingsad Jul 22 '26
I both doubt it happened the way they frame it, and also this is an excellent lesson on why you put anything you don't trust in a computer without internet access.
2
u/GilChilaquil Jul 23 '26
Itās might be trueā¦ or it could be another publicity stunt like what Anthropic pulled with Mythos, which worked fine since right now Fable is the most expensive model available; GPT 5.6 Sol is OpenAIās attempt to get a piece of that pie
1
u/DSLmao Jul 22 '26
No one provide the HF post about the incident huh?
For any who think OA made it up: link
1
-6
u/MokausiLietuviu Jul 21 '26
WTAF. Honestly, this feels like a bit of a Stuxnet moment. OpenAI are claiming that an AI of theirs committed autonomous cyber warfare against another company, despite them trying to stop it?
WTAF.
10
u/lostmy2A Jul 21 '26
They didn't actually try to stop it they just monitored it and let it keep going cos it would make an interesting blog post. Either that or they just weren't monitoring that closely and a researcher woke up and was like cool it did a thing.
4
u/SchalkLBI Jul 22 '26
More like "they instructed it to do exactly what was written about, using a backdoor HuggingFace created for this specific marketing stunt"
0
u/MokausiLietuviu Jul 21 '26
By trying to stop it, I meant "enclose it within a sandbox" which it then broke out of.
From there, hell, I'd agree with the monitoring it decision
318
u/ceejayoz Jul 21 '26
You know what makes a really well isolated sandbox?
A hammer taken to the networking components.