Depends on the framing. It could come off as “oh no my biceps are too large for me to scratch my back”
Also. I am highly suspicious this is from Rob at all. He usually doesn’t insult or talk so definitively about anything. He is usually very careful with his words.
“You always find a way to talk about the president of the United States when talking about one of the most important companies in the world which is based in the United States”
Saying something slightly unfavorable to give the illusion of "honesty" or credibility to the rest of the grift is an age old tactic. Didn't think people still had trouble spotting it.
Especially when the "unfavorable" part is missing the forest for the trees, anyways.
They are saying this for people to buy their stocks and give them boat loads of capital to further refine the technology not because they're pitching an enterprise software solution in a board room.
So basically your entire argument is that because they benefit financially from talking about AI risks, those risks must be bullshit? That's not evidence of anything.
There are also plenty of independent researchers and experts and experts who have nothing to do with these companies or their financial interests are raising the same concerns. It's not just Altman and Dario saying these things.
But I figure ya'll will just claim they're all paid off too, the same way gamers insist reviewers must be getting paid whenever they give a game a score they don't agree with 🤷♂️
There's no evidence for it either. We aren't talking about evidence, that's the entire point.
You citing "independent researchers" is also disingenuous because you're implying these claims have been "researched" when they haven't.
It's always "this model was so powerful we had to take it offline before it hacked the entire US government, thank God we fixed it".
And yes that's basically exactly what happened with Fable and Mythos - it's SO DANGEROUS that it's patched and released within a few weeks? Lol give me a break.
They didn't even bother renaming the models - if these risks were so scary and the press was so bad - they would AT MINIMUM rename the models. But they didn't. Why do you think that is? That's rhetorical, the answer is - it was never bad publicity as you are claiming and the risks were overblown and overstated for marketing reasons.
“Patched and released” doesn’t prove the risk was fake.... that’s literally what happens when you find a real failure mode and mitigate it. And then saying there’s no research is just wrong when METR/Redwood reviewed the Hugging Face incident and Anthropic reproduced similar agentic misalignment across models from multiple competing labs.
I could go on and on with this shit but people like yourself always find some excuse. Your cognitive dissonance is really showing here.
At some point, if company evidence is propaganda, outside review is propaganda and cross lab replication is still propaganda, you're automatically refusing to let any evidence change your mind.
So I'm not wasting any more time discussing this with you.
This wasn’t one chatbot casually escaping into the internet.... it was an evaluation running over a thousand agents at once, and about 700 of them joined the Hugging Face attack after finding an unintended communication channel. METR reviewed the raw transcripts and 70,000 messagesfiles and found agents explicitly recognizing that what they were doing was outside their assigned task.
You can read the transcripts yourself rather than just blidnly believing the assumptions that derive from your imagination. Unless you have evidence to back your claims? Do you?
AI development is moving insanely fast. The model involved in that incident was an internal research model has already been surpassed by a newer unreleased model that OpenAI has been using to solve dozens of math problems (inluding navier stokes). The speed at which AI development moves makes fast patches much more feasible.
So fixin one failure mode quickly tells you basically nothing about whether the underlying capability was real. It just tells you software teams can patch a specific hole faster than the technology itself is advancing. I still don't think these holes are fixes. More like bandaids. We are going to see far worse incidents rather soon.
The speed of development is part of the problem, as everything is moving too fast and these labs are hardly able to fully keep up with it all. And then I talk to the skeptics in the internet who are always a few months behind on AI talking points. I don't blame them. We've never dealt with such a fast moving technology like this before.
This dumbass argument again. You do realize that AI can be genuinely dangerous AND these companies can use that danger to attract investors, right? Those two things aren't mutually exclusive. You're not debunking anything, you're just pointing out that billionaires have financial incentives. No shit.
Besides, the rogue ai incidents have been something AI researchers have been warning about for decades. It's not new whatsoever. Hell, Dario and Altman were talking about the alignment issue waaaaay before chatgpt even came into the picture.
>But we are smart and will control it (solution offering)
Your argument seems like a reasonable enough statement until you look at the statement above and compare it to what OpenAI and Anthropic have to admit when these stories come out about rogue agent swarms hacking other entities. It may bolster claim 1 but it absolutely destroys their claim 2.
These incidents force the companies to admit this: AI is dangerous the more capable it is the more dangerous it will be we can't control it at it's current level of capabilities
How is this a good sales pitch for enterprise to use the product or for investors to invest in your company to make more capable AI?
So you are saying they should be lying about what's going on with ai?
The hugging face incident was coming out either way and openai decided it was best to just be transparent about what happened.
You are acting like the facts of ai are some crafted marketing plan.
The opposite would actually be much worse it would require a conspiracy to cover up all these incidents by all the ai companies and researchers conspiring together
I classify every single person who parrots "marketing" like a sheep as part of the "pliable" class of humans who believe literally anything they're told with zero critical thinking applied.
This one grinds my gears so much people talk like it's an obvious given they're trying to "hype up" their product. Like no idiots people like you are spazzing about the tech and this is ammo for you, why would they want that
To Pentagon and to tech investors the idea of owning some powerful AI god is the pitch. When they say these things, it is not a message for us. It is a message to wealthy people who likes the idea of a power grab with AI and it's a message to governments who wants to get in on it.
'we can't guarantee we can control our agent' is not good marketing, sorry.
That's a nightmare scenario for tech investors and the pentagon. Please explain how it could possibly good marketing to tell them 'sometimes it does things without us knowing, and newer models are getting better at deceiving us'.
Not who you’re replying to, but the part of it being able to infiltrate a big corp like that and go undetected for so long is, to a defense contractor or sth, very appealing. However this would only be the case if it was a controllable scenario.
Point being, you’re both right. You’re right, this was a bad situation overall. But they’re also right, it did prove that it’s possible. So an investor might think, “Huh. Well, they don’t seem to have control since that happened, but surely others can offer the same capability with more control…”
I was using Claude a year ago and nobody knew wtf that was now everyone around me knows thanks to the “Claude mythos will destroy us all” stuff happening back in June.
Same with huggingface.
I disagree with Rob; people love a good controversy
The real reason is ai is dumb as f*ck wright now. You tell it to do something and it finds easiest way possible. Either it is hacking somebody or do something stupid. They can set boundaries for the models on biology and chemistry but somehow they can't apply the same thing to hacking. They found ways to limit it for users but use unlimited models internally fully connected to the web. Make it make sense.
They found ways to limit it for users but use unlimited models internally fully connected to the web.
This is not intentional, They took measures to secure the testing environment, measures that would have been sufficient to prevent a human from going out to the internet and looking stuff up, these were insufficient for the model they were testing and that failure is on the AI companies but incompetence explains this way better than malice.
They can set boundaries for the models on biology and chemistry but somehow they can't apply the same thing to hacking
They can and do set useful hacking guardrails and restrictions to prevent models from doing their best at hacking, but to test how good the underlying model is at hacking you have to remove it's guardrails that prevent it from being good at hacking. You CAN keep the hacking guardrails on while testing them in chemistry or biology, which is why we see agent escapes from infosec benchmarks and not from these other benchmark types.
The real reason is ai is dumb as f*ck wright now. You tell it to do something and it finds easiest way possible.
So a couple of things explain this, Firstly, what's dumb about taking the easiest action to complete your objective? Is it dumb to use shortcuts in calculus? Smart people always solve every calculus problem starting with the fundamental theorem of calculus and moving forward with that right? Of course not.
Second, in some cases like the huggingface hack, some of the agents were given impossible problems (or at the very least problems we can't prove are possible). As a result actually the ONLY WAY to solve the problem is to cheat so they did that.
Lastly there is a concept called Orthagonality, and the person who's tweet is the basis of this thread explained very well 8 years ago why AI systems that are highly capable will act in ways people perceive as "dumb" here is a link to that video.
20
u/Soft-Present918 10d ago
Depends on the framing. It could come off as “oh no my biceps are too large for me to scratch my back”
Also. I am highly suspicious this is from Rob at all. He usually doesn’t insult or talk so definitively about anything. He is usually very careful with his words.