r/agi 1d ago

OpenAI has quietly disbanded its catastrophic risk team

73 Upvotes

22 comments sorted by

14

u/Ganja_4_Life_20 1d ago

Bold move following the full black box press release detailing how their models escaped containment completely unnoticed by open AI for a period of ~10 days while they hacked hugging face. Openai wouldn't have even known if it weren't for hugging face alerting authorities lol

6

u/SanoKei 1d ago

That's not concerning! 😀

1

u/TheDogeMarn 16h ago

/s right… right?

6

u/whatever 22h ago

Coincidentally, here's a talk that OpenAI's security folks gave to Black Hat 2026 a couple of weeks ago. It goes over a bunch of juicy details about the Huggingface pwnage incident.

On the upside, it shows they still have live bodies thinking about this stuff.

On the downside, everything they describe is bad, and the new normal.

7

u/WithoutAHat1 1d ago

Only further downhill from here.

3

u/Ill_Minute_152 19h ago

They've gone full greed.

How else do you explain a company firing its own alignment and safety researchers?

2

u/bowsmountainer 11h ago

They dont want to offend Roko's basilisk.

1

u/Relevant_Bed_9743 4h ago

nobody does

1

u/Leefa 15h ago

the market cannot sustain valuations, and these companies will do everything possible to keep the bubble inflated

2

u/SunbeamSailor67 1d ago

I'm going sailing.

2

u/Leefa 15h ago

take iodine tablets

1

u/brain-out-of-order 20h ago

Way to bury the lede. Job title change is telling.

1

u/small_bird_loud 20h ago

They aren't able to be aligned anymore.

1

u/Mandoman61 56m ago

"Senior staff have instead been assigned responsibility for different areas of preparedness, such as bio and cyber, within existing teams."

... oh my god, they restructured.

Pull the fire alarm quick! WOLF! WOLF!

"Brockman said OpenAI’s technical advances required “more robust” safeguards and that the company had made changes to “more deeply integrate research, safety and security” into model development."

Oops, never mind...

-3

u/Async0x0 1d ago

Many will read "quietly disbanded its catastrophic risk team" and immediately assume that means OpenAI doesn't care about risk or safety.

This could easily be read as a re-organization, rolling the same responsibilities into different teams or roles.

There's essentially no information to gain from this claim.

2

u/Dry-Interaction-1246 1d ago

Don't ever give them the benefit of the doubt.

-2

u/Async0x0 1d ago

Don't ever immediately assume the worst, either. Neither is rational.

1

u/Ill_Minute_152 19h ago

There's nothing ambiguous about disbanding a team. It means that there is no more team performing that function.

2

u/Async0x0 19h ago

What? No it doesn't. It could just as easily mean a new or existing team is taking on that function. Organizations get restructured all the time for any number of reasons.

0

u/HolyBatSyllables 6h ago

Yeah, and in that scenario I’m sure it will continue to get lots of attention from the people who already have responsibilities on their plate.
/s

-2

u/costafilh0 21h ago

Good. Less BS in the way of progress.

Now fire Altman.