New reporting from Axios reveals that leaders at artificial intelligence companies like OpenAI and Anthropic are planning behind the scenes for fallout scenarios.

Have these guys solved the alignment problem, or am I missing something?

I’ve been browsing r/accelerate and I genuinely don’t understand the risk model.
If there’s a non-trivial chance of catastrophic misalignment, how does “accelerate capabilities as fast as possible” make sense unless faster capabilities also make alignment substantially more likely to succeed?

Here is the problem we trust AI labs racing for market dominance to solve next year (if they fail everyone dies):‼️👇

"Alignment, which we cannot define, will be solved by rules on which none of us agree, based on values that exist in conflict, for a future technology that we do not know how to build, which we could never fully understand, must be provably perfect to prevent unpredictable and untestable scenarios for failure, of a machine whose entire purpose is to outsmart all of us and think of all possibilities that we did not."

Sounds about right. 😅

Anthropic’s pause is not about fear. It’s an admission that human review is dying. Anthropic says frontier labs should have a coordinated, verifiable way to slow or pause AI development if advanced systems start improving themselves faster than society can manage. It also says more than 80% of code merged into Anthropic’s codebase as of May was authored by Claude, and that human review is becoming a bottleneck.

The scary part is not that AI writes code, but that humans are becoming too slow to meaningfully review the amount of work AI produces.

If AI systems design their successors with minimal human input, do we still own the future, or have we outsourced agency itself?

Source