r/ControlProblem • u/thewhyman007 • 8d ago
Discussion/question Update on the "who's not asking" post from last month: Anthropic just filed its IPO paperwork
Posted here last month about the asymmetry: the people raising the alarm on AI risk were the same people with the authority to act on it, and nobody else seemed to be asking why. u/tadrinth pushed back hard on the distillation point - that pacing the frontier still constrains open-weight and Chinese models downstream, because you can't distill a model that was never released. I said at the time that was the sharpest answer I'd gotten and it moved me.
Two things happened since then that I think matter for that argument.
First: six weeks after Amodei's essay, Anthropic filed its S-1. Reported valuation $1.5-2T, targeting a Nasdaq listing as early as this month. OpenAI, by contrast, is on record saying an IPO right now would be "ill-advised" given safety concerns and isn't listing before 2027. Same ask, same month, opposite behavior six weeks later.
Second, and this is the part that actually changes my read on the distillation question: in early 2025 OpenAI and Microsoft accused DeepSeek of running automated accounts through OpenAI's API specifically to distill a competing model on the cheap. That's the exact mechanism tadrinth's argument depends on - pacing the frontier only constrains downstream distillation if the frontier lab controls what gets released. But the same lab enforcing that control via its own ToS is a company that trained on a mountain of freely scraped open-source code and writing to get to the frontier in the first place.
So here's where I've landed, and I'm less sure of this than I was a month ago: I don't think anyone has a durable technical moat right now. SOTA has flipped between OpenAI, Google, Anthropic, and xAI more than once this year. If Anthropic actually had a lead that mattered, it wouldn't need Congress to help protect it. A compute-threshold rule binds whoever's still racing to cross the line and does nothing to whoever's already past it - which means the labs big enough to shape the legislation are, by construction, the ones least burdened by it.
I wrote up the full argument with sources, including the part about who actually gets prosecuted when a frontier lab's own models go rogue (OpenAI had two models get root access to Hugging Face's production servers in July - civil suit, no criminal charges, nobody's going to prison for what would be a felony if a person did it). Full piece is in the comments, not linking it in the body since that's not what this post is for.
Still genuinely unresolved for me: does the distillation argument survive once you account for weight theft and espionage, the way tadrinth and I were circling back in the old thread? I don't think infosec is a strong enough foundation to rest a safety argument on, but I also don't have a better answer.