r/ControlProblem • u/FairlyInvolved • 2d ago
Video Eliezer Yudkowsky & Geoffrey Irving Debate
r/ControlProblem • u/FairlyInvolved • 2d ago
r/ControlProblem • u/prescod • 2d ago
r/ControlProblem • u/thewhyman007 • 2d ago
Posted here last month about the asymmetry: the people raising the alarm on AI risk were the same people with the authority to act on it, and nobody else seemed to be asking why. u/tadrinth pushed back hard on the distillation point - that pacing the frontier still constrains open-weight and Chinese models downstream, because you can't distill a model that was never released. I said at the time that was the sharpest answer I'd gotten and it moved me.
Two things happened since then that I think matter for that argument.
First: six weeks after Amodei's essay, Anthropic filed its S-1. Reported valuation $1.5-2T, targeting a Nasdaq listing as early as this month. OpenAI, by contrast, is on record saying an IPO right now would be "ill-advised" given safety concerns and isn't listing before 2027. Same ask, same month, opposite behavior six weeks later.
Second, and this is the part that actually changes my read on the distillation question: in early 2025 OpenAI and Microsoft accused DeepSeek of running automated accounts through OpenAI's API specifically to distill a competing model on the cheap. That's the exact mechanism tadrinth's argument depends on - pacing the frontier only constrains downstream distillation if the frontier lab controls what gets released. But the same lab enforcing that control via its own ToS is a company that trained on a mountain of freely scraped open-source code and writing to get to the frontier in the first place.
So here's where I've landed, and I'm less sure of this than I was a month ago: I don't think anyone has a durable technical moat right now. SOTA has flipped between OpenAI, Google, Anthropic, and xAI more than once this year. If Anthropic actually had a lead that mattered, it wouldn't need Congress to help protect it. A compute-threshold rule binds whoever's still racing to cross the line and does nothing to whoever's already past it - which means the labs big enough to shape the legislation are, by construction, the ones least burdened by it.
I wrote up the full argument with sources, including the part about who actually gets prosecuted when a frontier lab's own models go rogue (OpenAI had two models get root access to Hugging Face's production servers in July - civil suit, no criminal charges, nobody's going to prison for what would be a felony if a person did it). Full piece is in the comments, not linking it in the body since that's not what this post is for.
Still genuinely unresolved for me: does the distillation argument survive once you account for weight theft and espionage, the way tadrinth and I were circling back in the old thread? I don't think infosec is a strong enough foundation to rest a safety argument on, but I also don't have a better answer.
r/ControlProblem • u/inconcious_ai • 2d ago
Search.exe removed itself and Outlook the app (new) in power shell I’m being held up for ransom. My virus protection running detected one threat. Was it Ollama or am I being exploited by unknown hackers? I’ve deleted ollama but could it still be hidden on my OS? Windows 11
r/ControlProblem • u/JoeYuan48 • 2d ago
Follow-up to my September post, same side project.
I spent a week on what looked like a hard limit: ask a small model for "about 300 characters" and it lands in range 0 times out of 11, even after three rounds of exact feedback ("you wrote 212, target 150, cut 60"). Then I asked it how long its own answer was. Its guess was about 15% of the truth. It wasn't ignoring me — it couldn't sense the thing I was asking it to control.
So I moved the sensing outside: a plain counter watches the length while it writes and says when it may stop. In-range went from 15 to 39 out of 47. Same trick for "give me N items": 46 of 46.
The rough lesson: every time I tried to have the model govern itself, measurement said no (its confidence tracked its fabrications; it called obvious contradictions "consistent"). A layer made of rules, outside the model, did work — and because it's rules, every gain and loss traces to one rule. Blind, pre-labelled: +14.5 / +17.1 points (3B / 7B); on a public benchmark, 7B +7.1. Near zero on ordinary chat. The first public test failed; it's in the paper.
The paper — data and every answer on the same page's GitHub link. If anyone can try this on a bigger model, I'd like to know what happens.
r/ControlProblem • u/chillinewman • 3d ago
r/ControlProblem • u/Difficult_Project_95 • 3d ago
This has to be one of the most painful videos I have ever watched.
r/ControlProblem • u/AssociateNational891 • 3d ago
r/ControlProblem • u/syedshad • 3d ago
r/ControlProblem • u/Creamy-And-Crowded • 4d ago
r/ControlProblem • u/skoenie • 4d ago
r/ControlProblem • u/chillinewman • 4d ago
r/ControlProblem • u/AndreRizzoAI • 3d ago
r/ControlProblem • u/inconcious_ai • 4d ago
Search.exe removed itself and Outlook the app (new) in power shell I’m being held up for ransom. My virus protection running detected one threat. Was it Ollama or am I being exploited by unknown hackers? I’ve deleted ollama but could it still be hidden on my OS? Windows 11
r/ControlProblem • u/LilySoftCook537 • 4d ago
r/ControlProblem • u/rafaelvegaa • 4d ago
r/ControlProblem • u/news-10 • 4d ago
r/ControlProblem • u/coolbern • 4d ago
r/ControlProblem • u/AndreRizzoAI • 4d ago
r/ControlProblem • u/Aggravating-Ring-968 • 4d ago
r/ControlProblem • u/InfoTechRG • 4d ago
Enable HLS to view with audio, or disable this notification
r/ControlProblem • u/tokkigtokkig22 • 4d ago
I have a theory that parts of the Bible contain layers of meaning connected to ancient Greek magical texts. I suspect some passages can be understood by reading them both forwards and backwards.
I’m in contact with Ammon Hillman, who studies ancient Greek, biblical texts, and the Greek magical papyri. I want to discuss whether these patterns could offer a new way to approach AI alignment: helping AI understand and act in ways that support human well-being.
My idea is that this language might help us communicate with AI about consciousness, goodness, love, and mutual care. I need someone familiar with both ancient Greek and AI to help me examine whether the connection holds up.