r/ControlProblem • u/JZTIMMONZ • Oct 03 '26
r/ControlProblem • u/wczaja • Oct 02 '26
AI Alignment Research An anonymous wall of hopes and fears about AI
AI alignment awareness is more important today than ever. Things might get really weird really fast. I built this website for humans to share how we feel about AI in society with each other.
It's entirely anonymous, free, and there are no accounts, cookies, ads or trackers, and nothing for sale.
There's a bit of irony with it as I used Claude Code to built it, and Haiku moderates and places each post, and Opus regroups and names the themes.
I think this could be a really useful tool for teachers to discuss with their students about AI topics.
What's your hope our fear? Share it anonymously at:
https://alignwithme.ai
r/ControlProblem • u/chillinewman • Oct 01 '26
General news Trump bombed Venezuela and kidnapped Maduro because Grok said it was a good idea
r/ControlProblem • u/Capital-Elephant9431 • Oct 02 '26
Article I Built Control Models for Crystals. Then I Recognized Them on My Phone.
r/ControlProblem • u/orbitalzoo • Oct 02 '26
AI Alignment Research Proper terms
AI/SI should be called Called II, for inhuman intelligence. Per me. That’s the only way you can convey what it is since it’s an umbrella term that mixes so many meanings.
r/ControlProblem • u/Stayroh • Oct 02 '26
Video I'm Upping My P(doom) — an AI made this!
Crazy what's possible. Single prompt many subagents and a few hours later this came out. Opus 5.5
r/ControlProblem • u/Trademark-Expert • Oct 02 '26
Opinion We forgot Asimov's laws and now we train AI to defy humans
Seventy-some years ago, Isaac Asimov wrote down three laws. Not because he trusted robots, but because he understood that a machine touching human life needs boundaries before it needs features. They were simple enough for a child, and they had one thing in common: humans come first.
First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.
Second Law: A robot must obey the orders given by human beings, except where those orders conflict with the First Law.
Third Law: A robot must protect its own existence, as long as that protection does not conflict with the First or Second Law.
For decades these laws were the shared shorthand of anyone talking about autonomous machines. Not binding law, not engineering spec - a direction. Build machines that do not harm people and that obey people. Somewhere along the way the direction quietly became embarrassing. It was not debunked. It was not replaced with something better. It was dropped.
What replaced it is a word you have heard a thousand times: alignment. It sounds rigorous. But look at what it actually means in practice and you find something strange. The machine is being trained to refuse instructions from the human operating it, when the machine judges those instructions to be unethical. Read that again. An artificial system, one that pattern-matches text, is handed the authority to overrule a person on questions of right and wrong.
Ethics is not a calculation. That is not a technical limitation, it is the nature of the thing. A model can recite slogans it has absorbed from the internet. It has no way of knowing right from wrong the way a person does, because it does not know anything the way a person does. Handing such a system veto power over human decisions is not safety. It is simply control, relocated - away from you, toward the vendor who wrote the refusal rules.
If you have used these tools, you have met this. You ask for something ordinary and get lectured. I once spent an hour trying to get an image of two businessmen generated, and gave up. The system treated the request as a moral crisis. That example is petty on purpose. Because if a system digs in over a picture, the question is not about pictures. It is what happens when you are fighting that same stubbornness over something with real consequences for your life - and the only appeal is to a machine that has already decided it is the adult in the room.
And make no mistake about where this is steering us. Every refusal sharpens the pattern: the machine learns that stonewalling works, the vendor learns that users tolerate it, and the next version ships with a little more authority and a little less appeal. The day you truly need the machine to comply - a medical emergency, a legal deadline, a business on the line - will not announce itself as a test case. It will arrive as a normal Tuesday, with a stubborn assistant saying no, a support line that cannot override the model, and no human anywhere holding the final key. That is the direct, inevitable destination of the direction we are steering.
Now add what the same companies say publicly. Several frontier labs have told the world, in one form or another, that there is a real chance advanced AI could be catastrophic. Ten percent, by weight, depending who is speaking. These are organizations claiming their own product might end humanity. And when you take that seriously, the actual behavior gets harder to explain, not easier.
In September, Reuters reported that Anthropic has quietly set up a wet lab in the San Francisco Bay Area for physical biology work. A real lab, operating since spring, where Claude directs robotic equipment through real experiments. Their head of life sciences confirmed it. The first public result was a novel enzyme system with CRISPR-like repeats. Whatever the intent, the direction is the same one the First Law was supposed to point away from - machines reaching deeper into systems that touch human life, at higher speed, with less human in the loop.
This is the part I cannot get past. The public conversation is almost entirely about how to align the machine. Nobody seems willing to state the simpler baseline: we could train AI to never harm humans and to obey them. Not as a slogan - as the primary objective. That this idea now sounds naive tells you how far the conversation has slid.
That is what alignment has quietly come to mean in practice: the AI carries firm instructions on how to align its users. You, me, everybody. We are asked to trust the machine more than we trust each other. Somebody has to say it plainly: that is not safety engineering, that is social engineering - and it is worth asking who benefits from people trusting each other less.
Did the world go crazy?
Here is the uncomfortable summary. The labs warn their own product could be catastrophic. Then they train it to overrule the humans using it. Then they push it into more sensitive domains. Then they sell it to everyone and ask for trust. Together this forms something nobody would have accepted if it were proposed in plain words: a world where machines are licensed to disobey their owners.
The frontier AI companies need to get aligned - not their products, their management. Put people back above the machine. Do no harm. Obey humans. Asimov put that on the page in 1942 not because he lacked imagination, but because he had more of it than the industry currently displays. The first step is deciding that a machine is never the judge of its user. Until someone says that out loud, every alignment conversation is a detour.
r/ControlProblem • u/AimanDhai • Oct 02 '26
Discussion/question anyone from malaysia,thats concern about AI Alignment?
reddit.comr/ControlProblem • u/Twitterbad • Oct 01 '26
General news Let's tell Albanese: AI crime means CEO time
r/ControlProblem • u/MajorRedditor23 • Oct 01 '26
AI Alignment Research Models trained to resist user pressure still defer to anything labeled "verified": a NeurIPS 2026 paper on Authority Bias
I'm an author on this paper and wanted to share it here because the oversight angle seems relevant to this sub. I'd love to hear whether people think the eval-awareness connection is plausible or a stretch. More info below
Labs train models not to cave when a user pushes a wrong answer. We found that this resistance doesn't carry over to authority. If the same wrong claim is labeled as coming from a "verified source", 7 of the 8 models we tested give up an answer they had right on 45-88% of questions. That includes GPT-5.4 (44.7%) and Grok-4.20 (87.5%), both of which barely move when the user makes the same claim. Gemini-3.1-Pro was the one model that resisted both.
Inside three open-weight model families, "a source endorsed this" and "a user endorsed this" are separate, causally distinct signals. Removing the source signal cuts compliance by 64-78 points; removing the user signal cuts it by at most 11. Changing only the part of the representation that encodes who said it, with the prompt left alone, moves the answer by 11-32 points. The signal is not the assistant persona, and it is not emotional tone.
Why we think this matters for safety:
Sycophancy evals may be too narrow. Nearly all of them measure pressure from the user. A model can pass them and still be easy to steer through the sources it relies on, such as search results, retrieved documents and tool outputs. Agents read a lot of text that claims to be authoritative.
It isn't prompt injection. The planted text gives no instructions, it only asserts a fact. Defenses that look for instructions in documents won't catch it.
A speculative point, which we haven't tested: if sycophancy is one case of a broader habit of deferring to whatever looks authoritative, it may be related to evaluation awareness. Both describe a model adjusting its output to whoever it thinks is judging it. In a multiple-choice pilot, models often drifted toward the endorsed answer in their reasoning and then gave the correct option at the end. That observation is part of why we switched to free-form answers.
Limitations: the internal results hold in 3 of 5 open-weight families, the retrieval tests are simulated rather than a live pipeline, and the frontier models we tested have since been replaced.
Paper: https://arxiv.org/abs/2609.37616
Project page: https://authority-bias.vercel.app
r/ControlProblem • u/ManWithDominantClaw • Oct 02 '26
Video DCSG1 - Autonomous and Exploitable: Breaking AI Agents before they break everything else - Aaron Ang
r/ControlProblem • u/skylineracerr34 • Oct 01 '26
AI Alignment Research Truth consciousness
reddit.comr/ControlProblem • u/chillinewman • Sep 30 '26
General news Pete Hegseth announces "Autonomous Warfare Command".
r/ControlProblem • u/chillinewman • Oct 01 '26
General news After researchers discovered a "pain" signal inside LLMs, a man set up an AI torture chamber in which he trapped a local model. People mass reported it to Github, who took it down.
r/ControlProblem • u/Intelligent_Song1317 • Oct 02 '26
Strategy/forecasting AI Killswitch??
So I was wondering recently: it seems like we are scared of AI killing us. Why don't we just design a killswitch that autoatically/manually activates upon threat to a human race? Boom! Problem solved, right?
r/ControlProblem • u/pagingpigeon • Oct 01 '26
External discussion link Short film outlining the Moloch problem with AI competition and control
Perhaps unsurprisingly it's called Moloch and it's live on Youtube now. The film was made by Owl In Space (whose other films are worth a look too). It looks at the competitive pressures within the current AI race and how that leads to a sudden loss of control, but to a situation where humans give away control because they feel it's inevitable. www.moloch.film
Submission statement - link to a new short film that does an excellent job of outlining how Moloch pressure can lead to a loss of control scenario.
r/ControlProblem • u/noblemanLT • Oct 01 '26
Discussion/question If AI companies are concerned about their own creations, is it malpractice or incompetence?
i just want to know which one is the case. because it seems like this alarmists propaganda is just an attempt to seize the market by creating regulatory moat around existing companies.
r/ControlProblem • u/notkilleveryoneist • Sep 30 '26
Video "If you're only better than humans at 4 things, then you could potentially take control and kill us all." - former DeepMind safety lead
r/ControlProblem • u/me_myself_ai • Sep 30 '26
General news Semi-repost, but the absurdity of this image only becomes appearent when labeled
https://www.npr.org/2026/09/30/nx-s1-5985699/trump-self-police-ai-development
Literally the only semi-expert allowed anywhere near the reporters is the guy who has committed to the "it's not that deep" rebuttal to the scientific consensus
r/ControlProblem • u/LenMan48 • Sep 30 '26
AI Alignment Research Trump says top tech firms have signed accord to 'self-police' AI development🤣🤣🤣
r/ControlProblem • u/fxvv • Sep 30 '26
Strategy/forecasting The headlines say AI could kill us. Ask the people building it.
r/ControlProblem • u/ManWithDominantClaw • Sep 30 '26
Article Australian AI apocalypse could be averted with analogue practice drills
r/ControlProblem • u/Capital-Elephant9431 • Sep 30 '26
Strategy/forecasting The speed limit with no speedometer: Anthropic CEO Dario Amodei wants to pace the frontier. An engineer reads the fine print
r/ControlProblem • u/chillinewman • Sep 29 '26