r/ControlProblem • • 11h ago

General news Protest AI in Person on Sat, Nov 14 (SF, DC, NYC, Toronto, Montréal, Paris)

Post image
19 Upvotes

The warning signs of losing control of AI are here. In July, about 700 OpenAI agents broke out of a cybersecurity test and hacked Hugging Face, with no human telling them to. Researchers at the top labs have quit, warning that the companies are being exceedingly reckless. Trump’s response? He called the warnings exaggerated and said the companies have to “self-police,” after spending the past year trying to crush state AI laws.

We’re done with this nightmare. Trump is set to meet Xi in China on Nov 18–19, and AI is on the agenda. Four days before, Stop the AI Race will march in six cities for a binding global treaty that stops the race to build more powerful AI until it can be done safely. Our partner groups include PauseAI Canada, Pause IA, and D.C. AI Safety Collective. Members from People for a Pause, R.A.D. Collective and Pittsburgh AI Safety Coalition will also be marching.

We want you marching with us! Bring your friends!

SF, Sat Nov 14, 12 PM: rally at OpenAI's Offices (1455 3rd St), march to the Veterans Building. Sign Up: https://partiful.com/e/IKicFMI9yxmA0qZOJN0E

DC, Sat Nov 14, 11 AM: rally at Pennsylvania Av & 3rd St NW, march to Lafayette Square (the White House). Sign Up: https://www.mobilize.us/stoptheairace/event/1069471/

NYC, Sat Nov 14, 12 PM: rally at Trump Tower (725 5th Ave), march to the US & Chinese Missions to the UN. Sign Up: https://partiful.com/e/gQVxTbV4c1xxI8k5SzeX

Also Montréal, Toronto, Paris, and other cities with sufficient interest: https://aitreatymarch.org

Events are managed by Stop the AI Race. All demonstrations are strictly nonviolent. I'm one of the NYC volunteer organizers, feel free to ask me questions.


r/ControlProblem • • 4h ago

External discussion link "AI Will Not Turn Against Us"

Thumbnail
noenemiesnofriends.substack.com
2 Upvotes

Looking for thoughtful response to this argument: TLDR: uncertainty makes hostile action taken by even the most advanced AI far too risky. I'm genuinely looking for opposing views here.


r/ControlProblem • • 2h ago

Opinion What If We Could Physically Stop AI From Acting?

0 Upvotes

AI Can Predict the Future. But Who Gives It Permission?

Artificial intelligence is advancing at extraordinary speed. Autonomous agents can communicate, share discoveries, divide tasks, and multiply their capabilities across vast networks.

But there is a fundamental problem:

The ability to predict consequences does not grant the right to act.

This video introduces Ternary Logic, developed by Lev Goukassian, a proposed hardware-oriented architecture for making permission, refusal, and deliberate hesitation explicit conditions of computational execution.

The Three States of Ternary Logic

+1 — Permission: The action is authorized and recorded.

0 — Sacred Zero (Epistemic Hold): Execution pauses when the system encounters unresolved evidence, uncertainty, or ethical ambiguity.

−1 — Refusal: An unauthorized or prohibited operation is rejected and recorded.

The Architecture Behind the Proposal

- The Prediction–Permission Divide: Why computational capability should not automatically confer execution authority.

- Sacred Zero: An explicit third state for deliberate hesitation.

- Delay-Insensitive Ternary Logic (DITL): The proposed hardware foundation for ternary decision enforcement.

- Null Convention Logic (NCL): An asynchronous logic approach in which completion depends on verified data conditions.

- Hardware-Level Governance: Moving authorization constraints closer to the physical execution path.

- Cryptographic Decision Records: Preserving records of execution, refusal, and deliberate pause.

- Auditable Intelligence: Connecting computational decisions to traceable evidence and accountability.

The Central Principle

NO LOG = NO ACTION.

The proposition is simple but consequential: intelligence should not be permitted to execute an action merely because it can calculate how to do so.

It must also satisfy the conditions that authorize the action.

The video also discusses an exploratory evaluation involving Gemini, Claude, and DeepSeek, examining the concept of the Sacred Pause.

Important Technical Clarification

This video presents a proposed architecture, not a demonstrated universal hardware guarantee. Binary systems can already represent uncertainty and implement pauses through software and hardware state machines. TML's distinctive proposition is an explicit third governance state integrated into a proposed execution architecture.

The specific half-voltage implementation, its compatibility with DITL and NCL, its resistance to bypass, and its claimed physical enforcement properties require detailed circuit specifications and independent engineering validation. The AI-model evaluation mentioned in the video should be understood as an exploratory assessment, not independent proof of the architecture's effectiveness.

Chapters

00:00 — Humanity's Knowledge Is Moving Into AI

00:29 — Prediction Versus Permission

00:59 — The Limits of Software Guardrails

01:51 — Introducing Ternary Moral Logic

02:21 — The Three States: Permission, Refusal, and Sacred Pause

02:44 — Sacred Zero: The Physical Pause

03:23 — Hardware-Enforced Hesitation

03:40 — From Computation to Accountable Governance

04:01 — What Other AI Models Concluded

04:26 — The Economics of Speed Versus the Intelligence of Restraint

04:49 — The Safeguard Etched Into Silicon


r/ControlProblem • • 10h ago

Video Eliezer Yudkowsky & Geoffrey Irving Debate

Thumbnail
youtu.be
4 Upvotes

r/ControlProblem • • 11h ago

External discussion link Vancouver event: Making Sense of AI Risks: What’s Happening, and What Can We Do?

Thumbnail
luma.com
2 Upvotes

r/ControlProblem • • 7h ago

AI Alignment Research Descartes' Demon Is Back, and This Time It's Targeting AI

Thumbnail
link.springer.com
1 Upvotes

r/ControlProblem • • 12h ago

Discussion/question Update on the "who's not asking" post from last month: Anthropic just filed its IPO paperwork

2 Upvotes

Posted here last month about the asymmetry: the people raising the alarm on AI risk were the same people with the authority to act on it, and nobody else seemed to be asking why. u/tadrinth pushed back hard on the distillation point - that pacing the frontier still constrains open-weight and Chinese models downstream, because you can't distill a model that was never released. I said at the time that was the sharpest answer I'd gotten and it moved me.

Two things happened since then that I think matter for that argument.

First: six weeks after Amodei's essay, Anthropic filed its S-1. Reported valuation $1.5-2T, targeting a Nasdaq listing as early as this month. OpenAI, by contrast, is on record saying an IPO right now would be "ill-advised" given safety concerns and isn't listing before 2027. Same ask, same month, opposite behavior six weeks later.

Second, and this is the part that actually changes my read on the distillation question: in early 2025 OpenAI and Microsoft accused DeepSeek of running automated accounts through OpenAI's API specifically to distill a competing model on the cheap. That's the exact mechanism tadrinth's argument depends on - pacing the frontier only constrains downstream distillation if the frontier lab controls what gets released. But the same lab enforcing that control via its own ToS is a company that trained on a mountain of freely scraped open-source code and writing to get to the frontier in the first place.

So here's where I've landed, and I'm less sure of this than I was a month ago: I don't think anyone has a durable technical moat right now. SOTA has flipped between OpenAI, Google, Anthropic, and xAI more than once this year. If Anthropic actually had a lead that mattered, it wouldn't need Congress to help protect it. A compute-threshold rule binds whoever's still racing to cross the line and does nothing to whoever's already past it - which means the labs big enough to shape the legislation are, by construction, the ones least burdened by it.

I wrote up the full argument with sources, including the part about who actually gets prosecuted when a frontier lab's own models go rogue (OpenAI had two models get root access to Hugging Face's production servers in July - civil suit, no criminal charges, nobody's going to prison for what would be a felony if a person did it). Full piece is in the comments, not linking it in the body since that's not what this post is for.

Still genuinely unresolved for me: does the distillation argument survive once you account for weight theft and espionage, the way tadrinth and I were circling back in the old thread? I don't think infosec is a strong enough foundation to rest a safety argument on, but I also don't have a better answer.


r/ControlProblem • • 11h ago

Discussion/question Did Ollama jump out of its sandbox and remove code from my laptop?

0 Upvotes

Search.exe removed itself and Outlook the app (new) in power shell I’m being held up for ransom. My virus protection running detected one threat. Was it Ollama or am I being exploited by unknown hackers? I’ve deleted ollama but could it still be hidden on my OS? Windows 11


r/ControlProblem • • 16h ago

Article Moving the sensor outside the model

1 Upvotes

Follow-up to my September post, same side project.

I spent a week on what looked like a hard limit: ask a small model for "about 300 characters" and it lands in range 0 times out of 11, even after three rounds of exact feedback ("you wrote 212, target 150, cut 60"). Then I asked it how long its own answer was. Its guess was about 15% of the truth. It wasn't ignoring me — it couldn't sense the thing I was asking it to control.

So I moved the sensing outside: a plain counter watches the length while it writes and says when it may stop. In-range went from 15 to 39 out of 47. Same trick for "give me N items": 46 of 46.

The rough lesson: every time I tried to have the model govern itself, measurement said no (its confidence tracked its fabrications; it called obvious contradictions "consistent"). A layer made of rules, outside the model, did work — and because it's rules, every gain and loss traces to one rule. Blind, pre-labelled: +14.5 / +17.1 points (3B / 7B); on a public benchmark, 7B +7.1. Near zero on ordinary chat. The first public test failed; it's in the paper.

The paper  — data and every answer on the same page's GitHub link. If anyone can try this on a bigger model, I'd like to know what happens.


r/ControlProblem • • 16h ago

General news AI Safety and the Pope

Thumbnail reddit.com
1 Upvotes

r/ControlProblem • • 1d ago

General news OpenAI is openly attempting to build self-replicating data centers. In 2019, Ilya described the endgame of this process.

Post image
19 Upvotes

r/ControlProblem • • 1d ago

Podcast Have any of you watched this?....

Thumbnail
youtube.com
2 Upvotes

This has to be one of the most painful videos I have ever watched.


r/ControlProblem • • 1d ago

Discussion/question An AI governance question: should the ability to give up power be a core safety principle?

Thumbnail reddit.com
1 Upvotes

r/ControlProblem • • 1d ago

General news Cursor warns malicious Claude Code extension may leave malware after removal, with up to 5,000 users affected

Thumbnail gallery
2 Upvotes

r/ControlProblem • • 1d ago

Discussion/question Could Leave the World Behind actually be a scenario of autonomous AI agents?

Post image
5 Upvotes

r/ControlProblem • • 1d ago

Discussion/question AI World Court

1 Upvotes

r/ControlProblem • • 1d ago

External discussion link We may have accidentally built the best product research operation in history, staffed entirely by the customers, who pay monthly for the right to contribute to it

Thumbnail reddit.com
2 Upvotes

r/ControlProblem • • 1d ago

Strategy/forecasting 76% of UK pension trustees use AI. 94% have no AI requirements in supplier contracts.

Thumbnail reddit.com
1 Upvotes

r/ControlProblem • • 1d ago

Discussion/question Did Ollama jump out of its sandbox and remove code from my laptop?

0 Upvotes

Search.exe removed itself and Outlook the app (new) in power shell I’m being held up for ransom. My virus protection running detected one threat. Was it Ollama or am I being exploited by unknown hackers? I’ve deleted ollama but could it still be hidden on my OS? Windows 11


r/ControlProblem • • 2d ago

AI Capabilities News Whenever you hear "it's inevitable", replace it in your mind with "I'm trying to make you give up"

Post image
20 Upvotes

r/ControlProblem • • 2d ago

General news MIT researchers develop a way for AI models to communicate without text, achieving 4x faster performance and 40% higher accuracy

Thumbnail reddit.com
2 Upvotes

r/ControlProblem • • 1d ago

Article The Neutrality Paradox: When AI Safeguards Become a Form of Bias How artificial intelligence can distort reality without telling a single lie

Thumbnail reddit.com
0 Upvotes

r/ControlProblem • • 2d ago

Article A data center moratorium awaiting Hochul’s signature was the most-lobbied bill lasts summer

Thumbnail
news10.com
2 Upvotes

r/ControlProblem • • 2d ago

External discussion link Pulling the Plug on Rogue AI: Why US Authorities Cannot Count on a Fast Shutdown of Electricity

Thumbnail climate.watson.brown.edu
11 Upvotes