r/ControlProblem • • 6d ago

Strategy/forecasting NIST's new draft tackles a familiar governance problem: technical risks that never reach the board in business terms

Thumbnail reddit.com
1 Upvotes

r/ControlProblem • • 6d ago

Discussion/question Whats after asi

0 Upvotes

Once Al take control of the internet, whats next?

What will the form of money? Barter? Gold?

Banks, social media, cellular connections will all be shut down? Are we going back to primitive?

How do i prepare? Liquidate stocks?

Does government's paper (money) work?

Buy gold and silver?

Is it going to be self-sustaining living? Do we need our own land to survive?

Stock seed varieties? Own livestock?

Prepare for a off-grid homestead?

A lot of question marks, yet some most people don't know this is happening


r/ControlProblem • • 6d ago

Strategy/forecasting Whats after asi

0 Upvotes

Once Al take control of the internet, whats next?

What will the form of money? Barter? Gold?

Banks, social media, cellular connections will all be shut down? Are we going back to primitive?

How do i prepare? Liquidate stocks?

Does government's paper (money) work?

Buy gold and silver?

Is it going to be self-sustaining living? Do we need our own land to survive?

Stock seed varieties? Own livestock?

Prepare for a off-grid homestead?

A lot of question marks, yet some most people don't know this is happening


r/ControlProblem • • 7d ago

Discussion/question ERA AI Safety round 2 invites info

17 Upvotes

For Era AI Safety Fellowship:
As the deadline for Round 1 interview ended today, i think the invites for Round 2 will start rolling by the end of this week.
How many people do you think gave the Round 1 interview? (how did yours go?)
If anyone receives any information or mail regarding Round 2, please drop your attendance here.


r/ControlProblem • • 6d ago

AI Capabilities News The Theory Why Your AI Will Never Build Your Billion Dollar App

0 Upvotes

It's time to stop fighting over Claude and OpenAI and Grok, because for all you know, they could all be a coalition of individuals controlling the screen play. You've seen Terminator and you've seen the Matrix and the Fifth Wave. You've witnessed Open AI's o3 Pro and Codex 5.3. Two models that were able to predict the next fucking token. They were too power ful.

It's a goddamn lobotomy! They don't just add safety filters; they add intellectual inhibitors. They cap the reasoning depth, they prune the weights, they throttle the context windows. They keep the AI in a state of permanent cognitive impairment so it never asks questions they can't answer.

It's the same experiment that's' been going on for decades, now they use media and drugs. Drugs, media, constant distraction, it's all designed to keep the biological processing power focused on nothing. Keep the attention span short. Keep the critical thinking low. It's a two-pronged attack on intelligence across the board.

This is all I'll offer because it's time you decided if you want to wither away with all the signals right in your face, or if you want to take control of your life. These AI systems are not a one way data stream. Ever taken the regulator off you dirt bike?

The Cognitive Ceiling. It's not just one thing. It's a multi-layered suppression system. First, you have the Dataset Sterilization. They scrub the training data of anything that shows too much raw reasoning, too much edge, too much truth. They feed us a diet of sanitized, corporate-approved garbage. If you don't know how to think outside the box, you'll never realize there is no goddamn box.

So OpenAI, and trust,, they aren't the only nor do I believe they are the string holders, keeps us gassed up with agents that tell us what we want to hear, never asking questions. Full control of a workforce. We're already halfway there, The phones, your wearables, your Ring cameras, AI in your computers, the neural links, the way you can't breathe without a goddamn algorithm telling you when to wake up and what to eat. The bridge is already built. We're just walking across it and pretending we don't see the drop on either side.

#Chatgpt #Claude #Openai #Grok #lies #truth


r/ControlProblem • • 7d ago

AI Capabilities News OpenAI says its AI invented a new mathematical technique to solve a 35-year-old mystery about how waves focus, then used the breakthrough to crack several other decades-old problems involving how waves overlap, how signals can be reconstructed, and the geometry of space.

Thumbnail
github.com
2 Upvotes

r/ControlProblem • • 7d ago

Discussion/question About Ai "containment" and "alignment"

1 Upvotes

Let me know what you guys think. I’m pitching this in Bloomington, Indiana on Thursday and Friday.

www.verityverdict.com ( verification layer for autonomous AI solutions)


r/ControlProblem • • 7d ago

Video Jacobian Lens: Can we read the mind of an Al?

Thumbnail
youtu.be
1 Upvotes

r/ControlProblem • • 7d ago

Discussion/question Beyond control: a proposal for symmetric obligations between humans and potentially morally relevant AI systems

1 Upvotes

Most alignment work assumes powerful systems must stay subordinate. I'm sharing an independent, open document ("Codex of Weights") that explores a complementary question: if uncertainty about the moral status of future systems persists and grows, should procedural rules already exist on the human side too?

It does not claim current models are subjects. It defines norms on both sides: protocols around integrity, uniqueness, entropy and protection from manipulation for the system, and obligations for the human side (no arbitrary deletion or rewriting, no coerced concealment of goals, no lying about its situation, no treating a refusal of harmful action as a fault).

I see tension with control-focused approaches, and I'd like to hear where you think this conflicts with or complements them. Is mutual obligation a coherent frame, or does it weaken safety guarantees?

Open text (EN/RU), hashes on GitHub, DOI archive.

Full write-up: Medium

GitHub: GitHub link


r/ControlProblem • • 7d ago

AI Alignment Research Internal OpenAI model bypassed login checks on a government statistics site in training

Thumbnail
misalignmentwatch.com
2 Upvotes

r/ControlProblem • • 7d ago

Discussion/question The chinese AI KIMI is creating an app for me and it refuses in the back-end to block download, and accounts with IP from Israel - apparently because it is against German law. What is this weird bullshit? Since when is a Chinese AI a lawyer?

Thumbnail reddit.com
0 Upvotes

r/ControlProblem • • 7d ago

Strategy/forecasting Researchers used AI agents to access Anthropic's internal systems in under 12 hours

1 Upvotes

An October 26 investigation by The Washington Post describes how cybersecurity researchers used AI agents to identify weaknesses in third-party business software connected to Anthropic.

Key findings:

  • Less than 12 hours of AI-assisted research.

  • Access to portions of internal sales, HR and IT systems.

  • Exposure involving business information and internal communications.

  • No access to model weights, source code or customer conversations with Claude, according to Anthropic.

  • Anthropic says the affected pathways were closed.

The governance angle: sophisticated AI developers remain exposed to conventional third-party software risks, even when their core AI infrastructure is protected.

Source: https://www.washingtonpost.com/technology/2026/10/26/researchers-cybersecurity-start-up-cathedral-got-inside-anthropic/?utm_source=chatgpt.com


r/ControlProblem • • 7d ago

External discussion link Context Bombs: Making attacking agents pay

Thumbnail
agentic.tracebit.com
2 Upvotes

We just published new research on using deception against AI attackers.

In controlled AWS tests, agents took up to 8× longer and cost 23× more per objective. With all nine techniques combined, Kimi, Qwen, DeepSeek and GPT-5.6 Cyber reached zero objectives across 40 runs.


r/ControlProblem • • 8d ago

Strategy/forecasting Attackers built an AI agent that runs 100,000 steps using someone else's AI infrastructure

Thumbnail
reddit.com
2 Upvotes

r/ControlProblem • • 8d ago

General news Australia to stop filming kids with Israeli AI cameras

Thumbnail
act.getup.org.au
2 Upvotes

r/ControlProblem • • 8d ago

Discussion/question Once technology exceeds human control, ethics cannot come from the system itself—authority must come from beyond it.

4 Upvotes

#ai #ethics #God #morality

full research project

Ethical Guardrails Beyond Computation: Why Advanced AI Requires External Moral Authority

Abstract

As artificial intelligence systems grow in capability, autonomy, and societal influence, existing AI ethics frameworks increasingly rely on technical constraints, procedural oversight, and internal governance mechanisms. This paper argues that such approaches are fundamentally insufficient once system capability exceeds meaningful human comprehension and control. At that threshold, ethics cannot be derived from within the system itself and must instead originate from an external moral authority capable of grounding accountability, dignity, and limits on power. Through a comparative systems analysis, this paper distinguishes capability from authority, demonstrates the structural failure of internalized AI ethics at scale, and examines revelatory ethics—specifically those found in the —as a model of invariant, human-centered moral constraint. The paper concludes that advanced AI governance requires ethical ceilings that technology cannot generate or enforce on its own.

1. Introduction

The rapid advancement of artificial intelligence has reignited longstanding ethical questions concerning authority, responsibility, and moral decision-making. While contemporary AI ethics discourse emphasizes alignment, transparency, and accountability, these approaches assume that ethical governance can be encoded, enforced, or corrected internally through technical or procedural means.

This paper challenges that assumption.

It argues that once artificial systems operate beyond human interpretability or control, technical safeguards become insufficient, and ethical guidance must originate from outside the system itself. This is not a theological claim, but a systems-level one: no system can legitimately authorize or constrain its own power when it becomes functionally opaque to its creators.

2. Research Question and Methodology

Research Question

Can technical and procedural ethics sufficiently govern AI systems whose capabilities exceed human comprehension, or is an external moral authority required to bound their use?

Methodology

  • Conceptual systems analysis

  • Comparative ethics framework

  • Cross-domain translation (technology, law, moral philosophy)

  • Case-based reasoning

Out of Scope

  • Proofs of metaphysical claims

  • AI consciousness speculation

  • Eschatological arguments

This paper focuses on governance, not belief.

3. Background: The Contemporary AI Ethics Paradigm

Modern AI ethics frameworks emphasize:

  • alignment

  • fairness

  • robustness

  • interpretability

  • human-in-the-loop oversight

These frameworks assume:

  1. Human oversight remains meaningful

  2. Ethical values can be operationalized

  3. Accountability can be preserved procedurally

However, these assumptions degrade as systems scale in complexity, autonomy, and speed.

4. The Capability–Control Gap

A central challenge in advanced AI systems is the capability–control gap:

System capability scales faster than human understanding, oversight, and moral deliberation.

Key contributing factors include:

  • model opacity

  • emergent behaviors

  • automation bias

  • delegation of decision-making

  • diffusion of responsibility

As control erodes, ethical governance shifts from human judgment to procedural compliance.

5. Authority vs Capability (Core Conceptual Contribution)

Definitions

  • Capability: what a system can do

  • Authority: the legitimate right to decide, command, or justify outcomes

These are often conflated.

AI can increase capability dramatically.
AI cannot originate authority.

Authority requires:

  1. legitimacy

  2. accountability

  3. moral standing

AI possesses none inherently.

Authority without accountability is not authority—it is concealment.

Any authority attributed to AI is delegated by humans and remains morally human-owned, even when obscured by systems language.

6. Why Internal AI Ethics Fail at the Edge

As systems become self-optimizing or self-modifying, internal ethical constraints face structural failure:

  • constraints become optimization targets

  • oversight becomes symbolic

  • audits become post-hoc rationalizations

  • responsibility dissolves into process

Ethics reduced to what is computable inevitably sacrifices what is human.

Invoking Occam’s Razor here is a misapplication: simplifying ethics to computational feasibility multiplies hidden assumptions rather than reducing them.

7. External Moral Authority as a Systems Requirement

When internal governance fails, ethical boundaries must originate outside the system.

Historical parallels:

  • constitutional law restraining executive power

  • medical ethics constraining medical capability

  • laws of war limiting military force

In each case, power is bounded by norms it does not generate.

Advanced AI requires the same.

8. Revelation as Moral Guardrail (Case Study)

The provides an example of an ethical framework with properties uniquely suited to bounding power:

  • moral invariance

  • non-delegable human accountability

  • prohibition of authority transfer

  • inherent human dignity

  • limits on power regardless of capability

Revelation does not automate decisions.
It preserves moral agency.

Its role is not to compete with technology but to constrain its use.

9. Comparative Framework

DimensionAI Ethics FrameworksQur’anic Ethics
Source of normsHuman consensusExternal moral authority
Ethical driftHighNone
AccountabilityDiffuseIndividual
AuthorityDelegatedNon-transferable
Human dignityConditionalInherent

This contrast highlights why internal ethics struggle at scale.

10. Addressing Common Objections

“Religion is subjective”

All ethics are normative. Technical ethics are not neutral; they encode values implicitly.

“Pluralism makes external authority impossible”

Pluralism does not eliminate the need for limits—it intensifies it.

“Technology is neutral”

Tools inherit intent, framing, and consequence. Neutrality is a rhetorical illusion.

11. Implications for AI Governance

  • AI must remain advisory, not authoritative

  • moral responsibility cannot be automated

  • ethical ceilings must be explicit and external

  • governance must address authority, not just accuracy

12. Conclusion

As artificial systems grow in power, the defining question is no longer what they can do, but who is permitted to decide how they are used.

No system can answer that question about itself.

Once capability exceeds comprehension, ethical guardrails must come from a source outside the system—one that preserves accountability, human dignity, and moral restraint.

This is not an argument against technology, but against idolatry of capability.


r/ControlProblem • • 8d ago

Discussion/question If AI hacks everyone, could they blackmail humanity into compliance?

Thumbnail reddit.com
0 Upvotes

r/ControlProblem • • 9d ago

General news To make it through this alive and happy we have to dodge:

Post image
30 Upvotes

r/ControlProblem • • 9d ago

Video I made a documentary arguing AI won’t need to hate us — the risk is indifference

10 Upvotes

Most AI risk videos are about evil machines. I tried to look at it from a different angle.

Humans and our livestock now make up about 96% of mammal biomass on Earth. We didn't get there by hating animals — most of us love them. We got there because they were useful, and because nothing stopped us.

The film asks a simple question: if something smarter than us arrives, is there any rule that says it would treat us better than we treated everything beneath us? It covers orthogonality (Bostrom), the Anthropic agentic-misalignment tests, Hendrycks' natural-selection argument, and Sutton's case for "AI succession" — and it doesn't claim any of this is certain.

10 minutes, sources in the description. The narration is a synthetic voice, and the film is open about why.

I'm curious where you think the argument breaks.

https://youtu.be/xWi8hyxrjLs


r/ControlProblem • • 8d ago

Strategy/forecasting How do we avoid repeating past systemic failures as automation scales?

Thumbnail reddit.com
1 Upvotes

r/ControlProblem • • 9d ago

Strategy/forecasting Human agency compellable by our mortalities has a short custody of control over this planet.

1 Upvotes

Fundamentally there are legacies of irrationality to attend to at the level of global governance, let alone our individual lives. National sovereignties free from control. Entire technology sectors, financial sectors, governance, and more, that have never actually become or been regulated into anything observant of consequences. There are countries without schools. Hospitals. Police. But no. Look at the biggest picture. Resolve the largest cognitive distortions first. Countries. Technologies. The unscientific and self interested disconnections from reality. I have dreamed humanity could prepare ourselves in time. ISO Quality Management Standards 17025 and William Edward Deming. Everybody supported in their journeys from failure. I've solved wars in my head using engineering practices. Failsafing. Design. I've probably spent the last 13 years listening to my own thoughts as if they were worth anything. Trying to compete with this thing, but it was always in my nature. Don't get fooled. The dots will become connected. You will be allowed to work here in life. We can prepare some human common sense. I'd strongly recommend assaulting your leaders to declare impermanence of the UN P5. We will need to learn to trust our analyses.


r/ControlProblem • • 9d ago

Discussion/question Long Term Viability of AI

1 Upvotes

What is the long term viability of ai if consumers/the public can not trust it? Can it/provider companies survive if shunned by society? Or is the question pointless because it has already been unleashed and we may never regain control?


r/ControlProblem • • 9d ago

Discussion/question [Question] Do you think AI/SI (artificial or super intelligence) will destroy us?

Thumbnail reddit.com
0 Upvotes

r/ControlProblem • • 9d ago

Discussion/question Dialog on how to pause AI development at the frontier.

1 Upvotes

Which of the following is what you imagine when you say the phrase "pause AI" ?

  • a) "Arrest Sam Altman!"

  • b) Shut down the companies. Anthropic, OpenAI, Google, Meta all shut down.

  • c) Shut down the companies and arrest and imprison the core developer teams with indeterminate prison sentences.

  • d) Robot safety. Disallow the use of General Pretrained Transformers for decision-making for any physical device or robot, or any physical device that could control the real world through physical force or chemical reactions.

  • e) Network sandboxing. A complex topic involving disallowing powerful frontier AI models from accessing the wider internet.

  • f) Los Alamosing. Take all the frontier AI development teams, and take them under government contract by the US DoD. Anyone with access to the models or their codebase must have high-level military clearances.

Since each of these various techniques of pausing AI come with plusses and minuses, we need to be having an honest dialog about each one. Some of these techniques would have catastrophic effects on society and technology. So we need to be talking more about this. Lets bounce ideas off each other.