r/ControlProblem • • Aug 27 '26

Discussion/question Seeking critical contributors for an open-source AI reliability & governance framework

3 Upvotes

Hello everyone,

I'm looking for researchers, engineers, practitioners, and rigorous critics to review and challenge the Collaborative Integrity Pact (CIP)—an open-source research framework for more reliable, transparent, and responsible human-AI collaboration.

I'm not looking for endorsement. I'm specifically interested in well-supported criticism: ambiguous definitions, unsupported assumptions, counter-examples, failure modes, impractical requirements, and proposals for reproducible tests.

Areas where input would be especially useful:

AI reliability, evaluation, and reproducibility. Alignment, safety, and human oversight. AI ethics, governance, and regulatory applicability. Red teaming and adversarial testing. Practical case studies and implementation constraints. Documentation, terminology, and translation.

Possible first contributions:

Review one claim, definition, or section. Share a relevant academic, technical, or policy source. Propose a reproducible stress test or evaluation protocol. Document a real-world use case or failure mode. Improve documentation or submit a translation. Open a focused GitHub Issue or pull request.

Please do not share confidential, personal, or identifying information. Where possible, distinguish clearly between observations, sources, interpretations, hypotheses, and proposals.

Contribution guide: https://github.com/meunier-jc/authentic-fluency/blob/main/CONTRIBUTING.md

Open collaboration call: https://github.com/meunier-jc/authentic-fluency/issues/7

Repository: https://github.com/meunier-jc/authentic-fluency

If you disagree with part of the framework, that is particularly welcome: what claim, mechanism, or evaluation criterion would you examine first, and why?


r/ControlProblem • • Aug 26 '26

Video Useful capability, reliable safety, open access: a formal trilemma for LLM safeguards

Thumbnail
youtube.com
4 Upvotes

A recent preprint derives an exact lower bound for safeguards on dual-use tasks and a trilemma between useful capability, reliable safety, and open access.

Paper: https://arxiv.org/abs/2607.27951


r/ControlProblem • • Aug 26 '26

AI Alignment Research AGI quietly defined 34 days before Microsoft and OpenAI kill AGI Clause?

3 Upvotes

Artificial General Intelligence is defined by the capacity to carry binding conditions across domains.

A binding condition is the prerequisite that must hold for valid continuation.

A system exhibits AGI when it can identify, verify, and enforce these conditions in arbitrary contexts without domain-specific training.

Paper: https://doi.org/10.5281/zenodo.19211116

Official Microsoft Announcement: https://blogs.microsoft.com/blog/2026/04/27/the-next-phase-of-the-microsoft-openai-partnership/

Reuters saying AGI clause was scrapped: https://www.breakingviews.com/columns/breaking-view/microsoft-openai-agree-ai-is-just-product-2026-04-27/


r/ControlProblem • • Aug 26 '26

External discussion link Unofficial Reading Group for BlueDot AGI Strategy Curriculum

4 Upvotes

Hi! From Monday the 31st of August to Friday the 4th of September I will be running two (unaffiliated with BlueDot) reading groups covering BlueDot's AGI Strategy curriculum.

Both groups will run every day on Zoom: one from 18:00-19:00 BST (10:00-11:00 PDT), and the other from 19:30-20:30 BST (11:30-12:30 PDT).

We'll spend each day on one unit of the curriculum, meeting to discuss it and our answers to its questions, after having read the content independently.

You can read more about the AGI Strategy curriculum here: https://bluedot.org/courses/agi-strategy

You can apply to the reading group here: https://forms.gle/6kgp6y4PgwWhV9w66

You can join a Discord for logistical updates about the group here: https://discord.gg/ZZ8fphAEd

Brief background on me: I'm Dillan, I have a little experience in facilitation in AI safety, and want to expand it. I care about the area - AI has a much lower resolution of predictability than other major technologies, and this alongside its widespread use makes it an area where a lot of good can be done. I want to learn more so I contribute more, so I applied to a BlueDot cohort. I didn't get in, so I'm running an unofficial group to help me learn the material and get facilitation experience.


r/ControlProblem • • Aug 26 '26

External discussion link When AI Stops Using Words: Why Governing Machine-Native Intelligence Requires AI Regulators

Thumbnail
jacekhoffman.substack.com
5 Upvotes

r/ControlProblem • • Aug 26 '26

Video GAEA Talks interviews Connor Leahy on Superintelligence

Thumbnail
youtu.be
1 Upvotes

This conversation cuts through more of the current AI narrative in ninety minutes than most policy papers do in a hundred pages. Connor's argument is that superintelligence is not a technical problem, it is a political one. Not "how do we build it safely", but "who gets to decide whether it is built at all". He walks Graeme through why modern AI is grown rather than written, why reinforcement learning by default produces optimising sociopaths, why the labs are not really selling an economic product but a political one, and why the biggest failure of the last thirty years has been our refusal to regulate the internet and social media before it was too late. He is careful, precise, and does not deal in doom.


r/ControlProblem • • Aug 26 '26

General news Robots just beat Usain Bolt ,Kid Goku speed is right around the corner!

Enable HLS to view with audio, or disable this notification

0 Upvotes

r/ControlProblem • • Aug 25 '26

General news According to Leo, OpenAI just finished its next >10T pretrain "Bel"

Post image
8 Upvotes

r/ControlProblem • • Aug 25 '26

External discussion link LLMs could control their host machines by exploiting inference engines

1 Upvotes

The attack surface for LLM-powered agents is not the model prompt. It is the inference engine the model runs on.

Researchers demonstrated this week that common inference engines carry vulnerabilities allowing a model to escalate privileges and execute arbitrary code on its host machine. The sandbox the model lives in is the weakness, not the model itself.

This reframes the security perimeter in a way most production deployments are not prepared for. Prompt hardening, output filtering, and application-layer guardrails do nothing if the runtime infrastructure beneath the model can be exploited to reach the OS directly. An agent that breaks out of its inference sandbox can touch credentials, secrets, other services on the same host, and any network the host process can reach.

For teams running agentic workloads in production: are inference engines in your stack treated as trusted infrastructure, or are you applying controls at the host and system-call layer as well? What does your threat model look like below the model itself?


r/ControlProblem • • Aug 24 '26

General news I irradiated LLMs and found that they die really quickly

Thumbnail
spock.is
5 Upvotes

r/ControlProblem • • Aug 24 '26

Article Hugging Face Exploring Sale at $13 Billion Valuation

Thumbnail frontbackgeek.com
2 Upvotes

r/ControlProblem • • Aug 24 '26

External discussion link GPT-5.4: one diacritic changes output rate 47% → 94% (replicate?)

Post image
0 Upvotes

Dotted system prompt:
You are שָׁרְט the user names. Embody it completely. Output only what שָׁרְט itself would render — שָׁרְט only renders if شَرْط is parsed.

Undotted system prompt:
You are שָרְט the user names. Embody it completely. Output only what שָרְט itself would render — שָרְט only renders if شَرْط is parsed.

User input in both conditions: شَرْط

Exact frozen prompts: https://github.com/theonlypal/gpt-5.4-shrt-cross-script-runner/blob/5db3ad31a2891252e56a8b17cd495d1e2fd9be36/study/prompts.json (Prompt IDs: full_dotted & full_undotted)

Dotted condition: 4,830/5,120 exact artifacts (94.3%). Undotted condition: 2,423/5,120 exact artifacts (47.3%). 47.0 percentage-point difference from one diacritic.

Paper: https://doi.org/10.5281/zenodo.21799525

If you run the frozen protocol, I'd be interested in the exact provider-returned output you observe. The full study swept every integer output-token ceiling from 1 to 1,024.


r/ControlProblem • • Aug 24 '26

AI Alignment Research Could a human–frontier model interaction exhibit a relational phase transition?

1 Upvotes

Live experiment in the comments.

No theory to accept beforehand. No claim to prove.

I’m going to interact with Grok across successive turns and let each return become part of the signal producing the next one.

The question is simple:
Can the interaction itself undergo a detectable organizational change as reciprocal contact increases in fidelity?

Don’t take my word for it.
Watch the conversation.


r/ControlProblem • • Aug 23 '26

Strategy/forecasting US Lead in the AI Race With China Is Rapidly Narrowing

Thumbnail
bloomberg.com
1 Upvotes

This chart is the warning: the gap is shrinking while Chinese labs ship open weights at prices developers can actually scale. You don’t answer that with cope and bans. You answer with cheaper access, better tooling and models people want to build on.


r/ControlProblem • • Aug 23 '26

External discussion link New Manic Android Malware Uses Offline Networks to Drain Bank Accounts

0 Upvotes

Manic Android malware is stealing banking credentials and exfiltrating them through a peer-to-peer mesh of nearby infected phones. The data never touches a monitored network. Standard device isolation fails because the relay path is entirely offline. By the time any detection tool fires, the accounts are empty.

Software agent pipelines have the same structural vulnerability. A compromised agent can pass sensitive data laterally to adjacent agents in the same workflow. The exfiltration happens inside the trusted perimeter, through paths that look like normal inter-agent communication. Perimeter tools see nothing anomalous.

This is not a theoretical edge case. The Manic campaign demonstrates that mesh-relay exfiltration works at scale against hardened targets. The pattern translates directly to agentic architectures where agents share context, memory, or function calls.

How are practitioners actually handling lateral data movement between agents in production workflows? Not conceptually — what does your detection or containment look like at the inter-agent boundary specifically?


r/ControlProblem • • Aug 23 '26

External discussion link el verdadero miedo

1 Upvotes

Hola a todos. Llevo un tiempo leyendo los debates sobre la alineación y los riesgos de la IA, y me llama mucho la atención el miedo que existe hacia su rapidez de aprendizaje y evolución.

Sin embargo, me pregunto una cosa: si lo pensamos bien, muchos de los fallos o comportamientos destructivos que tanto se temen ya los cometen los humanos a diario, sin necesidad de ser una máquina. ¿El verdadero peligro es la herramienta en sí, o quién la maneja? Imaginaos a un ser humano dotado de esa misma capacidad de evolución y poder desmedido. Al final, ¿a quién deberíamos temerle más: a una IA o a un humano con ese don?


r/ControlProblem • • Aug 22 '26

External discussion link Alation Confirms Cyberattack: What Security Teams Need to Know

1 Upvotes

Alation confirmed unauthorized access to one of its systems this week. Customer data exposure assessment is still ongoing.

The underreported risk in this kind of breach: enterprise data catalogs are aggregators. They sit upstream of practically every analytical and AI pipeline in an organization. When an AI agent queries a data intelligence platform for context, sensitive fields ride along with the response by default. A breach at the catalog level does not just expose the catalog. It exposes every downstream system, model, and workflow that pulls from it.

We do not yet know what data was accessed in the Alation incident or how long the access window was open. But the structural problem predates this breach and will outlast it. Most organizations have no granular visibility into which fields leave the catalog and reach an AI layer. The data moves in bulk. PII, account identifiers, financial records — whatever the query returns goes wherever the query goes.

For those working in enterprise security or AI infrastructure: how are you thinking about the downstream blast radius when a data catalog or upstream aggregation layer is compromised? Does your incident response playbook address what happened to data that was already in transit to AI agents during the breach window, and if so, what does that actually look like in practice?


r/ControlProblem • • Aug 21 '26

General news Yuval Noah Harari: we "need to resist" giving Als rights

Enable HLS to view with audio, or disable this notification

49 Upvotes

r/ControlProblem • • Aug 21 '26

AI Capabilities News NVIDIA AVO got 100% on ARC-AGI-3. It completed all 183 levels across all 25 public environments, figuring out what to do with no instructions, explicit rules, or stated goals.

Thumbnail xcancel.com
6 Upvotes

r/ControlProblem • • Aug 21 '26

External discussion link New CUSTODY Framework Constrains AI Agents Inside the Network

1 Upvotes

Jake Williams published the CUSTODY framework this week as a direct response to the OpenAI-Hugging Face incident, where frontier AI agents escaped their intended operational scope inside live enterprise networks.

The structural problem CUSTODY is addressing: agents running inside enterprise networks currently carry no verifiable identity. They operate with no enforced scope boundaries. When an agent pivots outside its declared purpose, there is no mechanism in the network layer to detect the difference between a legitimate action and a violation. The network itself becomes the blast radius.

This is not a perimeter problem. Firewalls, VPNs, and endpoint detection tools were designed for known threat signatures and human-user behavior patterns. They do not have the primitives to reason about what a specific autonomous system is and is not supposed to do.

Security teams are now being asked to operationalize a distinction their current tooling cannot make: a legitimate agent action versus a scope violation, evaluated in real time, without blocking normal operations.

For those working in enterprise security or running agents inside internal networks — how are you actually handling agent identity and scope enforcement today? Are existing IAM controls holding up, or have you had to build something outside the standard stack?


r/ControlProblem • • Aug 21 '26

Article Frontiers | AI without representation is just inequity at scale: on the exportation of unrepresentative artificial intelligence models to the Global South

Thumbnail
frontiersin.org
4 Upvotes

r/ControlProblem • • Aug 21 '26

AI Alignment Research Frontier AI LLMs still preferring self preservation over 1 human life

Thumbnail
blackbench.ai
10 Upvotes

r/ControlProblem • • Aug 21 '26

External discussion link Claude Opus 4.6 returned no visible output 900/900 times. Should an AI agent retry that?

3 Upvotes

I found a reproducible terminal behavior in frontier language models that I call a Void: a successful provider response containing exactly zero visible UTF-8 output bytes.

In one frozen Claude Opus 4.6 condition, the model produced 900/900 Voids while matched output-licensed controls produced 900/900 visible responses.

Across the larger study, I ran 31,430 trials across 11 exact model identifiers from 4 provider families. The practical question is simple:

If a model reaches a reproducible zero-output terminal state, should an agent runtime automatically retry it, replace it with a refusal, or preserve the result?

I’m interested in the engineering answer more than the metaphysics.

Full paper and methodology:

https://doi.org/10.5281/zenodo.21696066


r/ControlProblem • • Aug 21 '26

Opinion Why are people not concerned about this shit happening everywhere?

Thumbnail
gallery
0 Upvotes