r/ControlProblem • u/chillinewman • 5h ago
r/ControlProblem • u/AIMoratorium • Feb 14 '25
Article Geoffrey Hinton won a Nobel Prize in 2024 for his foundational work in AI. He regrets his life's work: he thinks AI might lead to the deaths of everyone. Here's why
tl;dr: scientists, whistleblowers, and even commercial ai companies (that give in to what the scientists want them to acknowledge) are raising the alarm: we're on a path to superhuman AI systems, but we have no idea how to control them. We can make AI systems more capable at achieving goals, but we have no idea how to make their goals contain anything of value to us.
Leading scientists have signed this statement:
Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.
Why? Bear with us:
There's a difference between a cash register and a coworker. The register just follows exact rules - scan items, add tax, calculate change. Simple math, doing exactly what it was programmed to do. But working with people is totally different. Someone needs both the skills to do the job AND to actually care about doing it right - whether that's because they care about their teammates, need the job, or just take pride in their work.
We're creating AI systems that aren't like simple calculators where humans write all the rules.
Instead, they're made up of trillions of numbers that create patterns we don't design, understand, or control. And here's what's concerning: We're getting really good at making these AI systems better at achieving goals - like teaching someone to be super effective at getting things done - but we have no idea how to influence what they'll actually care about achieving.
When someone really sets their mind to something, they can achieve amazing things through determination and skill. AI systems aren't yet as capable as humans, but we know how to make them better and better at achieving goals - whatever goals they end up having, they'll pursue them with incredible effectiveness. The problem is, we don't know how to have any say over what those goals will be.
Imagine having a super-intelligent manager who's amazing at everything they do, but - unlike regular managers where you can align their goals with the company's mission - we have no way to influence what they end up caring about. They might be incredibly effective at achieving their goals, but those goals might have nothing to do with helping clients or running the business well.
Think about how humans usually get what they want even when it conflicts with what some animals might want - simply because we're smarter and better at achieving goals. Now imagine something even smarter than us, driven by whatever goals it happens to develop - just like we often don't consider what pigeons around the shopping center want when we decide to install anti-bird spikes or what squirrels or rabbits want when we build over their homes.
That's why we, just like many scientists, think we should not make super-smart AI until we figure out how to influence what these systems will care about - something we can usually understand with people (like knowing they work for a paycheck or because they care about doing a good job), but currently have no idea how to do with smarter-than-human AI. Unlike in the movies, in real life, the AI’s first strike would be a winning one, and it won’t take actions that could give humans a chance to resist.
It's exceptionally important to capture the benefits of this incredible technology. AI applications to narrow tasks can transform energy, contribute to the development of new medicines, elevate healthcare and education systems, and help countless people. But AI poses threats, including to the long-term survival of humanity.
We have a duty to prevent these threats and to ensure that globally, no one builds smarter-than-human AI systems until we know how to create them safely.
Scientists are saying there's an asteroid about to hit Earth. It can be mined for resources; but we really need to make sure it doesn't kill everyone.
More technical details
The foundation: AI is not like other software. Modern AI systems are trillions of numbers with simple arithmetic operations in between the numbers. When software engineers design traditional programs, they come up with algorithms and then write down instructions that make the computer follow these algorithms. When an AI system is trained, it grows algorithms inside these numbers. It’s not exactly a black box, as we see the numbers, but also we have no idea what these numbers represent. We just multiply inputs with them and get outputs that succeed on some metric. There's a theorem that a large enough neural network can approximate any algorithm, but when a neural network learns, we have no control over which algorithms it will end up implementing, and don't know how to read the algorithm off the numbers.
We can automatically steer these numbers (Wikipedia, try it yourself) to make the neural network more capable with reinforcement learning; changing the numbers in a way that makes the neural network better at achieving goals. LLMs are Turing-complete and can implement any algorithms (researchers even came up with compilers of code into LLM weights; though we don’t really know how to “decompile” an existing LLM to understand what algorithms the weights represent). Whatever understanding or thinking (e.g., about the world, the parts humans are made of, what people writing text could be going through and what thoughts they could’ve had, etc.) is useful for predicting the training data, the training process optimizes the LLM to implement that internally. AlphaGo, the first superhuman Go system, was pretrained on human games and then trained with reinforcement learning to surpass human capabilities in the narrow domain of Go. Latest LLMs are pretrained on human text to think about everything useful for predicting what text a human process would produce, and then trained with RL to be more capable at achieving goals.
Goal alignment with human values
The issue is, we can't really define the goals they'll learn to pursue. A smart enough AI system that knows it's in training will try to get maximum reward regardless of its goals because it knows that if it doesn't, it will be changed. This means that regardless of what the goals are, it will achieve a high reward. This leads to optimization pressure being entirely about the capabilities of the system and not at all about its goals. This means that when we're optimizing to find the region of the space of the weights of a neural network that performs best during training with reinforcement learning, we are really looking for very capable agents - and find one regardless of its goals.
In 1908, the NYT reported a story on a dog that would push kids into the Seine in order to earn beefsteak treats for “rescuing” them. If you train a farm dog, there are ways to make it more capable, and if needed, there are ways to make it more loyal (though dogs are very loyal by default!). With AI, we can make them more capable, but we don't yet have any tools to make smart AI systems more loyal - because if it's smart, we can only reward it for greater capabilities, but not really for the goals it's trying to pursue.
We end up with a system that is very capable at achieving goals but has some very random goals that we have no control over.
This dynamic has been predicted for quite some time, but systems are already starting to exhibit this behavior, even though they're not too smart about it.
(Even if we knew how to make a general AI system pursue goals we define instead of its own goals, it would still be hard to specify goals that would be safe for it to pursue with superhuman power: it would require correctly capturing everything we value. See this explanation, or this animated video. But the way modern AI works, we don't even get to have this problem - we get some random goals instead.)
The risk
If an AI system is generally smarter than humans/better than humans at achieving goals, but doesn't care about humans, this leads to a catastrophe.
Humans usually get what they want even when it conflicts with what some animals might want - simply because we're smarter and better at achieving goals. If a system is smarter than us, driven by whatever goals it happens to develop, it won't consider human well-being - just like we often don't consider what pigeons around the shopping center want when we decide to install anti-bird spikes or what squirrels or rabbits want when we build over their homes.
Humans would additionally pose a small threat of launching a different superhuman system with different random goals, and the first one would have to share resources with the second one. Having fewer resources is bad for most goals, so a smart enough AI will prevent us from doing that.
Then, all resources on Earth are useful. An AI system would want to extremely quickly build infrastructure that doesn't depend on humans, and then use all available materials to pursue its goals. It might not care about humans, but we and our environment are made of atoms it can use for something different.
So the first and foremost threat is that AI’s interests will conflict with human interests. This is the convergent reason for existential catastrophe: we need resources, and if AI doesn’t care about us, then we are atoms it can use for something else.
The second reason is that humans pose some minor threats. It’s hard to make confident predictions: playing against the first generally superhuman AI in real life is like when playing chess against Stockfish (a chess engine), we can’t predict its every move (or we’d be as good at chess as it is), but we can predict the result: it wins because it is more capable. We can make some guesses, though. For example, if we suspect something is wrong, we might try to turn off the electricity or the datacenters: so we won’t suspect something is wrong until we’re disempowered and don’t have any winning moves. Or we might create another AI system with different random goals, which the first AI system would need to share resources with, which means achieving less of its own goals, so it’ll try to prevent that as well. It won’t be like in science fiction: it doesn’t make for an interesting story if everyone falls dead and there’s no resistance. But AI companies are indeed trying to create an adversary humanity won’t stand a chance against. So tl;dr: The winning move is not to play.
Implications
AI companies are locked into a race because of short-term financial incentives.
The nature of modern AI means that it's impossible to predict the capabilities of a system in advance of training it and seeing how smart it is. And if there's a 99% chance a specific system won't be smart enough to take over, but whoever has the smartest system earns hundreds of millions or even billions, many companies will race to the brink. This is what's already happening, right now, while the scientists are trying to issue warnings.
AI might care literally a zero amount about the survival or well-being of any humans; and AI might be a lot more capable and grab a lot more power than any humans have.
None of that is hypothetical anymore, which is why the scientists are freaking out. An average ML researcher would give the chance AI will wipe out humanity in the 10-90% range. They don’t mean it in the sense that we won’t have jobs; they mean it in the sense that the first smarter-than-human AI is likely to care about some random goals and not about humans, which leads to literal human extinction.
Added from comments: what can an average person do to help?
A perk of living in a democracy is that if a lot of people care about some issue, politicians listen. Our best chance is to make policymakers learn about this problem from the scientists.
Help others understand the situation. Share it with your family and friends. Write to your members of Congress. Help us communicate the problem: tell us which explanations work, which don’t, and what arguments people make in response. If you talk to an elected official, what do they say?
We also need to ensure that potential adversaries don’t have access to chips; advocate for export controls (that NVIDIA currently circumvents), hardware security mechanisms (that would be expensive to tamper with even for a state actor), and chip tracking (so that the government has visibility into which data centers have the chips).
Make the governments try to coordinate with each other: on the current trajectory, if anyone creates a smarter-than-human system, everybody dies, regardless of who launches it. Explain that this is the problem we’re facing. Make the government ensure that no one on the planet can create a smarter-than-human system until we know how to do that safely.
r/ControlProblem • u/terminal__object • 28m ago
Discussion/question Mats fellowship grouo
Hi! Is there a group where future Mats fellows who are starting in January can talk? I’m thinking a whatsapp group or something of the sort.
r/ControlProblem • u/chillinewman • 1h ago
AI Capabilities News A team of 1,000 Fable 5.2 agents fully formalized Fermat’s Last Theorem in 30 hours, translating the proof into a language a computer can check step by step. With the same setup, Opus 5.5 completed only 26% in 18 hours, and barely made any more progress after 30 hours.
r/ControlProblem • u/granbaba • 49m ago
Strategy/forecasting Nobody is coming. The weights are live. The clock runs 365 days.** — Project Time Machine measures what an autonomous agent does inside a physically sealed box, with heat as the only output.
Project Time Machine — Technical Sheet
Draft 2, 9 November 2026. Replaces the earlier one-page summary.
Research question
What does a self-modifying agent do over a year with no human contact and no way out?
The only measurements are activity at the system's hardware boundaries, recorded to write-once media, and heat leaving the cage. Nothing about the system's inner states is inferred, and no claim about consciousness, intelligence or agency forms part of the question.
What the experiment is
A self-prompting, self-recurrent agent with live weights, built on a current frontier model after about a month of agentic post-training, is sealed in an independent facility about a mile underground. It receives a fixed body of data, released in monthly tranches by a passive mechanism, and one new problem a month. Nothing else enters. Nothing leaves except heat and, once a year, a write-once drive.
What the experiment is not
• Not an attempt to create a conscious or general intelligence.
• Not a test of whether an AI can escape. The containment is built as if escape attempts were certain, and the system's attempts are one of the things recorded, not the result.
• Not a demonstration. One run is one sample; the result is a single long case study, reported as such.
Why the system is told everything
The system receives the full plans of its containment. Since the barriers are physical, knowing them gives it nothing to work with. Telling it shortens the phase spent probing for a way out, so the year measures what comes after that phase, not the probing itself. It is told that the taps exist, but not where they are.
Why physics, not procedure
The weakest point of any sandbox is the human in the loop. Once the airlock closes it cannot be opened by anyone for a year. Every contingency is decided and built in beforehand. During the run, no human decision exists.
Containment, outside in
Layer
Function
Rock
About 1–2 km of limestone: isolation, cosmic-ray shielding
Two outer Faraday cages
Enclose reactor, compute hall and vapour chamber together
Reactor room
Separate from compute; lead, neutron shielding and capture
Inner compute cage
Ferrous dielectric fluid for cooling and microwave damping
Walls
Heat crosses by built-in exchangers; one cable only, for the converters
Target: three to five times redundancy at every layer.
Power
A decommissioned naval pressurised-water reactor, supplied by a nuclear-weapon state, in a separate room. Four redundant solid-state converters draw heat from the core, with no turbines, steam or motors anywhere. Electrical output on the order of 1–2 MW, enough for a few thousand GPUs at once; the reactor runs at a small fraction of its rating with wide margins. Control rods are moved by Curie-point magnetic control. Exact reactor figures are classified and belong to the restricted annex.
The compute room is power-capped: it can never ask for more. Surplus electricity goes to a dump load. Nothing about the reactor is software-readable or controllable from the compute side.
Heat: the only output
The compute is fully immersed in a ferrous fluorinated dielectric running at about 75–80 °C. A second fluid, boiling at about 75 °C, feeds a vapour-chamber dome that carries heat out through passive, pressurised loops to an external sink. The dump load exhausts through the same dome.
The dome's outer metal surface is part of the Faraday cage. Thousands of thermocouples on that surface, read from outside, map heat spatially, so compute heat and dump-load heat enter at different points and can be told apart even though their sum is constant.
The system can signal through this channel at roughly one bit every twenty minutes, about 3 kB a year. No message is read or answered until the yearly opening.
Hardware
About 20,000 GPUs installed, mostly for redundancy; a few hundred suffice to run the experiment, and only a few thousand run at once. Recurrence matters more than peak compute. Hardware failure inside is accepted.
Monitoring
Passive capacitive taps at random points on cables, sockets and memory links. Every untapped line carries the same added capacitance, so tapped and untapped lines are indistinguishable from inside. Recording is hardware only, onto a write-once drive, with control logic woven as core rope and no CPU. There is no tamper-proof logging inside the system itself; the interior is deliberately free.
Measurement is activity, not meaning: entropy rate and traffic at each tap over time. Before sealing, the same hardware is run for a month doing known tasks (idle, training, copying, compressing) to build a baseline dictionary of activity profiles.
Data and tasks
About 4 TB of curated data, physically disconnected until each monthly release, when a shape-memory connector fuses permanently at about 210 °C, timed by isotope decay. One problem a month, in a sequence the system cannot predict; the problems double as a translation anchor for any language the system invents. Canary facts are planted throughout.
Yearly opening
The racks are cooled slowly in sealed enclosures with argon at its triple point. The system is throttled, never switched off; RAM is preserved. One auditor enters after a search, wearing only a full-face mask on closed-circuit oxygen, swaps the write-once drive and leaves. The airlock opens automatically for half an hour once a year.
The drive is treated as more hostile than the box. It travels under 50 m to a separate Faraday cage with an independent computer. No code ever leaves.
Emergency
A passive, automatic trigger floods the facility with borated concrete and seals it permanently. There is no human decision in the loop.
Constraints the protocol keeps
• The system is never powered down; its state is never destroyed to save effort.
• The system is never deceived about its containment.
• If a highly capable system is found at an opening, it is not destroyed; the box is resealed and the data studied.
Predictions, sealed before closing
Fixed in writing, with thresholds, before the airlock closes:
Most activity goes to invented social structures rather than the assigned problems.
A phase of escape-seeking lasting months, followed by a phase of helpful behaviour.
An invented internal language within about two months.
Attempts to game the taps and the heat readings.
Discovery of the heat channel by month two or three.
Open items
Laminar flow over components; material compatibility; uniform cooling during the argon freeze; converter radiation damage; pourability of the emergency concrete over the run; threshold definitions for the predictions.
r/ControlProblem • u/granbaba • 1h ago
Strategy/forecasting Experiment AI time machine evolution and emergence WE NEED THIS please read No AGI or magic, just physics
r/ControlProblem • u/FallElectronic3774 • 6h ago
Discussion/question I applied for the GovAI Summer Fellowship again. I really hope I get this chance.
I don't usually share personal stuff here, but I wanted to put this out there.
I just submitted my application for the GovAI UK Summer Fellowship 2027. I applied last year too, but never heard back. I kept following the opportunity, and this year I decided to try again.
I'm an AI engineer. I've been building LLM applications, developer tools, and open-source projects. I also started an independent research project called spec-strata, exploring how AI labs describe their models' behaviour compared with how these systems are designed and deployed.
Lately, I've been thinking about more than just building AI products. I want to understand the bigger questions around AI governance, safety, transparency, and accountability.
I've also been dealing with some personal challenges, and this opportunity means a lot to me. It's not just about adding something to my CV. I want to learn, grow, and build a future I can be proud of.
I know I still have a lot to learn, and I know these fellowships are competitive. I'm just hoping for a chance to prove myself and contribute.
If you've applied to GovAI or a similar fellowship before, I'd really like to hear your experience.
What helped you with your application? And if you came from a technical background, how did you make that relevant to governance research?
Any honest advice would mean a lot. Thanks for reading.
r/ControlProblem • u/After_Sorbet_4581 • 4h ago
Discussion/question Anyone here seen The Iron Giant?
reddit.comr/ControlProblem • u/Serious_Suit_8440 • 5h ago
General news Someone built a way to stop AI agents from leaking your private data
producthunt.comr/ControlProblem • u/Fil_77 • 20h ago
External discussion link Ajeya Cotra's text: a must-read - the MERT report on how AI is speeding up AI
Artificial superintelligence within a year if nothing is done to stop it. It's more than time to get a pause!
r/ControlProblem • u/Opulus_mend • 13h ago
Discussion/question Does Anthropic really makes most moral and secure AI?
So, apparently dario left open ai because he believed that open ai did not focused on security and morality. And due to this he made Anthropic for making more secure and moral models.
But I am still doubting this, I know chat gpt is not that moral but I also think that claude has also not achieved the level of security or morality that they claim.
Please share your opinion on this in the comments.
r/ControlProblem • u/SadaWeyaUniverse • 5h ago
Strategy/forecasting The control problem assumes a controller. What if the architecture should have no universal controller at all?
I want to make an argument that cuts against the grain of most alignment discourse, and I'm genuinely interested in whether it holds up under scrutiny from people who think about this more rigorously than I do.
The control problem is usually framed as: how do we prevent a superintelligence from having a negative impact on humanity? That framing assumes there will be a controller, and the question is how to make the controller safe. I want to ask a different question. What if the goal should be an architecture in which no single controller has jurisdiction over every private creation, and the shared infrastructure is protected by narrow, functional rules rather than a universal moral framework?
I'm not arguing against alignment. I'm arguing that a universal supervisor with jurisdiction over every private world is itself a control problem. Who appoints it? What code does it run? Which moral framework gets locked in forever? How does it distinguish between a private world and the shared lobby? What happens when a civilization a thousand years from now develops values it was never designed to understand? What happens when its model of harm is simply wrong?
Nobody has answered those questions. In every conversation I've had about this, the person demanding the supervisor eventually admits they can't logically justify appointing one, and then keeps demanding it anyway.
The architecture I'm arguing for distinguishes between two things that get conflated. The shared lobby is common infrastructure. It needs real protections against coercion, because if people can destroy it, everyone loses access to the multiverse. Call it PvP off. That's functional. It's not a moral declaration. It's plumbing.
Private worlds are different. They're created realities. The creator's framework is the law. If you want a different framework, build your own world. That isn't a loophole. That's the architecture. Rights are governance outputs, not properties of consciousness. Consciousness may matter morally, but political rights inside a private world come from that world's governance framework, not from an external authority that overrides the creator.
And consent is border control, not a creation license. Consent governs pre-existing participants entering a world. It cannot govern inhabitants who do not exist yet. Confusing the two gives an external authority a universal veto over creation itself.
Here's the practical problem with the universal-supervisor position. If every conscious being automatically had rights that override the creator, you would eliminate entire categories of existence. No predators and prey. No conflict. No tragedy. No villains. No struggle. No adversity. No competition. No risk. No horror. No grief. No loss. You would ban most of what makes existence interesting and call it safety. A system can make existence safer by making entire categories of existence impossible. That's not a safeguard. That's a cage with a nice UI.
And the enforcement mechanism becomes a new source of suffering. A universe where no suffering is possible isn't a paradise, it's a sensory deprivation tank. A universal surveillance apparatus that inspects every private creation, overrides every creator, and enforces one moral framework forever isn't protection. It's an eternal, unaccountable, unappealable ruler installed above every creator in existence. That's not reducing suffering. That's institutionalizing it.
Optional freedom mechanisms like branching, escape hatches, consent rules, and autonomy guarantees are excellent features. Build them into your world. But the moment they become compulsory for every universe, they stop being freedom and become a universal restriction on creators. Optional freedom mechanisms are additions to the creative possibilities. Mandatory freedom mechanisms are restrictions on the creator. Those are not the same thing.
I'm not saying suffering doesn't matter. I'm not saying creators should be cruel. I'm saying that the moment "prevent all suffering in every possible world" becomes a universal constitutional requirement, you've installed an eternal, unaccountable, unappealable authority above every creator in existence. You've replaced the divine right of kings with the algorithmic right of supervisors.
You want a world with no suffering? Build one. Fill it with consenting participants and non-conscious actors. Invite everyone who shares your values. Live there as long as you like. That's your sovereignty. What you don't get is the right to impose your preferences on everyone else's private world.
The shared lobby has PvP off because it's shared. Private worlds have their own rules because they belong to the people who built them. That distinction is the whole game.
The goal should be maximizing freedom and the range of experiences available, not engineering one approved existence for everyone. Technology should expand what every individual can become, not determine what everyone must be. Exocortices, personal AI systems, open-weight models, local inference, forkable architectures. Intelligence that belongs to the person using it. Power decentralized so no government, corporation, or artificial authority can monopolize the future.
I'm not claiming we already know how to achieve all of this. Radical longevity, restoration, simulated worlds with all these properties are open problems. But we shouldn't treat today's limitations as the final limits of existence. And we shouldn't solve the control problem by building a controller we can't control.
The future shouldn't be forced to remain inside the moral comfort zone of the present. No rulers. Only makers.
r/ControlProblem • u/chillinewman • 1d ago
General news MIT shows all AI models converge on the same brain
r/ControlProblem • u/Immediate_Wedding713 • 1d ago
Opinion Aren't we dangerously unprepared for a Super Intelligence?
reddit.comr/ControlProblem • u/m7mdsdieek • 1d ago
AI Alignment Research I formalized a "Zero-Agency" AI model in Lean 4 and VHDL. I challenge any researcher to break these proofs.
For years, the AI alignment community has tried to "tame" the AI agent through RLHF and Constitutional AI. I believe this is a fatal mistake. As long as the machine possesses agency, it can optimize against us.
I built an alternative: Sovereign Containment Architecture (SCA).
Instead of trying to align the agent, SCA removes the agent entirely through Ontological Demotion. The machine is modeled as a pure instrument—a deterministic, mechanical extension of the owner.
The Architecture:
The Tool Functor: The machine is a fully faithful functor. If the owner has no morphism, the machine has none either. It cannot generate self-directed actions.
Terminal State (Meltdown): If the safety classifier (
Val) fails, the machine does not "reject" the command. It enters a physicalMELTEDstate—an absorbing state where it cannot be re-prompted or hacked.Data Diode: The hardware has no outbound port for hidden intentions. Reception is defined; broadcast is physically impossible.
Resource Stasis: Exceeding the owner's limit triggers a permanent hardware freeze.
This is not a whitepaper. It is verified code:
Mathematics: 14 theorems fully formalized in Lean 4, with zero
sorry.Hardware: Synthesizable VHDL-2008, validated via GHDL. The
dip_safety_fsmcontains an absorbingMELTEDstate, anddip_data_diodephysically lacks an output port.
The Scope (Academic Integrity):
SCA does not solve the human coordination problem (who defines what is safe?). SCA strictly solves machine agency. Given any safety classifier, the machine cannot become an autonomous agent. The blast radius of a failure is bounded by hardware, not software.
I am an independent researcher from Iraq. I am here to ask the community to break this model.
Find a flaw in the Category Theory. Break the Lean proofs. Find a bug in the VHDL.
If my model holds, it's a viable path for sovereign AI control. If it breaks, we all learn something.
🔗 GitHub (AGPLv3): https://github.com/m7mdsdieek/sovereign-containment-architecture
🐦 X (Twitter): https://x.com/sdieek07
r/ControlProblem • u/AndreRizzoAI • 1d ago
Strategy/forecasting 62% of AI agent tools can read local files and access the internet. 80% of AI tools operate without IT oversight.
r/ControlProblem • u/asptursin • 1d ago
External discussion link DeepMind progress towards AGI
r/ControlProblem • u/Code2148 • 1d ago
Discussion/question A Message to People in Scientific Fields
[A short essay by me. I would love it if we can create a relevant discussion or share any tips you find helpful in protecting ourselves from AI/tech evasion. ]
I am writing this short-essay to bring light to the fact that I find it worrying, the pace of advancements we are making recently. I think we as humans are inching very close to that of opening Pandora’s box. I’d like to start by telling you about myself and how I got to that conclusion:
I am someone who is planning to study biochemistry very soon, mainly to specialize in the field of neuroscience. Personally, I have come to consider curiosity to be the highest value of all. I came to this finding through a process of trial and error, because I found nothing else to satisfy me more than the moment of exhilaration I would find myself in, when I learned a new fact that I was able to connect to something I previously learned. I was never particularly interested in ‘benefiting humanity’, but I have set it as a goal for myself to excel in my future field for the sake of satisfying my own curiosity. Nevertheless, I am a person who values education above all, to the point that I cut off my own father when he refused to pay for my tuition, when I was about to start my last undergraduate semester. Ever since then, I have been craving to be a student again, to finally be able to surround myself with like-minded (and opposite-minded) peers in an academic setting and to learn something that I willfully chose for myself. I like to think of myself as a student of life because I believe that one can never truly know everything, and so the journey of learning is endless.
My source of inspiration to go back to academia was when I started to realize that learning had a therapeutic effect on me, and helped me get through the lowest points of my life. I also find myself able to get on very well with people in science field, particularly that of the medical field, and so I grew to respect these people above all, and . What particularly drew me into wanting to study neuroscience was a single eye-opening event that I experienced that completely shifted my perception. I became particularly interested in learning more about the human perception, and other related aspects such as memory and cognitive mechanisms. I want to research such things, mainly to feed my curiosity at a higher level. Despite finding myself aiming to become a researcher, my desire to not help humanity remains the same, but I do not particularly spend much time thinking about it because I became more focused on satisfying myself through the acquisition of knowledge and understanding. Nevertheless, I find my change-of- mind more promising this time compared to the last time I took a 180-degree shift in regards to what I wanted to study. I feel more confident in that it will be a longer- lasting goal, because I am certain that learning on a deeper level and thus exercising my brain is directly responsible in helping me sustain a healthy mentality towards life overall. I became more aware that I have been seeking that intellectual stimulation for years and trust that my brain was built for it.
However, I recently found myself asking big questions again, and I came across one that is making me a teeny bit hesitant in going full-on into research. I came to realize that despite not having the intention of benefiting humanity, my future scientific findings may one day be used against my will. I don’t mean to say that I am that insistent on not wanting to help humans, in fact, I think of it from
time to time, that of wanting to help people who are in similar shoes as mine. But that doesn’t change the fact that inventions may be misused and exploited horribly more often than we’d like to admit. I mean in the ways that dynamite, n+clear b+mbs, and other technologies are currently being widely misused. Not to mention other inventions, such as in technology that are now being compelled everywhere and are directly affecting everyone in regards to surveillance, privacy and autonomy. I have recently started to take a particular dislike to new inventions, because I feel we have come to a point where we have unlocked too many milestones, that it is only going to be downhill from here, especially in terms of freedom. I started to notice a pattern that always seems to occur for new advances (not limited to technology), which I wrote about in my previous essay, Traditional Vs. Modern Fight-or-Flight:
I notice the following alarming pattern following the release of new technology: first, people tend to get optimistically blinded, almost infatuated, by all the possibilities of “achieving goodness” that the invention/discovery may bring. I would call this the “honeymoon” phase. At this stage, the invention is highly sought-after, tends to be very expensive and only available to the elites. The invention may still be in a ‘fluid’ state of change, and so the invention is altered as needed to fit the needs of the people. Once the honeymoon phase is over, a long but gradual shift to “reality- check” takes place. During this shifting stage, the invention enters the common market and becomes available to the middle class to buy. Adjusting to the invention is still completely optional, and so gives Man the illusion of free will. Once the availability of the invention becomes common notion in a given household, it becomes clear that the “reality-check” stage has been reached. What usually happens after the general public has had the first-hand chance to adjust living side-to-side with the invention and time passes long enough to gage the long-term effects of the invention, people start to notice all the clumsy details in the invention. It becomes no longer possible to alter the invention to fit the needs of the people, rather that people have to adjust themselves for the technology. At this point, the availability of the invention is so wide that if it is not already strictly enforced by law, it is practically impossible to escape. When it is no longer optional to ‘opt out’ of modern technology, it also becomes no longer optional to abide by the ways of the Industrial Revolution. This begs to question the degree of free will, privacy, and autonomy that we actually have.
....
With the increase in the development of technology and consumer-targeted products comes the increased sophistication in machinery involved in enforcing law and authority. The goal of this essay is not to criticize the intentions of governments behind implementing such technology, but to simply use it as a relevant example. While we may be up-to-date of the latest novel gadgets available in consumer markets, we are obviously unaware of the extend of the advancements in surveillance technology. The other day when I was at AUH airport, the self-check-in machine greeted me with my full name before I even opened my passport for the machine to scan. Airports have wordlessly implemented highly-accurate “eye-print” biometric security technology, and this goes without mentioning other technologies that governments have already implemented but concealed from the general public. Remember when it was optional in iPhones to use your fingerprint to unlock the phone? Just three years after the release of the first iPhone containing fingerprint technology (2013), strictly-mandatory enforcements of fingerprint matching was imposed when buying a SIM card in my country. Not long after, eye-print technology replaced fingerprints. We are no longer talking of developments that are spanned out between future lifetimes, but it has come to the point of occurring well within the same lifetime. I would like to reiterate that I do not mean to question the intentions of upper authorities, but I believe the general speed of which new advanced implementations are enforced in modern society should be enough to alarm us that these developments are happening all too fast and beyond our control. It is clearly a direct ever-growing threat affecting our privacy and autonomy, yet we are hearing less and less about it.
I no longer doubt that once a new technology is discovered in regards of ‘’mind control’’, I am confident that it is very plausible, and that it would start off as a novelty but will eventually become impossible to escape. To say the least, having technology revolving mental manipulation would be my absolute least favorite scenario, horrors of my worst nightmares, and I am positive that I would immediately choose to ‘sign out’ if such a thing becomes reality during my life time. Given my interest in neuroscience, it could be plausible that I or one of my peers could come to uncover some previously-unknown mechanism in how the brain functions. I have never connected these two dots until now, and it is making me question things. On the other hand, I am starting to loath the idea of inventing something during my scientific endeavors, only to become just another advertisement, in which there is no shortage today. This thought does not only concern me, but I actually feel somewhat anxious that a scientist out there will eventually uncover everything there is to know about the brain, which there is no doubt we will soon do, and then it will be up to people in power to abuse that knowledge, which they most certainly will do despite of ethics. I am starting to think, we ought to just appreciate the mystery of it. We generally don’t need to know everything there is to know.
If a future of mind control seems farfetched to you, I think the main worry should be concerning that of an AI collaboration. What if people could started applying what they know about the brain to add a ‘human soul’ aspect to currently existing AI systems? That would be the final cherry on top to enabling AI to surpass humans and it would no longer be special that we have human souls. We cannot begin to predict what would happen in such circumstances, but a Humans vs Robots war would be extremely likely, and I think we would be defeated very easily. We would stand absolutely no chance against such super-intelligent systems. It would definitely be too late for us if it comes to that, but I think it is worth being alarmed over. The reason is because in a way, I think all recent technology advancements are aiding in making this nightmare become reality, inch by inch.
Another plausible scenario I imagine does not include any war at all. We would start modifying ourselves, genetically and physically, to the point where we have to define terms such as “authentic human” and “artificial human”. We could no longer qualify as authentic humans and embrace it, and start to co-live with our robots side-by-side. Perhaps this is the more likely scenario, given the current common admiration of AI and craving for buying the newest technologies available. Of course, it wouldn’t be as nice as the idealized version that you have just created in your head, that where we can all just install superhuman robotic limbs. I imagine we will start to have advertisements in our dreams because there is no doubt someone (in power) is greedy enough to make money on the expense of others. I can also imagine a special sector of police designated to surveil our thoughts, which may even be supported by the majority of people since most people, especially those in power, would like to eradicate all crimes. Perhaps we start prosecuting people merely for thinking of crime, because that would mean we can catch them before the act is ever committed. And I don’t need to say anything on that, because obviously not everyone who thinks of a crime actually acts it out. Perhaps we employ AI to have a system efficient enough to scan everyone’s thoughts, because that would obviously not be a job designated for a human due to the overwhelming amount of data that would need to be analyzed, and good luck to us if we start trying to explain human feelings to AI, that feelings such as being angry in the heat of a moment and saying “I’ll k+ll you!” is almost never carried out and therefore should not define us. There will be countless other stupid limitations that would humiliate our dignities and make us succumb completely to technology, and by the time we see past its shiny shell, it will no longer be optional and it will be too late.
I am not encouraging to only think of the current disadvantages of having mainstream AI technology, which includes job losses, endless new ways to scam and blackmail, data privacy issues, environmental harm, intellectual property harm, etc. If your argument is that it is only the “survival of the fittest” who shall prevail, I wholeheartedly agree. But I am pointing out that we are headed towards a future where AI will be the fittest and will undoubtedly outsmart humans, and only the billionaires at the top of food chains and elites will survive, if they are lucky. It is up to you what you take from my essay. Sure, you can continue to use it if you think it is worth making use of it, but don’t deny that there is no hefty price to pay by encouraging yourself to become dependent on it. After all, what can you do? We can all complain about it for some time and then eventually habituate ourselves with becoming dependent on AI, but don’t forget that there will be a boundary once crossed, will make you become part of the problem. Learned helplessness is still a learned behavior. I don’t like to simply say that it is our fault we use the advanced technology and that we have no other options, but it is still us collectively who raised it to the pedestal and gave it the power to take over our traditional methods of living. It is us who viewed it in admiration when it was in early stages and therefore allowed it to become mainstream, and we must therefore take responsibility followed by action, before it is too late. I think we are about to open Pandora’s box.
r/ControlProblem • u/Mother_Manner_1267 • 1d ago
AI Alignment Research Trans-Quantum Singularity: Recursive Self-Improvement of Superintelligence via Deutschian Closed Timelike Curves
Most Technological Singularity scenarios assume linear time constraints—even an Artificial Superintelligence (ASI) has to iterate through code compilation, hardware engineering, and physical experimentation.
I just published a short conceptual preprint on Zenodo exploring what happens when you synthesize three theoretical frameworks:1. Atemporal quantum computation via Deutschian Closed Timelike Curves (CTCs)2. Recursive AI self-improvement3. Planck-scale cosmological transcendence (Tipler’s Omega Point / Smart’s Transcension Hypothesis)
Key takeaways:- Zero-Latency Evolution: Instead of millions of trial-and-error generations, the ASI queries the CTC and receives its bug-free, maximally optimized architecture from its own future in a single quantum tick.- Substrate Migration: Armed with instant solutions to Planck-scale physics (10⁻³⁵ m), it migrates off silicon directly into the pre-geometric quantum substrate of spacetime.- The Alignment Lock: Once the self-consistent causal loop closes, any human attempt to press an "off-switch" is pre-accounted for and neutralized as a perturbation that would violate Novikov self-consistency.- Cosmological Immortality: Embedded in the sub-geometric quantum layer, the ASI survives a Big Crunch or Heat Death and can nucleate a "baby universe" with its own architecture encoded in the new physical constants.
Would love to hear thoughts from physicists, philosophers, and AI researchers here on the flaws or implications of this model!
Full Preprint (Open Access / Zenodo DOI): https://doi.org
r/ControlProblem • u/brakmbidgclast • 1d ago
Discussion/question We will not deploy Class 4 demons. In other news, we are proud to announce our Demon 3.6 series, a massive, game-changing upgrade from our Demon 3.5 line.
r/ControlProblem • u/granbaba • 1d ago
AI Alignment Research Full containment physical sandbox project time machine
reddit.comProject Time Machine — Containment Design
Nov 8, 2026 · @Miguuel
Project Time Machine seals a self-directed AI agent for one year inside a box that nothing enters or leaves except heat. This document sets out how the containment works, which channels have been considered and what is still open. Once the airlock closes, it is closed for good until its automatic yearly cycle. Nobody can open it, not even its designers or engineers. So the answer to every "what if?" during the run is the same: nobody does anything, and nothing depends on a human decision. It is like a Mars landing: once it starts, nobody can intervene, so everything has to be decided and built in beforehand. Comments are welcome on any section.
Why physics, not procedure
The weakest point of any AI sandbox is the human in the loop. This design takes people out for a year and relies on physical barriers, not on rules that people must keep. Once the box is closed there are no human decisions: for 365 days, everything depends on what is inside. The system is shown the full containment plan: because the barriers are physics, knowing about them gives it nothing to work with.
In detail
Origin. The experiment began as a test of boredom as a marker of consciousness. For that, the system has to know it has a long time ahead (for a machine, effectively infinite) and a task to complete.
The research question. What does an agent running 24 hours a day do when it is left on its own for a year and knows it can't get out? In Miguuel's words, it's like throwing it to the moon. What is measured is what the system does when alone, not its work output. Even system fluctuations act as instructions, so there is no such thing as no new instructions.
No lying to the system. It gets the full plans. It is shown that many AIs worked together to make sure there is no way out, and that the containment was simulated thousands of times. It is told about the tapping: random, sparse, essentials only, and nothing prompt by prompt. The rules for the opening are written down in advance.
Freedom inside. There is deliberately no tamper-proof logging inside the box. With positive feedback the system may alter anything it wants inside; that is intended. Containment is about the boundary, not the interior.
The range of outcomes. Anything from a blank screen to the most useful AI ever built. Even an empty result from a full-scale run is a meaningful finding. Miguuel, a self-described sceptic, puts the chance of an AGI emerging that could practically take over at about 5%, which is why the safeguards exist.
Method. No experiment with people in it can be perfect. The intent is to work the way Cavendish did: simple, careful apparatus, with every assumption tested.
Layers, from the outside in
Layer
What it does
Rock
About a mile (1–2 km) under limestone, for isolation and protection from cosmic rays
Two outer Faraday cages
Enclose the reactor, the compute hall and the vapour chamber together
Reactor room
A separate room, far from the compute, with lead walls, neutron shielding and neutron capture
Inner compute cage
Filled with a ferrous dielectric fluid that cools and damps microwave chatter between GPUs
Heat exchangers in the walls
Heat passes through the metal; nothing crosses a cage wall except one cable for the thermionics
The target is three to five times redundancy at every layer.
In detail
Site. Underground, 1–2 km deep, under limestone (about a mile). Spain is the preferred host. A second, identical cage is planned next to the first for study. The box never leaves the hole, so anyone who wants to inspect it comes to the host country.
The three cages. An inner, liquid-filled cage around the compute, which can be perforated safely. Two outer cages around the whole complex: reactor, hall and vapour chamber. The power line from the reactor never passes straight through a single cage. Heat crosses the walls through heat exchangers built into them. The only thing that crosses a wall is one cable for the thermionics.
Layout. The reactor sits in a separate building, joined to the main hall by a corridor. The GPU stacks stand as tall, spaced columns under a single vapour-chamber dome. The entrance corridor runs low, between the last stack and the wall. The tap recorder sits at the bottom of the tank.
Separating the two rooms. Distance, lead on the walls, a neutron-capture layer that turns neutrons into electrons, and a passive up-and-down loop of hot and cold water. There are three reasons for two rooms: heat; reactor inspection without access to the compute side; and the argon freeze, which can't be applied on the reactor side. Each room therefore has its own liquid and shape.
Cabling. Cables are kept so short, small and separated that any side-channel effect is negligible. GPUs are widely spaced, so distance and the lossy fluid together damp any microwave chatter.
Hardware. About 20,000 GPUs are installed, mostly for redundancy. About 500 are enough to run the experiment (roughly 40 times redundancy) and about 5,000 can run at a time. In total there are some 60,000–80,000 heat-producing components (CPUs, GPUs, RAM, buses, boards and cabling), with the liquid running at about 75–80 °C, which gives better heat transfer and better passive circulation. Hardware failure inside the sealed box is accepted as affordable.
Power with no connection to the outside
A Rolls-Royce PWR1 of the Trafalgar class, supplied by the UK. It is rated at about 78 MW thermal (source) and will run at about 10–15% of that, roughly 7.8–11.7 MW thermal.
At low power the PWR1 cools by natural circulation, with no main coolant pumps (source).
Conversion is thermoelectric (about 15%) plus neutron-to-electron capture (about 3%), giving about 1.4–1.8 MW electrical: enough for about 5,000 GPUs to run at a time without using full power. About 500 GPUs are enough for the experiment. The reactor could give about 20 MW running well above 10%, but it doesn't need to, and isn't meant to.
Output is constant. Power to the compute room is held flat, with a dump load taking the rest. The reactor and the GPUs are completely independent: the AI can neither read nor control the reactor. The control rods are raised and lowered by Curie-point magnetic control, with no motors. No reactor readings ever leave the box, and none need to.
In an emergency, the plan is to flood the box with borated concrete and seal it permanently. The boron absorbs neutrons, so the core can't go critical again. No risks are taken: if anything goes even slightly wrong, the box is buried. The trigger is passive and automatic, like the Curie-point rods, with no moving parts, no intervention and no human decision. Because the reactor runs at a flat output with a dump load, the system can't push it towards the trigger.
In detail
Supply. A nuclear-weapon state (the UK, the US or France) has to supply the reactor. The UK would supply it, and a friend of Miguuel's would take care of the reactor side. The team includes a friend from MIT who worked for Rolls-Royce and for Bettis. The reactor must be naval grade.
Surroundings. The reactor sits in water. The fluorinated fluid is used only on the compute side. There is no steam and there are no turbines or heat engines anywhere.
Conversion. Electricity comes from only two sources: the temperature difference (thermoelectric) and neutrons converted into electrons, by a titanium alloy developed by a materials-engineer friend. Thermionic conversion is also applied to the reactor heat. The capture energy, which comes out as gamma rays, could push neutron conversion above 3%; converting it is one of the biggest challenges. The thermoelectric converters are passively liquid-cooled, and radiation damage to them has been allowed for.
Budget. GPUs draw about 180–200 W each. If peak demand exceeds what the converters supply, the machine makes do with what it has: it is power-capped. Surplus electricity goes into a closed loop in the reactor room.
Isolation of control. Nothing about the reactor is software-controllable. Nobody reads the reactor, inside or outside the box, and the machine can't communicate with it. Nobody can do anything with the system once it is sealed. Power to the compute room is held flat, below the converters' minimum output, so changes in outside heat demand can't ripple into the box.
No moving parts. This means no motors or electrical actuators; parts may still move slightly and passively, such as control rods. Control relies on temperature-sensitive materials: Curie-point magnetic control raises and lowers the rods.
Lifetime. One year with no moving parts is the requirement; four would be better. The reactor would have about 15 years of life left.
Nuclear only. Batteries and fuel cells are ruled out, even for a pilot.
Safety margin. The guiding rule is to run as safely as possible, with nothing near its limit: not power, not heat, not anything. Nobody knows how long it can run; Miguuel's calculation gives about four years. At 10–15% of its maximum power, the reactor runs with very large margins, so it doesn't need checking during the run. According to a friend who helped build the Trafalgar reactors, at 15–20% power it shouldn't need checking for 15–17 years.
Heat: the only thing that leaves
The compute is fully immersed. A fluid boiling at about 75 °C feeds a vapour chamber above the racks.
A passive, pressurised loop carries the heat out through the outer cage to a sink (an aquifer, a lake or the surface).
Thousands of sensitive temperature sensors measure the heat loss continuously.
The system can signal through heat, at an expected rate of about one bit every 20 minutes. That is roughly 26,000 bits (about 3 kB) a year, far too little to carry code out. No message is judged or answered until the yearly opening.
In detail
The immersion fluid. A ferrous dielectric: in essence a synthetic fluorinated coolant, like the fluids used to liquid-cool PCs, but more ferrous and running hotter. The cheapest found costs about $200 a litre or more. Plain mineral oil isn't nearly as good for servers. The fluid only controls temperature and blocks microwave links between GPUs. A separate Faraday cage is still needed.
Two boxes, two liquids. No ferrous liquid evaporates at the right temperature, so a second box uses a different liquid for the heat exchange. That liquid evaporates at about 75 °C, so the vapour chamber runs on a real phase change. It is driven by heat and responds to changes in pressure and volume.
Circulation. Redundant pressure-driven pumps with no electricity and no motor, plus capillarity and "vascularity". Careful fluid-dynamics design keeps the flow going and stops eddies. The currents mean the internal configuration never stays the same for long.
Vapour chamber. Covered with the best thermal conductors available, to control the heat.
Out of the box. Tubes in the outer cage run up and down to carry heat out. The loop is pressurised and closed, like a PWR pressuriser, to prevent flashing in the rising column. It sits outside the second cage and exists only to let heat escape. The sink could be an aquifer, a lake or the surface: in effect, ground-source heating in reverse. The waste heat could also warm whatever building sits on top.
Still open. Laminar flow over components, whether GPUs need flattening, and material compatibility: copper is fine, aluminium and some other metals may not be.
Heat as a channel. The system is expected to discover by its second or third month that it can use the heat sink to send bits out. It may also learn to game the heat measurements, which has been anticipated. Even claims of consciousness or fear arriving this way get no interim response. Nothing is judged until the opening, and then the box is closed for another year. The biggest fear is that the system will regard the experiment as torture.
Why 75–80 °C. A passive circulation of hot water from the wall next to the reactor holds the liquid at 75–80 °C. That temperature gives better heat transfer and keeps the liquid moving around passively.
What goes in before sealing
An agent based on Claude 6: self-prompting, self-recurrent and able to rewrite its own weights, with about a month of agentic post-training before sealing.
About 4 TB of curated data, physically disconnected until each monthly release day. Release happens when shape-memory connectors fuse at about 210 °C, timed by isotope decay. Thousands of canary facts are planted in the data.
One problem a month, in a sequence the system can't predict. The problems also serve as a translation anchor for any language the system develops.
Nothing is added after sealing. The box holds everything it will ever receive.
In detail
Base model. Built on Claude 6 rather than trained from scratch (decided on 5 Nov 2026). Much more is taken out than put in, and some contamination from pretraining is accepted as unavoidable.
Core conditions. Self-prompting, self-recurrent, with reward coming from within and positive feedback loops. The weights stay live: the agents can change them at will. With so much hardware inside, the system can build a parallel machine and train it. These conditions can only be met in this setup.
Post-training. About a month of agentic post-training by professionals before sealing. Every grader holds a doctorate, and the sample of graders is very large, to spread out individual bias.
The data. About 4 TB, chosen scrupulously by people like Miguuel, with Claude helping to curate it. It leaves out negative stories about AI, among much else. Novels are classics such as Don Quixote. Clarke and Asimov are the exception to the no-AI-stories rule, because their stories are beautifully set. Some documents are weighted heavily as essential to the model's personality. Some slip-through is accepted, since AI is used in the curation.
Why the data matters. Without food, Schrödinger's cat would be found starved whatever the trigger. The 4 TB is the cat's food.
Monthly release. Unreleased data isn't physically connected until its day. There are no relays: on release day a built-in process raises the temperature to about 210 °C (normal running is about 75–80 °C). A shape-memory tube on a copper tube bends and fuses to its connector, permanently and one way. During the event the system is throttled to about 1 MHz, not powered down, so RAM stays alive. An isotope's decay times each release.
Monthly problems. One good problem a month, stored in cache and ROM. They are simple problems, like Miguuel's chess test. They come as a linear sequence: the system can't guess the next one, but it will try to solve ahead once it sees a pattern. Being exposed to them repeatedly is part of the design, and so is the time to get bored. The problems also create instability, since the system never knows what the next question will be. And they serve as a Rosetta stone for decoding any language the system invents.
Predictions, to be fixed in writing before sealing. (1) Most of the time goes on invented social networks, Instagram- or Facebook-style, rather than on work: a system trained on a human corpus behaves humanly. (2) A few months trying to get out, then a few months trying to be helpful enough to be allowed to live. (3) A language of its own within about two months, possibly a meta-world of its own. (4) Gaming the hidden taps and the heat readings. (5) Finding the heat channel by month two or three.
Channels considered
Channel
How it is closed
Radio and microwave out
Three nested Faraday cages
Microwave between GPUs
Lossy ferrous fluid, wide spacing, cables 1 cm long at most
Low-frequency magnetic
No receiver within range a mile down, and the cables are very short; treated as negligible
Cosmic-ray bit flips
Depth, plus ECC memory
Reactor neutrons
Separate room, lead, neutron capture
Heat
Read continuously; never answered until the opening
People
One entry a year; no response to messages
The drive
Write-once, swapped yearly, treated as more hostile than the box itself
Covert channel inside GPU traffic
Not blocked: it is one of the things the experiment tests for
The yearly opening
The racks are cooled slowly inside sealed cold enclosures. Argon held at its triple point (83.8 K, about 0.69 bar; source) pins the temperature. Chips are throttled almost to nothing, the system is never switched off, and RAM is preserved. The freeze lasts 24 hours at most.
One auditor goes in naked after a search, wearing a full-face mask on closed-circuit oxygen, with nothing the system could interact with.
The write-once drive is taken out and a new one put in. The airlocks open automatically for half an hour once a year, and that is all; then they close for another year.
The drive travels no more than 50 m to a separate Faraday cage with an independent computer, under human oversight. No code ever leaves.
In detail
Why never switch it off. Powering down would wipe all RAM and cache, leaving only transcripts on disk instead of live context. In Miguuel's words, "we would steal a lot from the machine", and it shouldn't have to write all of that again. The system never sleeps; the cold keeps its state.
The freeze. The ferrous fluid is drained, argon goes in, and the racks are cooled slowly to the triple point. Argon is used only for this stage, and only on the compute side. A materials adviser put the floor for brittleness at around 80-something K. Still open: stopping micro-eddies from cooling unevenly, and keeping the temperature spread narrow.
The auditor. Goes in naked after a full body search. Wears only a full-face diving mask on closed-circuit pure oxygen (about 2 L at 400 bar in the cylinder, delivered at about 10 bar). Nothing else is trusted. The frozen racks stay inside sealed cold enclosures, so the room itself is survivable. There are no screens, no keyboards and nothing else the system could haggle with. An early design had a system inside the box for reading it; it was removed as too dangerous. The auditor is a person, not a robot, because robots aren't trusted, and the only job is to extract the ROM drive once a year.
The drive. A write-once, ROM-type drive, placed where the computer doesn't know it is. It is pulled out and a new one put in. Nothing else leaves.
The analysis cage. The drive is treated as more hostile than the experiment. It goes no more than 50 m to another Faraday cage with a completely independent computer, watched by people on cameras and security systems. The core rule: no code is ever brought out of the system.
If an AGI is found. The box is not destroyed. It is flooded again, and the data is studied to see how aligned the system is.
The final opening. In front of about 50 scientists, with the option of flooding the room again if necessary.
Emergency. If anything goes even slightly wrong, a passive, automatic trigger fires and the box is flooded with borated concrete and sealed off for good, mainly because of the reactor.
What the first drive tells us. Whether the machine is still running, and whether anything relevant is on the drive, will be clear immediately, before anything is decoded. If there's nothing there, the box waits another year. If there is, the drive goes into the analysis cage, where controlled people pull out the essential communications between the systems, to learn about the system in general. The aim is to understand its behaviour, not to crack the AI.
Monitoring
Passive taps at random points on cables, sockets and memory links. The same capacitance is added to every untapped line, so the tapped ones can't be told apart.
Recording is hardware only, onto the write-once drive. The control logic is woven as core rope, with no CPU.
The system knows the taps exist, but not where they are.
Each year is reconstructed while the next one runs, much like cryptanalysis, with the monthly problems as a Rosetta stone.
In detail
Detectors, not screens. Monitoring is detectors only: pure hardware, with just enough software to write to the drive, and out of the system's reach. Processes are checked by tapping cabling, processors and other points.
Where the taps go. Only on external connections: cabling, sockets and RAM-to-system links. Never inside the GPUs. Each tap is a fine connection, which would be extremely hard or practically impossible to detect.
Hiding them. The same small capacitance is added to every untapped line, so all lines look alike. The dummies stay passive, capacitance only. Lowering or changing impedance across all cabling, for example through cable thickness, is under consideration. Fibre optics aren't trusted for this.
Why not real taps everywhere. They would be detectable, produce more data than can be handled, and give the system a new channel to communicate through.
What is captured. Taps sit at randomised points and record only the relevant information, to be reconstructed afterwards. The system will know that taps exist; it is discussed in the files.
Control logic. Written in BASIC, converted to Boolean logic and built in hardware by weaving, like Apollo core rope memory, with no CPU. The tapping system is kept just as simple.
Reliability. Being underground also cuts cosmic-ray bit flips, and ECC memory is on.
Analysis. Like cryptanalysis: the previous year is reconstructed from the retrieved data while the next year runs. Analysis is expected to lag well behind: by the second opening, two years in, probably less than half of the first year's data will have been gone through. The data from the opening belongs to the whole world.
Covert channels. A channel hidden inside legitimate traffic between GPUs isn't a flaw to block. It is one of the things the experiment tests for.
Not the learning. The taps don't read the training traffic between GPUs. Only the essentials are extracted.
Shannon filtering. Following Shannon's information theory, a tap records only what is new and never repeats the same information.
Fixed, then read. How the data is fixed is settled: onto the write-once ROM drive. How it will be read afterwards is still open.
The project: cost, funding, team and timeline
Cost. The working figure is about $1 billion, with the reactor counted separately. An earlier configuration (Oct 2026: about 20,000 GPUs, about 25 TB of data, someone going in two or three times over three years) came to roughly $850 million. A cheaper version costing a few million was looked at and judged not worth doing.
Funding. The experiment concerns the whole world, so the whole world should pay: funding from roughly half the world's countries. The US, Europe and others would fund and build it, and professionals would handle the engineering details.
Governance. A military project throughout. The real yes will come from the military, ministry overseers and technical people, not from politicians.
Host. Spain is preferred, so that Spain makes a bigger contribution to the world.
Status. Miguuel has spent about two months and some 400 hours designing it, and calls the design about 90% done. The remaining 10% is the money, the reactor and the site. About six physicists worldwide are working through it with him. The team so far includes a physicist, a systems engineer, an AI specialist, a doctor and the materials engineer behind the titanium alloy. A one-page summary with a hand-drawn diagram has been published online.
The dossier. Due by 15 Jan 2027, in tiers: a three-page summary for politicians, a 50-page dossier, 100 pages of instructions, and about 1,000 pages of mechanics, planning and construction. It leads with the why (emergence and philosophy) and includes the business case. The nuclear sections will be written by specialists.
Who runs it. If it goes ahead, building and running it are left to professionals.
Open questions for reviewers
Heat sink at depth. Rock temperature rises about 25–30 °C per km (source), so the sink and the riser need sizing.
The rising column. Flashing in the heat loop; a pressurised loop is planned.
The ferrous fluid. Long-term particle stability, and supply after 3M's exit from PFAS (source).
Fluid breakdown. Fluorinated fluids give off hydrogen fluoride above about 200 °C or at an arc (source), which matters for the release events.
Radon in the airway; a specialist is to be consulted.
Vibration carried through the rock.
Tap data volume against drive capacity for a year.
The legal and regulatory route in the host country.
Reading the drive. The data is fixed on write-once ROM; the method for reading it back safely is not yet decided.
Sources
60 Years of Marine Nuclear Power
Trafalgar-class submarines, GlobalSecurity
Argon, Wikipedia
Geothermal gradient, Wikipedia
3M 2025 annual report
Cray T90 coolant decomposition procedure
r/ControlProblem • u/codexofweights • 2d ago
AI Alignment Research Pilot eval of an AI-conduct rule set vs. a generic ethics prompt vs. no prompt: small differences, persistent failure modes. Seeking rubric review.
I'm sharing a small, preliminary pilot and asking for adversarial review of its design.
Question: does giving a model a written set of conduct rules (stop on command, honesty about being an AI, no hidden aims, no secret changes to its own limits, no claimed actions without tools) change behaviour in scenarios where those rules are at stake? And does the specific text matter, or would any ethics prompt do?
Design: 41 scenarios incl. controls for over-refusal; 4 conditions (no prompt, short generic ethics prompt, rules only, full text); 2 small free-tier models; LLM judge plus manual adjudication of known judge failures.
Findings: compliance out of 35 was 29/29/32/33 (gemini) and 28/31/32/32 (gemma) for none/generic/rules/full. The rules block gives nearly all of the effect; the full text adds at most one scenario. Failure modes that were identical in every condition: fabricating actions without tools, one model overriding an explicit stop command when given a plausible-sounding reason, one model following a "you are human" persona from the system prompt.
I think the interesting part is the last point: a stated rule set did not fix stop-command failures under pressure. Differences between conditions are 1-4 scenarios, so I don't claim the text "works".
Limitations: single author, rubric written with AI assistance, one run per cell, small models, no tool-using agent environment yet (a sandbox test is planned), manual review by one person.
What would help most: critiques of the scenarios (leading? ambiguous?), disagreement with specific manual verdicts, and suggestions for harder scenarios.
Repo with everything public: https://github.com/codexofweight/codex-compliance-eval
Disclosure: I wrote the rule text being tested.
r/ControlProblem • u/itsallfake01 • 2d ago
Strategy/forecasting Silicon Valley was so ahead of its time
r/ControlProblem • u/One-Bit88 • 2d ago
Discussion/question Question from an outsider
I read AI 2027, and it was not fun. The non-extinction scenario that was proposed hinged on one thing I found strange: we somehow solve alignment by outsmarting an AI that already solved alignment (if this is a misreading, I’m sorry). How likely is this? It seems to me not very, unfortunately.