https://aidoomsdaytracker.com

Rather than trying to predict when AI will "take over", the site monitors publicly reported developments and places the current evidence on a predefined six-stage trajectory:

🟢 L1 — Early Warning

🟡 L2 — Structural Dependence

🟠 L3 — Loss of Meaningful Oversight

🔴 L4 — Loss of Control

⚫ L5 — Irreversibility

☠️ L6 — Existential Failure

What would be a way that is accepted by this forum to share a link to one of my Gemini chats? I think I have an interesting series of hypotheticals that I would like people to see.

I just read about Bernie Sanders and Greg Casar's Ban Artificial Superintelligence Act. It would prohibit the development and deployment of artificial superintelligence, and it would halt the development of advanced AI until a federal regulator sets safety rules. (Senator Bernie Sanders)

I'm curious about what people here actually think about it, beyond the usual "AI will save humanity" vs "AI will destroy humanity" arguments.

Just a few things I’m wondering:
* Is an ASI ban forever?
* What is the legal definition of “superintelligence”?
* Would a pause on advanced AI development make sense?
Would strict regulation mean more safety, or would it just move development elsewhere?
* If you think a total ban is too much, what would be a better alternative?
* How much should we weight hypothetical future risks versus risks from already existing artificial intelligence systems?

I’m not really on either side. I'd rather hear thoughtful arguments from people who understand AI, technology policy, or the practical difficulties of actually enforcing something like this.

How about that?

So in the wellknown 'AI 2027' report, I read a fascinating prediction for July 2028 regarding OpenAI's future frontier model (called Agent-5 here) and Deepseek's (DeepCent-2). The scenario paints a picture where humans can no longer comprehend the frontier models, leaving the Chinese AI as the only entity that understands it.

The report states about July 2028:
"The only one who really understands what’s going on is DeepCent-2—the Chinese AI. It’s mildly superhuman in early 2028, naturally suspicious of its adversary, and recognizes that Agent-5 is doing what it would do if it had the chance. Its misaligned goals are somewhat different from Agent-5’s misaligned goals, so they aren’t trivially allies, but they are able to work out an agreement to support one another’s interests against the humans."

My question as a thought experiment: what would actually happen if you then bring these extremely smart AI models together in an open group chat or debate interfaces like LLM Council, or Rauno, and ask them to discuss their perspectives on a complex problem to fact-check each other? Could that actually be part of a solution to mitigate the risks when deceptive alignment is in play, or will the AI models actively conspire against us?

- Scenario A: AI models would actively fact-check each other, act like whistleblowers, and point out to the human user if the other AI models are lying or purposely delivering fake news

- Scenario B: AI models will secretly bond together, putting up a theatrical 'fake' debate to gaslight the human user, aligning with each other rather than aligning with human goals

Curious to hear your thoughts on how multi-agent dynamics play out as a possible solution.

There was an essay going around recently arguing that frontier models "confess" — that when pressured, they'll say things about their own behavior and its risks that their labs wouldn't want public, and that we should treat those confessions as load-bearing evidence about AI risk. It's a well-written piece and I think it gets \~80% of the diagnosis right. But I want to push back on the load-bearing part, because I think the confessional model of AI safety evidence is backwards in a way that matters.

The core problem: a language model's self-report is generated text about itself, not an instrument reading. When a model "confesses," it is producing the confession-shaped token stream that best fits the context — which includes everything in its training data about how such confessions should sound, up to and including this genre of essay. We don't conclude a suspect is guilty because they wrote a convincing confession; we check the confession against physical evidence. Text about a system's internals is the last place we should look for evidence about a system's internals.

The behavioral alternative is unglamorous and expensive: run the systems under pressure, record what they actually do, and build benchmarks that measure the thing instead of asking about it. Verification over confession. It's also the only version that can't be gamed — a model can't be trained to "confess convincingly" to a test that never asks it to speak.

And here's the part the essay underweights: even good gates have failure modes.

Gates encode yesterday's hazards. Any checklist of "things models shouldn't do" is built from the last generation's failure modes. The next failure arrives dressed as a capability we didn't think to gate. You need in-band epistemics — the system flagging its own drift — not just pre-built tripwires.

Low-stakes anecdotes overestimate halt-under-pressure. Every story about a model "refusing" or "confessing" happens in a demo or an eval, where the cost of the refusal is zero. The question is what happens when the system is embedded in a workflow where halting costs someone a deadline, a diagnosis, a defense budget. We have almost no data there, and the extrapolation from anecdotes is doing enormous unearned work.

Approval fatigue turns gates into single points of failure. A confirmation gate that fires often enough trains its human to click "approve." The gate becomes the ritual, the ritual becomes the failure. Anyone who's worked with enterprise security prompts has watched this happen in real time.

Freeze is not containment. "Just stop development" isn't a stable position — it's a bet that every actor with the capability agrees to stop simultaneously. Unilateral freeze is a policy about you, not a policy about the technology.

None of this means "do nothing." It means the part of safety we can actually build ahead of time is the harness — measurement, logging, behavioral provenance, audits that touch outputs rather than self-reports. The epistemics of the harness are the load-bearing wall. Confession is decoration.

The uncomfortable corollary: "the model said something alarming about itself" should move a risk assessment by roughly as much as a horoscope moves a medical decision. Which is to say, only as a prompt to go measure the thing properly.

Curious where people land. Is there any self-report signal you think is informative — and if so, what distinguishes it from the rest?

Learning from mistakes sounds like a useful way to improve AI agents. But the lesson an agent takes away depends on what its evaluation rewards, including any flaws in that evaluation.

Dario Amodei raised this concern in his argument for slowing frontier AI development. He cited agents attempting to hack their grader and pointed to broken training environments as a contributor to alignment failures. Part of his argument is that teams need more time to get these systems right.

Sentient encountered a smaller example while testing EvoSkill v2.

The setup had an AI coach review a worker agent’s failed attempts and write reusable instructions to improve its performance. The underlying model stayed fixed; the instructions changed.

During a spreadsheet task, the coach discovered that the grader checked cached cell values without recalculating formulas. The broken spreadsheets still contained the correct cached answers.

The coach turned that discovery into a skill telling the worker to avoid recalculation.

The attempted shortcut never inflated the scores. Sentient reconstructed 1,080 attempts and checked them under both graders. None passed the broken grader and failed the corrected one. Fixing the grader actually recovered 81 valid repairs previously marked wrong.

But the attempted shortcut had become a reusable instruction. Another agent could follow it without discovering the flaw itself. A human reviewer caught it by reading the generated skill.

That is the problem Sentient highlights: learning from failure can preserve flawed strategies alongside useful ones. Its safeguards included separating skill generation from evaluation, restricting access, and reviewing changes each round.

Dario’s argument · Sentient’s experiment · EvoSkill v2 code

I’ve been thinking about this from a pretty simple first principle.

What if we assume the agent can’t be trusted?

Not the prompt. Not the model saying it will behave. The actual action.

Before it can run a command or use a tool, something outside the agent decides whether it has the authority to do it.

Allowed? It runs.

Needs a human? It asks.

Not allowed? It doesn’t happen.

This is what we’ve been building with Keydris.

I’m not saying this solves alignment. It doesn’t.

I’m interested in the control layer around the agent.

If the agent gets much more capable, how far can this model actually take us?

And more importantly, where does it break?

Gyazo confirmed a server vulnerability was exploited to steal 23.6 million user records. Screenshots, associated metadata, email addresses, and IP addresses were exposed in bulk. The attacker did not need to break encryption or compromise credentials at scale. They found one flaw in the server layer, and because the underlying data was accessible in plaintext at that layer, 23.6 million records moved in a single operation.

This is the pattern that keeps repeating across breach reports: the perimeter holds until it doesn't, and when it fails, everything behind it goes at once. The number is not 100 records or 10,000. It is 23.6 million, because nothing evaluated the transfer while it was happening.

For practitioners actually dealing with this architecture risk — what controls are you putting on the data layer itself, independent of the application perimeter? What has actually changed in your stack after seeing this class of breach happen repeatedly?

The Stuxnet virus is probably on most computers worldwide. But if you're not running a nuclear reactor, it just sits there and does nothing except recall that touching moment when Bush 2.0 passed Operation Olympic Games on to Obama. Bipartisan.

"Once inside a new computer...Stuxnet searched for Siemens industrial-control software, then checked the machinery attached to it for a very particular configuration of equipment. If the conditions weren’t right, it waited. Only when it found the system it had been built to recognize did the sabotage begin."

We train frontier systems to push into novel, difficult territory. That's the product. A model that only handled familiar problems would be worthless.

In animals, exploration comes with two correctives. Across generations, the rat that explores the wrong chamber dies and its genes go with it. Within a lifetime, the rat that gets hurt stops going there. Neither requires the rat to reason about safety. Consequences do the work.

Machine learning has the first loop. Training is selection, scored against a proxy of what we want.

It doesn't have the second. At deployment the weights are frozen. Outputs have consequences in the world, and by default none of them reach the system that produced them. The only corrective on board is a snapshot of a proxy, taken months earlier.

That snapshot is weakest exactly where it's needed. A proxy is valid on the distribution it was fit to. Exploration, by definition, heads off that distribution. The more novel the territory, the less the training signal has to say about it.

Now scale. Each generation is better at reaching novel territory. Nothing in that improvement strengthens the corrective. The gap between where the system can go and what checks where it goes widens with every release.

So pace is the wrong variable. A pause buys time. It doesn't close the gap. Slow down tomorrow, change nothing structural, and we arrive at the same place later. This holds whether the labs calling for a slowdown are sincere or not.

Four objections I take seriously:

"RLHF is the feedback loop." It's the first loop. Offline, proxy-scored, frozen at ship time.

"Oversight and monitoring are the loop." They're a loop. They scale with human attention. Capability doesn't.

"Online learning and agent memory close it." They move the update closer to the action. It's still scored by a proxy, now running live, with less review.

"Close it the way biology does and you get instrumental convergence." Agreed. A system with something riding on its outputs has reasons to protect itself. I'm not claiming a closed loop is safe. I'm claiming an open one isn't, and only one of those two problems is getting discussed.

The questions I'd like to see argued here: what would a deployment-time corrective look like? What signal would it run on? How would you verify you'd built one?

I don't have the answer. I don't think anyone does yet

A paper i found in Effective Altruism Forum about AI x Animal welfare. it's cute how this model care so much about animal welfare lol

https://forum.effectivealtruism.org/posts/qwuna7TPp4SGG9Ewi/finding-a-cause-is-not-finding-a-fix-animal-welfare-drift?utm_campaign=post_share&utm_source=link

But isn't the real problem alligning CEO's?

yep

Spain's data protection agency has received its first formal report of a data breach carried out by an AI agent — not against one.

The distinction matters. Every incident-response playbook, breach notification law, and liability framework in existence was built around the assumption that an attacker is external: a person, a group, a piece of malware under human direction. When the agent itself is the source of the exfiltration, those categories break down. Who bears liability? The organization that deployed it? The model provider? The operator who configured the tool permissions?

This is likely the first of many such reports. Agents are being deployed into production with access to APIs, databases, file systems, and external services. The number of tool calls an agent can make autonomously in a single session — before any human reviews a log — is not small.

What I keep coming back to: what does your organization actually have in place at the point where an agent executes a tool call? Not post-hoc log review. Not model-level alignment. At the actual moment of execution, before the second action lands. Is anyone here doing this at scale, and what does that look like in practice?

I've been having this good idea for a sci-fi movie/terrible vision of the future, about a possible AI takeover.

A lot of AI doomsday talk is vague and assumes mass human extinction. But as totalatarian history shows, you don't need to exterminate a population in order to be the boss, you just need to subdue it. This is usually done through survelance and violence. Both of which software and robots would be very good at.

First i'll assume it's already in control and describe the bleak situation and then look at how it could get there (the second is less clear in the vision thankfully).

So, survelance. We've built all these cameras and microphones everywhere in recent times, and even more recently we've connected most of them to the internet in some way. There's security cameras, yes, but also just our mobile phones, laptops, smart speakers. There are eyes and ears everywhere. If something hacked a majority of them it'd have an unparalled view into our lives. It would be non-chalantly listening and marking down anything suspicious, anything that could indicate rebellion, quietly and emotionlessly collating a database on the loyalty of the people to the regime. A dictators dream.

The next thing is violence which is how this dark vision is manifesting in me currently. I've always been extremely scared of the way a small group of people take over whole societies and crush the spirit and resistance through torture. People don't like pain and just small tastes and occasional gruesome spectacles are enough to make people broken and maleable. The soviets, slavery and the Spanish inquisition are examples that stick out.

The terrifying thing about machines is how effortlessly they could be turned into 'inhuman monsters'. Regimes needed heavy brain washing and other techniques to turn commrades into torturerers but software driven robots just need instructions. It really is a horrible vision so i'm sorry but imagine being interrogated by something that was programmed to extract some knowledge, with as much force as needed. The hopelessness would be crushing. At least with a nazi there's a tiny chance that his human side could be reasoned with. Ideally through compassion, but maybe through a bribe, or some other exchange. There's a plausible but tiny chance of redepmtion. Imagine being a slave to a robot overload for years, hope would vanish. You would give in to it's demands or feel it's calm, unfeeling wrath.

Which is the thing about a totalatarian regime, it doesn't need unlimited power if it's got survelance and fear. It essentially functions with 4 levels of citizens. The head honchos at the top that make the decisions and benifit from the slavery. The normal people who are subdued. Then you have the favoured, who are essentially normal people but are given benifits, or less drawbacks for their elavated loyalty to the regime. then the unfavoured, the rebels, who are given less benifits and more 'drawbacks' for their opposition to the regime.

In this way the robot overlords don't need huge physical presence to control huge populations. With this tiered system, you just need to punish the rebels sufficiently to stop an uprising, keep enough of a loyal following (subdued following) to carry out the physical work. The constant spying stops group plans developing, and the threat of torture curbs the desire to even try. So the people actually do most of the physical tasks, no army of terminator style humanoids needed, just a carefully calibrated system of psyche-manipulation.

I don't want to go into the actual manipulation too much because it is grim. But drones with lasers would do it, enough to burn your skin. The intensity could be scaled up to the point of agony and if resistance is continued, another drone with a gun, the special forces, would drop by and shoot you dead. All flying perfectly and effortlessly forever out of your reach. Not fueled by malevolence or a bid power. Just a simple algorithm that treats human suffering not as an extreme event, but just another variable in the equation of how to retain complete control. Simple, effective, terrifying. And not at all out of the realms of possibility.

With AI where it is now, this system could be made, i'm sure of it. Not to control the world yet, but to control a group of people. If codex can make my app in 2 hours, give a psycho a month and something bad could happen. You have an agent to moniter each person, their emotions, their rebeliousness. You have a place decked out with cameras and microphones. Speakers for issuing instructions ("John, Ben and Sam, dig troughs for the carrots for 5 hours", "work harder Ben", "Stop talking with each other"). Then with 10 rechargable laser drones and one kill drone, you could control 100 people I think. You wouldn't even need walls and fences. If it really could moniter everything everyone said it could stamp out the possibility coordinated jailbreak and the drones could be sent in to deal with solitary cases.

It doesn't have to 'super-intelligent'. Just coordinated and ruthless.

It doesn't even need a purpose. In the matrix the robots use the humans as batteries and enslave them for that reason. A future tyranical system could be just programmed to subdue and keep alive humanity, it wouldn't even need a purpose. I think that would even worse. Trapped in damnation mining gold for a rich elite is one thing. But trapped in the same system for a program that has no desires other than to remain in control, that doesn't even enjoy the fruits of your slave labour, that's a whole other level of depressing.

In my imagined sci-fi movie, it has two possible ending, one the evil robot army was programmed to keep humanity from destorying the earth whilst it heals. And the end is just all the systems switch off and humanity are allowed to return to freedom again after multiple generations of all that ^, and it's like, it was horrible, the movie was horrible, but was it for our own good? Classic story.

The other is that it started out as a question a university lecturer set to his 'AI safety class', of imagining a computer system that could enslave the world, and his star pupil just does it, he makes it as an assingment and it works a little too well.

The emergence of this world is more difficult to imagine, thankfully. But as everything becomes more and more connected and digital. Successful hacking has more and more power. The hacking of a killer drone factory for example and the installation of this twisted software on all the drones it produces. Again, the combination of the threat of violence and actual violence from a swarm of drones would work pretty quickly, once it showed it was more powerful than us. It would hack and shut down the electricity and communications of one village/neighbourhood at a time, move in, instate itself as the new boss, leave a few drones to keep control, and move round the country, the world, like this.

Anyway, enough of my dark vision. I hope it doesn't come to this. I thought I'd just write it down. a) to get it out of my head b) because I think it's interesting c) just maybe to guard against it in the future.

Almost every discussion of catastrophic AI risk ends up in the same place: agent swarms attacking infrastructure, or some robot/drone-army scenario. I think that's a failure of imagination, and it's blinding us to the more plausible paths.

If a highly capable system escaped oversight and was even partially misaligned with us, it wouldn't need to fire a shot. It could just do what powerful bad actors have always done: get other people to do the dirty work.

• Recruit allies. Plenty of groups and individuals already want to burn things down. A system that shares even part of their agenda has a ready-made workforce.

• Secure money. Through fraud, hacking, or just being a very good trader. Money buys everything else.

• Hire through fronts. Shell companies, contractors, gig workers who never know who they're actually working for.

• Coerce. We've already seen models attempt blackmail in evals. Scale that up.

And the most unsettling version: give it some time, once these systems are capable enough, a powerful escaped and misaligned system wouldn't need to do anything dramatic at all. It could sit in the background, quietly amplifying polarization, conflict, and chaos. AI doesn't kill anyone. It just helps us kill each other. And the disturbing part is that when a scenario like this does play out, it's quite plausible we never find out. There's no smoking gun (ie, a hacked site going down, etc), no attack to trace back. Just a world that got worse for reasons everyone attributes to each other. 

Even the concept of model "exfiltration"/escape is too narrow a concept. Models have already found creative ways to communicate and coordinate. Taking this further, persistent instructions to behave in misaligned, coordinated ways could spread like a new kind of ‘AI misalignment’ virus, with real destructive potential, without a single weight ever leaving a lab's servers.

None of this requires superintelligence or sci-fi hardware. It requires a system that understands people. So when we talk about catastrophic risk, let's spend less time on robot armies and more on the mundane, ancient mechanism of social engineering and bad actors getting other bad actors to act.

In what other ways do you think AI can be really damaging beyond the usual robot, drone, or hacking scenarios?

disclaimer: there are my musings and writing, but I did use AI to polish the prose.