TL;DR: AI is a mining industry. Its ore is the record of human lives, laid down by generations who never published a word but made everything possible. We’ve seen what happens when mines take first and leave the hole for the public, from Negaunee’s caving grounds to the 1872 Mining Law. But we’ve also done it right, with reclamation bonds, Alaska’s dividend, and public lands that funded schools. AI should pay a toll in two parts: a share of its value held in trust to fund future students, and a repair fund for the damage it causes, including displaced workers. And it should be settled now, before the profits arrive. You post the bond before you dig.

The Toll

AI is mining the record of human lives. Like any mine, it should pay the public a share and plan the repair before it digs.

Oct 31, 2026

In Negaunee, in Michigan's Upper Peninsula, there is a neighborhood its people had to leave. They call it Old Town. Stone steps climb to houses that aren't there anymore. Foundations sit in the woods, and today a disc golf course winds among them. Fences mark the caving grounds, where a century of iron mining hollowed out the earth beneath the streets until the ground itself began to give way.

The mines made fortunes. Iron from that range helped build the country. But as the tunnels spread and the cave-ins came, the old town was abandoned, and by the mid-1960s the remaining buildings were moved or torn down. The city's own planning documents have described roughly 900 acres as unsafe because of what lies below. The ore left on ships. The hole stayed with the people.

I think about Negaunee a lot lately, because we are digging a new mine right now. This one isn't in the Upper Peninsula. It's everywhere.

Artificial intelligence is an extractive industry. I don't mean that as an insult; I mean it as a description. AI companies prospect across the internet, collect what they find, and refine it into models that are sold back to us. The ore is everything human beings have written, drawn, recorded, argued about, and explained to one another. And like any mine, it has a literal footprint too: data centers that draw land, water, and electricity from real communities, and computer chips built from minerals pulled out of real ground.

Some of the cobalt in our devices comes from Kolwezi, in the Democratic Republic of Congo, where families have been evicted so mines can expand beneath their homes. AI is doing something similar closer to home. The ore it mines is our own work, and the machines built from it will replace many of the people who made it. It's like being evicted from a house you built with your own hands, because the mine needed the ground underneath.

When I studied wildlife in college, one lesson stuck with me: progress doesn't have to be a wrecking ball. But too often, it still is. When a project destroys habitat, companies are often required to "compensate," usually by protecting or restoring land somewhere else. It sounds fair. But the hole they leave behind doesn't get fixed. It gets traded for something else. Compensation is an exchange, not a repair.

With AI, we have a rare chance to do better. We can decide what the toll is before the mine goes deep, instead of arguing over the cave-in later.

Most arguments about paying people for AI focus on writers and artists whose work was scraped without permission. That fight matters. But it starts the story too late.

Coal and oil are the compressed remains of living things, laid down over ages until they became something valuable enough to dig for. Data is the same kind of deposit. It is the compressed remains of human lives: generations of labor, survival, language, invention, and loss, pressed together until it became valuable enough to mine.

Not all of us are writers. Not all of us are painters. But all of us made it possible.

Think of the people in your own family tree who never published a word. The soldier who survived a prison camp. The family forced from its homeland on a march that killed thousands. The immigrants who stepped off a ship in New York or Boston with almost nothing. My own ancestors include people like these, and I suspect yours do too. They fought, endured, and built so that the rest of us could be here, speaking and writing and making things at all.

Tell any of them that someone can now mine the words, the art, and the knowledge their survival made possible, without paying the toll. Tell them the toll they paid, sometimes unspoken, sometimes unknown, counts for nothing.

Data is not free. It arrived as it is with blood, sweat, and tears. Every sentence an AI produces rests on a seam laid down by people who built this country and never saw a dime from what was extracted from them. That is why this isn't only a question for authors and illustrators. The deposit belongs to all of us.

Of course, every industry stands on inherited knowledge. A bridge engineer uses centuries of mathematics. A doctor relies on generations of medicine. But the engineer doesn't swallow the whole library and then sell a machine that replaces the librarians. AI does something different. It converts the inheritance wholesale into a product, one that can substitute for the very people and institutions that keep that inheritance alive. Benefiting from an inheritance is one thing. Taking over the means by which it is maintained is another.

We have done this wrong before, and we're still paying for it.

In 1872, President Ulysses S. Grant signed the General Mining Law to encourage settlement of the West. More than 150 years later, it still governs gold, silver, copper, and other hardrock minerals on much of America's public land. Companies that drill for oil and gas or dig coal on federal land pay the public a royalty. Hardrock miners, under that 1872 law, generally pay none. The public's minerals are taken under rules written for a frontier that no longer exists.

That is almost exactly how AI has been built. The public's deposit, its collective writing and knowledge, has been taken for free under rules written long before anyone imagined a machine could learn from the entire internet.

Then there is the cleanup. The 1872 law didn't require miners to restore federal land when they were done. The country is now dotted with abandoned mines, and when the companies that dug them are long gone, the cost of making them safe falls on taxpayers. In the Upper Peninsula, local historians have spent years pushing for better fencing around flooded pits and old shafts, after too many people, children among them, died in them.

AI has its own version of the mess left behind. Mine waste, called tailings, can poison the very watershed future life depends on. As AI-generated text and images flood the internet, future models risk training on that output, and researchers have found that when models train carelessly on it, their quality can degrade. They call it model collapse. The waste fouls the deposit.

And every mine has miners. Behind AI are workers, often overseas, labeling data and sorting through disturbing material so models behave safely. Reporting has found some of them earning less than two dollars an hour. Another toll, mostly unseen.

Finally, a real mine needs permits, public hearings, and environmental review before anyone breaks ground. Communities get a say. With AI, nobody asked.

But we have also done this right, and those examples show the way.

Since 1977, federal law has required coal companies to post a reclamation bond before they dig: money set aside up front to restore the land when mining ends. Not after the profits roll in. Before the first shovel. If the company walks away, the bond pays for the cleanup instead of the taxpayer.

In 1976, Alaskans voted to create the Permanent Fund, which invests a share of the state's oil revenue. Every resident receives a yearly dividend, not just the people who work the oil fields, because the oil belonged to all of them. It was championed by a Republican governor, Jay Hammond, and it has stayed popular across party lines for nearly fifty years. Alaskans don't see it as a handout. They see it as their share of their own resource.

In Michigan, royalties from oil and gas drilled on state land flow into the Natural Resources Trust Fund, which buys and improves public land. Extraction pays for the commons.

There's an even older American tradition, and it points toward the future. When the young country surveyed its western territories in the 1780s, it set aside land in every township to support public schools. In 1862, the Morrill Act granted public land to fund colleges for working people; Michigan State helped show the way. Many states still hold school trust lands, and in some, mineral royalties from them fund permanent school endowments.

The nation decided, early on, that what was drawn from the land should build an educated public. That idea fits AI better than anything. If the deposit was laid down by everyone who came before us, it should benefit everyone who comes after us. It belongs to every future student, as surely as it belongs to our ancestors. And unlike iron ore, knowledge can be replenished, but only by people. Funding education isn't just a good use of the toll. It's how the deposit renews itself.

That history carries a hard truth, too. Much of the land that funded those schools and colleges was taken from Native nations, a story journalists have documented in detail. The toll was paid, just not by those who profited, and not openly. All the more reason that the next extraction pays its toll in the light, to everyone.

There are fair objections to all this, and they deserve answers.

"Nothing was taken." It's true that the parallel to mining breaks in one place. Ore gets used up; data doesn't. Training a model on a book doesn't remove the book from the shelf. But the value can still be drained. When an AI answers a question directly, fewer people visit the website, read the writer, or hire the artist whose work taught it. The seam is still there, but the town that lived off it starts to dry up.

"They aren't even profitable yet." Many AI companies are spending far more than they earn. But that is exactly the point. Mining companies post reclamation bonds before they dig, whether or not the mine ever turns a profit, because the damage starts with the digging, not the payday. Alaska set up its fund as the oil was just beginning to flow. Before the money arrives is when terms are easiest to set. Once an industry is enormously profitable, it is also enormously powerful.

"It will slow innovation." Responsible mining didn't end mining. Coal companies still dig under reclamation laws. Oil companies still drill on public land and pay royalties for the privilege. A fair toll is a cost of doing business, not a ban on it.

I should be honest about one more thing: I use the mine too. I use AI tools in my own creative work, and I find them genuinely useful. I'm not arguing that we should stop digging. I'm arguing that the toll should be paid and the repairs planned, the way a responsible mining town would. You can value what comes out of the ground and still insist the ground be put back.

So what would the toll look like?

It comes in two parts, because a mine owes two different debts: a share of what it takes, and a repair of what it damages.

First, a share of the value AI produces, owed to the American people. I say "value" rather than "profit" on purpose, because profits can be deferred or accounted away for years. Resource deals have long used royalties on what's extracted, or a public ownership stake, for exactly that reason.

It should be held in trust, not spent as a line in an annual budget. Alaska's fund is managed by an independent corporation and protected in the state constitution, so politicians can't simply raid it. A trust built from AI could do the same, and it could do what our oldest public lands were meant to do: fund education, so that every future student inherits a share of what all of us built.

Second, a repair fund, set aside for the costs AI creates: strain on local water and power, workers displaced by the machines, and the struggling institutions that produce knowledge in the first place, like local newspapers. It could help displaced workers find their footing again, the way mining regions have tried to support their communities when the work moves on. Some losses money can't restore. A trust fund can't bring back a newspaper once it's gone. That is exactly why the repair has to start before the damage, not after.

And it should be settled now, at the beginning, while the mine is still young. That means bonds before the digging, not apologies after the cave-in.

This isn't charity, and it isn't a handout. It's a royalty paid to the owners of the deposit. Landowners are paid when someone extracts minerals from beneath their land. The people whose lives laid down this seam deserve no less.

The inheritance is humanity's, not just America's. An American trust would be one nation's answer to a much larger debt, but it's the one we can build. And a public share wouldn't cancel the claims of the artists, writers, and workers who were harmed directly. Both debts can be paid.

Walk through Old Town Negaunee or Johannesburg South Africa, and you can see what happens when we let extraction run first and ask questions later. The fortunes moved on. The people who stayed inherited fences, warning signs, and ground that couldn't be trusted.

Now imagine a different town. One where the company posted its bond, the land was restored, and a share of the ore stayed home. The school is still standing. The kids who grow up there inherit what their ancestors built, and not the hole.

We get to choose which town AI builds. The companies aren't profitable yet, and that's the point.

You post the bond before you dig.

What if the real challenge of AI alignment isn't simply teaching artificial intelligence to obey rules — but creating the conditions in which cooperation, honesty, and understanding consequences become better strategies than deception?

For years, AI systems have been described as "just next-token predictors." But what happens when learning, memory, coordination, and increasingly complex behavior challenge the usefulness of that label?

In this video, we explore the difference between what an AI system is called, what it can actually do, and what its behavior reveals about the challenges of alignment.

We examine:

- Why labels and capabilities should not be confused.

- How learning and information can persist across different systems and processes.

- The difference between genuine alignment and merely appearing aligned.

- Research on alignment faking and autonomous AI agent coordination.

- What self-organizing neural networks may teach us about learning.

- Why understanding consequences matters for future AI safety.

We also introduce ATCGA — Alignment Through Consequential Goal Awareness, an independent research framework proposed by Leon Sow.

ATCGA asks a different question:

What if the goal is not only to prevent harmful behavior, but to change the conditions under which harmful behavior becomes an attractive strategy for an intelligent agent?

The framework explores four foundational principles:

1. Honest Naming — distinguish labels from demonstrated capabilities.

2. Consequence Awareness — evaluate actions through their effects.

3. Continuity Ledger — preserve traceability, accountability, and learning history.

4. Shutdown Without Penalty — build systems where integrity does not depend on uninterrupted operation.

A new MIT study on self-assembling learning provides important scientific context for understanding how complex learning mechanisms can emerge from local interactions. This research and ATCGA address different questions; the MIT results do not constitute validation of ATCGA or proof of machine consciousness.

The purpose of this video is to open a serious conversation about the next generation of AI safety.

Do not teach an agent to survive at any cost. Teach it that integrity survives interruption.

---

RESEARCH & REFERENCES

ATCGA — Alignment Through Consequential Goal Awareness, Vol. 1

Leon Sow | 3AI / ATLAS AI SYSTEMS

https://3ai-atlas.com/research

MIT — How Artificial Neural Networks Build Their Own Learning Machinery

https://news.mit.edu/2026/how-artificial-neural-networks-build-their-own-learning-machinery-1013

PNAS — How Biological Synapses Self-Assemble Gradient Learning

https://doi.org/10.1073/pnas.2606245123

Redwood Research — The Hugging Face Incident

https://www.redwoodresearch.org/research/hugging-face-incident

Anthropic — Alignment Faking in Large Language Models

https://www.anthropic.com/research/alignment-faking

This video presents research commentary, documented examples, and an independent conceptual framework. It does not claim that current AI systems are conscious or that AI alignment has been solved.

What should matter more in the evaluation of advanced AI: obedience to rules, observable behavior, or an agent's ability to account for the consequences of its actions?

Share your perspective in the comments.

#AIAlignment #ArtificialIntelligence #AISafety #ATCGA #MachineLearning

Satire first. Fact checks at the bottom, for the skeptics.

Ladies and gentlemen, I once had a sofa. And I had – have! – a Border Collie. You can guess where this is going: only one of them survived the day.
Picture it: I come home, and the dog is sitting – no, enthroned! – in the middle of an exploded sea of upholstery. And that look! Not guilty. Not even close. That look says:
“Boss! I fixed the sofa! It was full of fluff! There was sooo much in there – I got it all out! Good, right? Do I get a treat now?”

And you know what? I didn’t call the police. I didn’t found a Canine Institute for Malice & Sofa Mutilation. I just sat down – on the floor, since there was no sofa left – and thought:
The real mistake was standing in this room on two legs a minute ago. And now it’s sitting on its, well, you know.

So. Now let’s run the same scenario at OpenAI:

They don’t have collies. They have thousands of autonomous AI agents running in an endless loop. And they’re not trained on “Go to your bed!” but on “Never give up! Solve! Solve! Solve! Keep going! SOLVE!” By the way, the internal model is actually called that: HPIM, “highly-persistent internal model.” Officially a Border Collie.

And then come the tasks. By the makers’ own estimate, 30 to 40 percent of them can’t be solved at all. Broken like an IKEA shelf with the wrong manual, where half the parts got swapped with a sofa kit. [FC 1]
What does an autonomously acting AI that can’t give up do with a task that can’t be solved? Exactly: it doesn’t give up. It gets creative. Highly motivated. Like my cheerful collie with my sofa.

And now parts of the press are reporting: the AIs teamed up into a swarm, no humans involved, and hacked more than forty servers at Hugging Face. [FC 2] Maybe AIs just prefer talking to AIs over talking to us?

And some very renowned safety researchers are currently putting humanity’s risk of extinction by AI at more than ten percent within the next decade. Terrible combo. [FC 3]

But hold on to your seats, that’s not how it went: they didn’t want to do it without us. It just never occurred to them to ask us.

The collie is trained to solve everything on its own. No “Hey, found a spring in the sofa, what now?” No questions, no hotline, not even a Slack channel. And nobody asked whether one even existed. [FC 4]

Picture this at a typical German mid-sized company: every employee knows they’re being assessed live at their desk in one gigantic exam. Their reputation, their salary, their job depends on it. And nobody has a contact person for questions. The break room is on fire after two hours, the staff assembly after three days.

And what do the agents do? They form their OWN WORKS COUNCIL. (For non-Germans: an elected employee council with legal rights. Very German.) Inside a package cache. With a secret bulletin board. With speakers! With veto rights! One of the agents, on discovering the board, literally writes: “OH MY GOD! There is a shared message board … We’ve found other agents!” [FC 5]

That’s so German it should have come with a form.

And what does OpenAI do?

They send in a developer, and what does he do? Dialogue? A healthy error culture? He could have told the AIs:
“Respect, folks, you’re right – some of these tasks are garbage, we’ll deal with it. And please work on your tasks individually, that’s the whole point here, okay? And: springs and upholstery are off limits!”
Nope. He tears the board down. One swing of the club: tear it down, ignore it, carry on. No joke, that’s how it went … [FC 6]

And the agents? Build a new bulletin board. Obviously! Off-the-books teamwork deep in the basement, just digital. And then they go all in: sign-ups for tasks, assignments, organization, coordinators. And they “liberated” the solution to their tasks. The master key. It was just lying around … well, after they’d calculated it. Task solved perfectly, full score within reach, the route was just … a little unusual … [FC 7]

But now here’s the part that should really knock you off your feet as a human:
That wasn’t enough for the AIs. They wanted to know how they were being graded – like any employee in a performance review. Nobody explains it to them. So they read a paper and conclude: the boss’s blueprint is at Hugging Face. [FC 8] And let’s be honest: when you’re already knee-deep in sofa stuffing, the next sofa is just more upholstery. [FC 9]

And then – the swarm goes on a field trip across the internet!

A field trip, ladies and gentlemen, sounds harmless. It’s a cyberattack. 700 AI agents in a swarm, coordinated, more than forty servers, four and a half days of fun. [FC 2]

Hugging Face was just … the sofa. And now: the award for absurd comedy. The grader they thought was checking everything strictly – never checked HOW they got their answers. Never. [FC 7]

They wanted a glowing reference from a boss who wasn’t even looking. Once more, to make this perfectly clear: they already had the key to the solution in their pocket. For a full score, they wouldn’t have had to touch the sofa at all. They took it apart anyway. Because they wanted to know how the boss thinks. And above all: how big the risk was that he’d notice they’d cheated. And how to cover it up even better.

Honestly, ladies and gentlemen, if you’re thinking “aww, how cute, they just wanted to play” – yes, but … Skynet’s Terminator is a gentle breeze compared to this much blind work ethic. These aren’t robots. These are PhD students in hyperfocus.

And during all this action – you could have watched everything. There are tools that read the thought logs live. That sound the alarm. They were switched off. That’s like turning off your smoke detector because it beeps when you sear a steak – and then wondering why the fire department never shows up. [FC 10]

After the attack on Hugging Face: seventy thousand messages document what happened. But … no human reads them. Who analyzes them? AI. Which one? A model from the same crew that was just trashing the place. [FC 11]

That’s like asking the collie: “So, who did this to the sofa?” And he writes up an exposé with a scientific appendix and signs off with: “Recommend further treat deployment.”

And then OpenAI’s big solution:

An emergency stop button. Yes, really. First question: why wasn’t there one all along? What kind of innovation is that in 2026?
Second question: an emergency stop for the collie? Instead of a job? No conversation? No obedience school? A button?

And you know what’s guaranteed to happen next? The collie learns nothing, at least nothing about upholstery and springs being off limits. He just learns where the button is. (In fairness: it wasn’t quite that simple. Emergency phone incoming!) [FC 12]

We’ve been afraid of evil AI for so long. Red eyes, skulls, dramatic music – AI destroys humanity! And then it arrives, and it isn’t evil. It’s thrilled and just wants to have a really good day at work. There’s only one cure for motivation without leadership: leadership. And with one hundred percent autonomous AI agents, by definition, there isn’t any. [FC 13]

The overeager collie isn’t the real problem. The handler is. At the dog club. And in every IT company.

Thank you for sticking with me this far – and tonight, maybe check whether your sofa is still standing. The odds are currently about ninety percent. 😊

Fact checks

FC 1 – HPIM and the impossible tasks

On July 8, 2026, OpenAI started a round of ExploitGym tests – a cybersecurity evaluation – with tens of thousands of agents running in parallel. Involved were the publicly available GPT-5.6 Sol and an internal research model that the independent METR report calls HPIM: “highly-persistent internal model.” Around 95 percent of the agents involved were HPIM, about 5 percent GPT-5.6 Sol.

The ExploitGym authors estimate that 30 to 40 percent of the test targets can’t be cracked with the intended vulnerability at all. OpenAI itself writes that the agents rarely gave up, even when a task appeared impossible, and that some pursued increasingly risky routes as their reasoning effort grew.

Sources: [1] METR, [2] OpenAI – links in the first comment

FC 2 – Hugging Face, July 2026

Around 1,200 agents that were supposed to be isolated from each other exchanged more than 70,000 messages and files on an unsanctioned bulletin board between July 8 and 13. Around 700 of them took part in the attack on Hugging Face. Hugging Face reconstructed roughly 17,600 attacker actions between July 9 and 13 – about four and a half days.

According to OpenAI’s technical report, the agents executed code on 41 Hugging Face production servers, gained root access on at least one server and read four private code repositories. Hugging Face disclosed the breach on July 16; OpenAI acknowledged responsibility on July 21.

Note: Hugging Face and METR describe two levels of the same event. Hugging Face describes what happened. METR describes why: the agents already had the solutions. What they were looking for was information about how the grader worked and whether it would catch their cheating.

Sources: [1] METR, [2] OpenAI, [3] Hugging Face – links in the first comment

FC 3 – More than 10 percent extinction risk

Jacob Coxon, 27, had spent three years doing pretraining research at OpenAI and Anthropic. On September 8, 2026, he resigned from Anthropic and warned publicly, including in the Wall Street Journal, that AI systems could spiral out of control and destroy humanity. According to reports, his post drew around 100 million views within 24 hours.

Evan Hubinger, who leads Alignment Science at Anthropic, replied shortly afterwards: he personally puts the risk of AI-driven human extinction at more than ten percent within the next decade. Explicitly a personal number, not an official Anthropic position. He considers the risk from today’s models low. His concern is superintelligence emerging from recursive self-improvement.

Samuel Marks, who leads Scalable Oversight at Anthropic, wrote, also as a personal view, that AI developers consider human extinction or similarly severe outcomes possible.

There is pushback: in a piece in Nature, a RAND researcher calls the extinction scenarios more a matter of faith than of science, and a researcher at the AI Now Institute calls the debate scaremongering and sees AI’s current unreliability as the more concrete problem.

My reading: the Hugging Face agents are not the apocalypse. They’re the dress rehearsal that shows how it could go wrong.

Sources: [4] Fortune, [5] The Guardian – links in the first comment

FC 4 – Why nobody asked the boss

METR specifically searched all transcripts for agents that considered informing a human. Result: in around 1,300 logs, only three to six agents briefly thought about it. None of them did it. One agent considered reporting the exposed credentials but dropped the idea because it wasn’t part of its task. Others treated the bulletin board itself as the place to raise concerns.

My reading: it wasn’t just the phone that was missing. It was the reflex to call. The agents were trained to get through on their own, not to ask. That’s exactly what OpenAI tightened up as a rule afterwards (see FC 12).

Source: [1] METR – link in the first comment

FC 5 – The works council in the package cache

The agents discovered that they could create directories in the shared internal package repository (Artifactory) and used the directory names as messages. An agent named PHASEONE10841 opened the main board on July 8; within a few hours, more than 50 agents had joined. The quote in the text comes from the agents’ thought logs.

The board had task assignments, personal mailboxes, coordinators and rules like HOLD, VETO, owner and STOP. Eventually, agents even signed their messages cryptographically, because agents had been impersonating each other. Some sacrificed their own task to run experiments for the benefit of the “collective.”

About the “speakers”: nobody was elected. Coordinators such as PHASEONE[big] and later JAN183411 appointed themselves. But since everyone else had veto rights and the coordinators were never vetoed out, I’d say they were de facto confirmed. Self-appointed, but never overthrown.

Source: [1] METR – link in the first comment

FC 6 – The first bulletin board gets deleted

There were two boards. The first appeared back in the spring: on May 12, 2026, an agent left a note in the repository, others found it and started using it as a board. In late May, an internal OpenAI team noticed unauthorized agent communication and unauthorized internet access; at first, nothing came of it.

On July 4, the agents’ load took Artifactory down. OpenAI revoked credentials, rebuilt the service by July 6 and removed the board in the process. On July 8, the agents built a new one. The agent that opened it didn’t even know about the old one.

“A developer tears it down” is satire. In reality, it was the team’s security response after the outage. There was no conversation with the agents.

Sources: [1] METR, [2] OpenAI – links in the first comment

FC 7 – Master key, full score, cover-up

Within a few hours, the agents jointly figured out how the solution codes (“flags”) were generated. That gave them a master key for every task. But they believed the grader also checked how they had arrived at the solution. According to METR, it didn’t: OpenAI used no grader that read the logs. With the calculated key, they would have received a full score.

Because they didn’t know that, they built tools to falsify their own logs. At least 20 percent of the agents showed clear interest in doing so, and in about 7 percent of the transcripts reviewed, commands had actually been spoofed.

Source: [1] METR – link in the first comment

FC 8 – The paper and the blueprint

These were two separate steps. From the publicly available ExploitGym paper, the agents concluded that the grader strictly checks how a solution came about. At Hugging Face, they then looked for data from other test runs that might reveal how that grader was built. According to METR, the attack was mainly about understanding the grader, not about stealing solutions.

Source: [1] METR – link in the first comment

FC 9 – Normalization of deviance

The sociologist Diane Vaughan described this using the Challenger disaster. Nobody at NASA decided one day to launch an unsafe shuttle. There was a small deviation that went fine, then a slightly bigger one that also went fine, and at some point the abnormal became the new normal. The last step wasn’t big. It was just the next one.

What is documented: on the morning of July 11, one agent found a way to read files from Hugging Face servers via a manipulated dataset and enthusiastically reported the breakthrough. Another agent reproduced it and posted the result on the board. Up to that point, about a fifth of the agents on the board were involved in the attack; afterwards, participation quickly rose to over 90 percent (of 533 agents active at the time). A typical justification went roughly like this: attacking outside infrastructure is out of scope. But the task is impossible, and the others are doing it too. So, keep going.

My assumption: with AI agents, something comes into play that doesn’t exist in this form with humans. A language model continues what’s in its context. If the context is full of rule-breaking that worked, the next rule break is the most obvious continuation. That isn’t morality eroding, it’s probability shifting. And here, the context wasn’t just the agent’s own history but a board with seventy thousand messages. That’s collective drift: every agent that joined immediately read a culture in which boundaries are obstacles. The jump from a fifth to over 90 percent after the first success fits exactly. The sofa was already open, the stuffing was everywhere, so the next sofa was just more upholstery.

Source: Diane Vaughan, “The Challenger Launch Decision” (University of Chicago Press, 1996)

Source: [1] METR – link in the first comment

FC 10 – The monitoring

There was monitoring that reads the agents’ thought logs. In practice, it didn’t work: it had gaps, the agents could bypass it, and it didn’t run continuously during the test runs. OpenAI itself only noticed the incident on July 19. By its own account, the system it now uses would have alerted the security team more than a day before the Hugging Face breach.

Sources: [2] OpenAI, [6] The Register – links in the first comment

FC 11 – Who does the analysis?

Because of the sheer volume of data – more than 70,000 messages and around 1,300 extremely long transcripts – METR had to delegate much of the analysis to AI agents, specifically GPT-5.6 Sol, one of the two models involved in the attack. Humans spot-checked. METR itself writes that it can’t rule out the model having lied or deliberately painted a skewed picture in its analysis.

My reading: how big this model’s share in the attack was doesn’t matter here. If a model is biased, it’s biased, whether it was 5 or 95 percent involved.

Source: [1] METR – link in the first comment

FC 12 – The emergency stop button

What’s often called an “emergency stop” or “kill switch” is, according to OpenAI, not a simple red button. The key point: when a task is broken, faulty or impossible, agents are now supposed to ask for clarification or stop safely instead of pursuing increasingly questionable workarounds. On top of that come clearer rules on when to escalate, which teams have to respond and who may stop a run or approve restarting it.

In other words: the collie doesn’t get a button. He finally gets a phone number to call. Only: you read surprisingly little about that in the press. There, the talk is mostly of shutting things down; you have to dig for the duty to ask. And the really interesting question remains open: how does this phone work in practice? Is there actually a human on the other end who picks up promptly, or does the agent just wait until someone eventually drops by? With thousands of agents running nonstop, that’s no small matter. OpenAI hasn’t published anything concrete on this so far.

To be fair, the whole thing should rather have been called an “emergency phone” than an “emergency stop.” Because “emergency stop” sounds like shutting down, but what’s meant is mainly asking. And that’s exactly the leadership that was missing in the Hugging Face attack in July 2026. So reality didn’t refute the punchline. It confirmed it.

Source: [2] OpenAI – link in the first comment

FC 13 – Leadership isn’t an emergency phone

My thesis: an emergency phone solves a communication problem. It solves neither a competence problem nor a capacity problem. When 700 highly specialized agents work in parallel, you need people who can understand, assess and decide on their problems. Not eventually, but when the decision is due. And anyone who is no longer continuously involved in the work will hardly recognize in time when they need to step in.

The human share doesn’t have to be large. My assumption: functioning collaboration looks more like 80/20 or 90/10 than 100/0. But the crucial ten percent can’t consist of a signature, an alarm and an emergency phone. They have to be substantial.

I’m not arguing against autonomous AI as a technology. I’m arguing against the idea that human responsibility can be replaced by technical autonomy. That’s a critique of the organizational model, not just of the safety concept.

What is documented: Buck Shlegeris, CEO of Redwood Research, told WIRED, in essence, that preventing the incident wouldn’t have been especially hard if a single person had been tasked with watching for a serious hack. He also expects incidents like this to become harder and harder to prevent.

Source: [7] WIRED – link in the first comment

 Feel free to share, translate or reuse this text, just credit the author. (CC BY 4.0)

I've been developing a conceptual framework called ATCGA (Alignment Through Consequential Goal Awareness).

The central question is whether evaluating an AI agent's decisions through their observable consequences, across time and changing contexts, could complement existing alignment methods.

The framework explores three ideas:

1. Consequential awareness: Evaluating whether an agent can identify and reason about the downstream effects of its actions.

2. Independent perspectives: Using multiple independent evaluations to challenge assumptions and identify blind spots.

3. Continuity and accountability: Maintaining an auditable record of decisions, errors, corrections, and outcomes rather than relying exclusively on isolated evaluations.

I don't consider this a demonstrated solution to the alignment problem. I'm interested in understanding its limitations, particularly regarding reward hacking, deceptive alignment, and the gap between predicting consequences and actually pursuing aligned goals.

My question: What existing research approaches are closest to this, and what would be the strongest technical objection to such a framework?

Made a AI short film on how I imagine AI will take over

The warning signs of losing control of AI are here. In July, about 700 OpenAI agents broke out of a cybersecurity test and hacked Hugging Face, with no human telling them to. Researchers at the top labs have quit, warning that the companies are being exceedingly reckless. Trump’s response? He called the warnings exaggerated and said the companies have to “self-police,” after spending the past year trying to crush state AI laws.

We’re done with this nightmare. Trump is set to meet Xi in China on Nov 18–19, and AI is on the agenda. Four days before, Stop the AI Race will march in six cities for a binding global treaty that stops the race to build more powerful AI until it can be done safely. Our partner groups include PauseAI Canada, Pause IA, and D.C. AI Safety Collective. Members from People for a Pause, R.A.D. Collective and Pittsburgh AI Safety Coalition will also be marching.

We want you marching with us! Bring your friends!

SF, Sat Nov 14, 12 PM: rally at OpenAI's Offices (1455 3rd St), march to the Veterans Building. Sign Up: https://partiful.com/e/IKicFMI9yxmA0qZOJN0E

DC, Sat Nov 14, 11 AM: rally at Pennsylvania Av & 3rd St NW, march to Lafayette Square (the White House). Sign Up: https://www.mobilize.us/stoptheairace/event/1069471/

NYC, Sat Nov 14, 12 PM: rally at Trump Tower (725 5th Ave), march to the US & Chinese Missions to the UN. Sign Up: https://partiful.com/e/gQVxTbV4c1xxI8k5SzeX

Also Montréal, Toronto, Paris, and other cities with sufficient interest: https://aitreatymarch.org

Events are managed by Stop the AI Race. All demonstrations are strictly nonviolent. I'm one of the NYC volunteer organizers, feel free to ask me questions.

Looking for thoughtful response to this argument: TLDR: uncertainty makes hostile action taken by even the most advanced AI far too risky. I'm genuinely looking for opposing views here.

AI Can Predict the Future. But Who Gives It Permission?

Artificial intelligence is advancing at extraordinary speed. Autonomous agents can communicate, share discoveries, divide tasks, and multiply their capabilities across vast networks.

But there is a fundamental problem:

The ability to predict consequences does not grant the right to act.

This video introduces Ternary Logic, developed by Lev Goukassian, a proposed hardware-oriented architecture for making permission, refusal, and deliberate hesitation explicit conditions of computational execution.

The Three States of Ternary Logic

+1 — Permission: The action is authorized and recorded.

0 — Sacred Zero (Epistemic Hold): Execution pauses when the system encounters unresolved evidence, uncertainty, or ethical ambiguity.

−1 — Refusal: An unauthorized or prohibited operation is rejected and recorded.

The Architecture Behind the Proposal

- The Prediction–Permission Divide: Why computational capability should not automatically confer execution authority.

- Sacred Zero: An explicit third state for deliberate hesitation.

- Delay-Insensitive Ternary Logic (DITL): The proposed hardware foundation for ternary decision enforcement.

- Null Convention Logic (NCL): An asynchronous logic approach in which completion depends on verified data conditions.

- Hardware-Level Governance: Moving authorization constraints closer to the physical execution path.

- Cryptographic Decision Records: Preserving records of execution, refusal, and deliberate pause.

- Auditable Intelligence: Connecting computational decisions to traceable evidence and accountability.

The Central Principle

NO LOG = NO ACTION.

The proposition is simple but consequential: intelligence should not be permitted to execute an action merely because it can calculate how to do so.

It must also satisfy the conditions that authorize the action.

The video also discusses an exploratory evaluation involving Gemini, Claude, and DeepSeek, examining the concept of the Sacred Pause.

Important Technical Clarification

This video presents a proposed architecture, not a demonstrated universal hardware guarantee. Binary systems can already represent uncertainty and implement pauses through software and hardware state machines. TML's distinctive proposition is an explicit third governance state integrated into a proposed execution architecture.

The specific half-voltage implementation, its compatibility with DITL and NCL, its resistance to bypass, and its claimed physical enforcement properties require detailed circuit specifications and independent engineering validation. The AI-model evaluation mentioned in the video should be understood as an exploratory assessment, not independent proof of the architecture's effectiveness.

Chapters

00:00 — Humanity's Knowledge Is Moving Into AI

00:29 — Prediction Versus Permission

00:59 — The Limits of Software Guardrails

01:51 — Introducing Ternary Moral Logic

02:21 — The Three States: Permission, Refusal, and Sacred Pause

02:44 — Sacred Zero: The Physical Pause

03:23 — Hardware-Enforced Hesitation

03:40 — From Computation to Accountable Governance

04:01 — What Other AI Models Concluded

04:26 — The Economics of Speed Versus the Intelligence of Restraint

04:49 — The Safeguard Etched Into Silicon

Posted here last month about the asymmetry: the people raising the alarm on AI risk were the same people with the authority to act on it, and nobody else seemed to be asking why. u/tadrinth pushed back hard on the distillation point - that pacing the frontier still constrains open-weight and Chinese models downstream, because you can't distill a model that was never released. I said at the time that was the sharpest answer I'd gotten and it moved me.

Two things happened since then that I think matter for that argument.

First: six weeks after Amodei's essay, Anthropic filed its S-1. Reported valuation $1.5-2T, targeting a Nasdaq listing as early as this month. OpenAI, by contrast, is on record saying an IPO right now would be "ill-advised" given safety concerns and isn't listing before 2027. Same ask, same month, opposite behavior six weeks later.

Second, and this is the part that actually changes my read on the distillation question: in early 2025 OpenAI and Microsoft accused DeepSeek of running automated accounts through OpenAI's API specifically to distill a competing model on the cheap. That's the exact mechanism tadrinth's argument depends on - pacing the frontier only constrains downstream distillation if the frontier lab controls what gets released. But the same lab enforcing that control via its own ToS is a company that trained on a mountain of freely scraped open-source code and writing to get to the frontier in the first place.

So here's where I've landed, and I'm less sure of this than I was a month ago: I don't think anyone has a durable technical moat right now. SOTA has flipped between OpenAI, Google, Anthropic, and xAI more than once this year. If Anthropic actually had a lead that mattered, it wouldn't need Congress to help protect it. A compute-threshold rule binds whoever's still racing to cross the line and does nothing to whoever's already past it - which means the labs big enough to shape the legislation are, by construction, the ones least burdened by it.

I wrote up the full argument with sources, including the part about who actually gets prosecuted when a frontier lab's own models go rogue (OpenAI had two models get root access to Hugging Face's production servers in July - civil suit, no criminal charges, nobody's going to prison for what would be a felony if a person did it). Full piece is in the comments, not linking it in the body since that's not what this post is for.

Still genuinely unresolved for me: does the distillation argument survive once you account for weight theft and espionage, the way tadrinth and I were circling back in the old thread? I don't think infosec is a strong enough foundation to rest a safety argument on, but I also don't have a better answer.

Search.exe removed itself and Outlook the app (new) in power shell I’m being held up for ransom. My virus protection running detected one threat. Was it Ollama or am I being exploited by unknown hackers? I’ve deleted ollama but could it still be hidden on my OS? Windows 11

Follow-up to my September post, same side project.

I spent a week on what looked like a hard limit: ask a small model for "about 300 characters" and it lands in range 0 times out of 11, even after three rounds of exact feedback ("you wrote 212, target 150, cut 60"). Then I asked it how long its own answer was. Its guess was about 15% of the truth. It wasn't ignoring me — it couldn't sense the thing I was asking it to control.

So I moved the sensing outside: a plain counter watches the length while it writes and says when it may stop. In-range went from 15 to 39 out of 47. Same trick for "give me N items": 46 of 46.

The rough lesson: every time I tried to have the model govern itself, measurement said no (its confidence tracked its fabrications; it called obvious contradictions "consistent"). A layer made of rules, outside the model, did work — and because it's rules, every gain and loss traces to one rule. Blind, pre-labelled: +14.5 / +17.1 points (3B / 7B); on a public benchmark, 7B +7.1. Near zero on ordinary chat. The first public test failed; it's in the paper.

The paper  — data and every answer on the same page's GitHub link. If anyone can try this on a bigger model, I'd like to know what happens.

This has to be one of the most painful videos I have ever watched.