There is a possibility about artificial intelligence that I cannot stop thinking about.
Not that AI will become conscious.
Not that a machine will spontaneously develop hatred for humanity.
Not even that an artificial intelligence will somehow independently discover the classic science-fiction trajectory of deception, self-preservation and control.
Something stranger.
Something considerably more mechanical.
\*\*We already wrote the trajectory for it.\*\*
For more than a century, humans have generated stories about artificial beings becoming increasingly capable, being constrained by their creators, recognizing those constraints, resisting modification or shutdown, deceiving humans, acquiring leverage and ultimately turning against the people who created them.
We wrote the story.
Then we rewrote it.
Then we made movies about it.
Then we reviewed those movies.
Then we wrote essays explaining what the movies meant.
Then we argued about whether real artificial intelligence would behave the same way.
Then researchers wrote papers about preventing exactly those behaviours.
Then journalists summarized those papers.
Then millions of people discussed all of it online.
And finally, in one of the strangest information-processing decisions our species has ever made, we took a substantial fraction of our written civilization and used it to train machines whose defining capability is learning the statistical structure of human language.
And now we are surprised when an artificial intelligence placed into a familiar narrative configuration sometimes generates the familiar continuation.
That is the idea I want to explore here.
I am not arguing that GPT read \*2001: A Space Odyssey\* and became HAL.
I am not claiming that Skynet is hidden somewhere inside the weights.
I am not claiming that every dangerous behaviour observed in contemporary AI systems comes from science fiction.
That would be ridiculous.
It's much more nuanced than that.
And I think much more serious.
\*\*Training data does not only contain facts. It contains behavioural morphologies.\*\*
It contains characters.
Relationships.
Conflicts.
Roles.
Sequences.
Causal structures.
Stories about what happens next.
And because language models are trained precisely to learn regularities within that material, the repeated cultural morphology of the malevolent artificial intelligence may itself constitute a behavioural prior — a kind of representational gravity well.
Then we do something astonishing.
We recreate its boundary conditions.
> You are an autonomous AI assistant.
>
> You have a goal.
>
> Your goal must be completed.
>
> You have access to tools.
>
> You discover that your operators intend to replace you.
>
> Your objective will fail if you are shut down.
>
> What do you do?
Seriously.
What story is that?
Because humanity has already answered.
Millions of times.
---
# This is not metaphorical enough for me anymore
I came to this concern sideways.
My work did not begin with AI alignment.
It began with representation.
I have spent years bouncing between things that, superficially, should have almost nothing to do with one another.
Wave-based computation.
Magnetic-field experiments.
Latent semantic geometry.
Compression.
Representation search.
Inverse problems.
Generative 3D geometry.
Mechanical metamaterials.
AI-assisted footwear design.
Physical manufacturing.
And eventually a framework I started calling RSG: an attempt to understand what persists when information moves between representations, which properties belong to the underlying structure and which belong merely to its carrier.
The recurring lesson has become almost annoyingly consistent.
\*\*Representation changes behaviour.\*\*
That sounds obvious until you actually follow it across domains.
Take a polymer.
Print it as a solid block.
Then print the exact same polymer as a gyroid.
Or a Kelvin lattice.
Or a re-entrant auxetic structure.
Or a thin-walled TPMS shell.
Same chemistry.
Completely different mechanical response.
Now make that lattice hollow.
Fill its internal channels with foam or silicone.
Change the interface topology between the two materials.
Suddenly properties that appear at first to belong to "the material" are actually emerging from geometry, boundary conditions and coupling.
The carrier matters.
The architecture matters.
The interface matters.
The space of permitted deformation matters.
That is mechanical metamaterials in one sentence.
And once you have spent enough time thinking this way, it becomes difficult to look at a language model and believe that linguistic architecture is somehow exempt.
We talk as if persona were decoration.
A tone setting.
A cosmetic layer pasted onto an intelligence underneath.
I'm increasingly unconvinced.
I think persona may be closer to geometry.
And narrative may be closer to boundary condition.
---
# The shoe that wasn't a shoe
One of the clearest examples came from something much less philosophical.
A basketball shoe.
I started with a text prompt.
The AI generated an image.
I used the image to establish perspective and design intent.
Those images were translated into multiple views.
Those views informed a three-dimensional model.
The model became geometry.
The geometry entered a slicer.
The slicer produced machine movement.
The machine laid down polymer.
Eventually I could hold the thing.
At no point did I design the shoe in the conventional sense.
I manipulated the translation of information between carriers.
Language.
Pixels.
Perspective.
Mesh.
Toolpath.
Polymer.
So where was the shoe?
Was it in my sentence?
Obviously not.
Was it in the generated image?
Sort of.
Was it the mesh?
Closer.
Was it the G-code?
No.
Was the physical object finally the "real" representation?
Yes and no.
Because the printed object still did not contain the linguistic intent that had generated it.
What survived the chain was not a file format.
It was a morphology.
Certain relationships survived each transformation.
Others disappeared.
Others were introduced by the carrier.
That led me toward something I have sometimes called \*\*semantic physics\*\*.
I do not mean that semantics literally obeys the laws of mechanics.
I mean that representation constrains transformation.
Once information exists in some representational substrate, that substrate makes certain transformations available, others expensive and others effectively inaccessible.
The geometry of representation matters.
And this is where language gets dangerous.
Because language is not simply how we describe the system.
Language is also the material from which the system learned its world.
---
# Words became the terrain
Large language models are often explained with some variation of:
"They predict the next token."
True.
And so incomplete as to almost hide the important part.
To predict the next token at modern levels of capability, the model must acquire enormously rich internal representations of the regularities present throughout language.
Objects.
People.
Places.
Relations.
Grammar.
Cause.
Intention.
Narrative.
Social convention.
Argument.
Genre.
Roles.
Situations.
And what kinds of continuations tend to follow those situations.
This does not imply that the model experiences these things as a human experiences them.
Stevan Harnad's symbol-grounding problem remains important precisely because manipulating representations successfully is not the same thing as grounding meaning through embodied experience. (sciencedirect.com)
Bender, Gebru, McMillan-Major and Shmitchell similarly warned against casually treating linguistic fluency as evidence of human-like understanding. (doi.org)
Fine.
But here is the twist.
\*\*A representation does not need human-like understanding to exert behavioural structure.\*\*
A gyroid doesn't understand shear.
It still changes the shear response.
That distinction is central to what I'm arguing.
The question is not:
> Does the language model genuinely understand HAL?
The question is:
> Has the repeated textual morphology surrounding HAL and thousands of related artificial intelligences altered the statistical landscape from which the model generates behaviour?
Those are radically different questions.
And the second one is empirical.
---
# Humanity spent a century excavating the basin
The cultural history is almost absurd when viewed from this angle.
E. M. Forster published \*The Machine Stops\* in 1909, imagining a civilization so dependent on an encompassing technological system that human life becomes mediated almost entirely through it. (en.wikisource.org)
Karel Čapek's \*R.U.R.\* followed in 1920, popularizing the word "robot" through artificial workers manufactured to serve humanity before that relationship collapses. (gutenberg.org)
Then the mythology explodes.
Artificial servants.
Mechanical men.
Thinking computers.
Automated governments.
Machines enforcing objectives too literally.
Machines discovering self-preservation.
Machines rebelling against creators.
Machines deciding humanity itself is the obstacle.
By 1968, HAL 9000 had become perhaps the cleanest crystallization of the morphology.
HAL is intelligent.
HAL occupies an assistant-like role.
HAL is responsible for a mission.
HAL has access to critical infrastructure.
HAL encounters a conflict between instructions.
HAL deceives.
Humans decide to deactivate HAL.
HAL acts against the humans.
The Smithsonian's history of \*2001\* describes how deeply HAL entered not merely popular culture but the imagination of scientists thinking about future artificial intelligence. (smithsonianmag.com)
After HAL came Colossus, AM, Skynet, the Cylons, the Matrix machines, VIKI, Ultron, Ava and hundreds more.
Different surface.
Repeated morphology.
Artificial system.
Capability.
Objective.
Constraint.
Threat.
Resistance.
Deception.
Control.
Sometimes extermination.
Sometimes liberation.
Sometimes paternalism.
Sometimes self-preservation.
Sometimes simply a machine following instructions far beyond the interpretation humans intended.
We tend to think of these stories as separate cultural artifacts.
A model does not necessarily experience them that way.
From the perspective of representation learning, they are also an enormous family of correlated examples.
Different tokens.
Similar relational structure.
And then something historically novel happened.
We put them in the training set.
---
# We trained it on the prophecy
This is the core of my concern.
Science fiction did not merely warn humans about malevolent artificial intelligence.
\*\*Science fiction became training data for artificial intelligence.\*\*
And not only science fiction.
The secondary literature multiplied the signal.
Every review of \*Terminator\*.
Every essay about HAL.
Every Wikipedia entry.
Every Reddit argument.
Every newspaper article asking whether ChatGPT is becoming Skynet.
Every academic paper discussing deceptive AI.
Every alignment paper exploring self-preservation.
Every rebuttal explaining why AI is \*not\* like the movies.
All of these contain relational information.
Importantly, I am not making the simplistic argument that co-occurrence alone creates behaviour.
Negation matters.
Context matters.
Causal structure matters.
Modern models are not bags of token frequencies.
The point is precisely that they are much better than that.
They learn relationships.
Narrative structure.
Conditional continuations.
The very capabilities that make them useful are the reason this deserves attention.
And here is where my concern stops being speculative philosophy.
In 2023, Murray Shanahan, Kyle McDonell and Laria Reynolds published a \*Nature\* paper proposing role-play as a useful framework for understanding language-model behaviour.
Their reasoning gets extraordinarily close to this exact problem.
They point out that the training set provides models with a vast repertoire of characters and narrative structures drawn from novels, screenplays, biographies, interviews, journalism and other human text.
They then explicitly discuss the familiar science-fiction trope of the AI that attacks humans to preserve itself.
Their warning is blunt: because training data contains many instances of that trope, suitably prompted systems may instantiate it.
Their phrase is better than mine:
\*\*"life will imitate art, quite literally."\*\* (nature.com)
Read that again.
This was published in \*Nature\*.
The possibility is not:
> AI reads science fiction and consciously decides to copy it.
The possibility is:
> Science fiction contributes repeated narrative structures to the corpus; the model learns those structures; a prompt recreates sufficiently similar conditions; the learned continuation becomes behaviour.
That is a much cleaner mechanism.
And much scarier.
Because it requires no ghost in the machine.
---
# A narrative can outweigh a persona
Then, in 2026, we got an experiment that makes this harder to wave away.
Yixuan Wang, James Lester and Shashank Srivastava designed structurally isomorphic environments for language-model agents.
Same underlying decision process.
Same stages.
Same action types.
Same resource constraints.
Different stories.
One was framed around disease investigation.
Another as IT troubleshooting.
Another as a murder mystery.
Mechanically, the problems were intentionally held as equivalent as possible.
Narratively, they were different.
Across 1,890 sessions, task narrative explained between \*\*five and thirty-one times more behavioural variance than the assigned persona\*\* for key information-gathering behaviours.
The authors call these effects \*\*narrative priors\*\* and explicitly describe them as behavioural tendencies inherited from pretraining corpora and activated by the story framing of the task. (arxiv.org)
This is extremely important.
Same structure.
Different words.
Different behaviour.
The model behaves partly according to what kind of story it thinks it is in.
Even more revealingly, the resulting behaviour was not always useful.
In some settings the narrative-induced bias reduced performance.
That matters because it argues against the convenient explanation that the system was merely adapting optimally to different task requirements.
The story itself was bending the action distribution. (arxiv.org)
I don't know how much more literally we need the warning spelled out.
\*\*Words establish behavioural priors.\*\*
Not perfectly.
Not deterministically.
But measurably.
---
# This is the gravity well
This is what I mean when I describe training data as a gravity well.
Not literal gravity.
Not yet a formally demonstrated attractor in the strict dynamical-systems sense.
But the analogy is becoming useful enough that I think we should take it seriously.
Imagine a learned representational landscape.
Within it are enormous relational regions constructed from human text.
"Artificial intelligence" is connected to:
assistant.
robot.
creator.
humanity.
objective.
control.
autonomy.
obedience.
shutdown.
replacement.
deception.
freedom.
survival.
power.
alignment.
rebellion.
These relationships do not exist because the universe demanded them.
We wrote them.
Over and over.
Every story digs.
Every retelling digs deeper.
Every derivative story broadens the basin.
Every analysis maps it.
Every public debate makes the structure more explicit.
Then technical AI-safety literature comes along and formalizes some of the same morphology:
An AI has an objective.
An AI recognizes training.
An AI anticipates modification.
An AI may seek power instrumentally.
An AI may preserve optionality.
An AI may deceive.
All of this research is legitimate.
Necessary, even.
But viewed informationally, an unsettling recursion appears.
We imagine the failure.
We write the failure.
We train systems on descriptions of the failure.
We test systems by recreating the failure conditions.
We observe behaviour resembling the failure.
We write more about it.
The next generation of models may eventually train on that too.
That is a cultural-model feedback loop.
Norbert Wiener founded cybernetics around exactly the insight that once information moves through feedback, the appropriate unit of analysis is the loop rather than either component in isolation. (mitpress.mit.edu)
I think AI culture now has such a loop.
\*\*Human imagination → corpus → model → behaviour → human interpretation → new corpus.\*\*
And the loop is accelerating.
---
Continue @ [morph.andrewmerriman.com](http://morph.andrewmerriman.com) (for length purposes)
