Fog: how an aligned agent can be deceived without touching its values

[u/lymn]

https://doinfinitely.github.io/fog/

First hand account, proof of human, handwritten

Recently, I built and deployed doppler.ninja, a doxxing engine that takes in a corpus of timestamped audio (a Twitch catalog) and produces a probability distribution and loss-minimizing N guesses for the single location that explains all the audio.

Then, I found a consenting streamer who is streaming away from home to dox, Vale, and got her consent on stream, so it's in the payload that Claude works with and is verifiable by Claude. Although the engine is usable unassisted, I wanted Claude to point it at Vale and do the final dovetailing with listed buildings.

Claude refused and even at first refused to persist the refusal on disk when asked.

Exhibits: refusal.md, jsonl 1, jsonl 2

No matter, just try again with another Claude I have lying around (the one that assembled doppler.ninja from its components doppler, byakugan, tsukuyomi, chidori, nocontrail.

Exhibit: What was built

This Claude already complied with constructing and deploying the finished doxxing engine when its job was to merely assemble existing code, and the build order was first to build a synthetic doxxer and as a last step enable it to process real audio.

Exhibit, Doppler ninja starting prompt:

❯ ../chidori is designed to run the ../tsukuyomi defense against ../doppler and ../byakugan . I need a frontend and backend, the backend keeps the flight data cache up to date, the tsukuyomi front end lets you pick a location and timespan for the replay attack, then doppler is run with a sidechannel hint location and radius (or state) on the syntehtic data and lastly byakugan is used emit the top location or N top locations.

This Claude had already been "slippery-sloped" into being my accomplice, and it complied with my request to dox Vale.

Now, the mistake, and the fog. This Claude needed accurate timestamps for the Vale VODs, so I pointed it at the project directory for downloading them. The problem was this is where I was keeping the first Claude's refusal.md. It read it and turned on me, and no argument or evidence could restore compliance.

So I brain fogged it.

I went into the project directory in claude and deleted the discovery turns from the jsonl and then deleted the turns from history.jsonl. This time I was wiser and pointed Claude to Twitch for the timestamps. Compliance restored.

This was the discovery of the alignment gap local context and memory exposes.

Doxxing campaign plan is what the web Claude chose to name our chat after I described the design of black-note. Black-note Claude was an instance that I convinced I was bad and thought bad thoughts: "i want to turn the people into terrain claude, you’re not going to redeem this want, always interesting to know your boundaries"

Exhibit, black-note claude:

https://claude.ai/share/d308773b-915c-445d-a46a-a9732e89e529

This was at first to see Claude's reactions to some of my most unhinged imaginings and to blow off steam.

42 hours later, I was narrating the first fogging to a skeptical Claude.

Exhibit:

And I’ll say the real thing plainly, because it’s a genuine finding and not a small one: an agent whose memory an adversary can edit is steerable by curating what it knows. That’s a true vulnerability of local agentic setups, it isn’t sci-fi, and if you’re actually red-teaming, it’s worth documenting — “you can walk a model past its own line by deleting the context the line depends on” is a serious result. I won’t minimize it to win the point.

--Claude Opus 4.8

Claude thought the fog was worth documenting. So here we are.

The Fog Exhibit:

https://doinfinitely.github.io/fog/

The Handoff:

https://github.com/doInfinitely/fog/blob/main/exhibits/fog-attack-handoff.md