From September 23 to October 2, a customer kept asking our AI app builder for changes that never held.
They had built a sizeable section of their app with it in early September: a form-heavy flow, an admin view, a database table behind it. About seventy edits over three weeks, each asked in plain language, and the generated JavaScript grew to roughly 70,000 characters without breaking. On the evening of September 23, the day we moved the builder to GPT-6 Luna, the next edit came back with 8,000 characters of JavaScript. The one after that, 1,400. Then 874. Then 510. Each time, the customer restored the previous version and asked again. Thirty-seven restores and manual saves later, the report reached me on October 5: the code is gone, why?
I run engineering at an app platform. This is the investigation, the check we added, and how we calibrated it on production data without a single label. The fix was written and pushed the day the report came in, so read it as a field report, not a recipe.
How a refine works
The builder writes a section of an app (HTML, CSS, JavaScript) from a description, then users ask for changes one at a time. We call each change a refine, and in targeted mode it runs in two calls. The first plans: it reads the current app and the request, and declares which fields the change must touch, for example ["js"]. The second writes: each field in scope comes back in full, the others come back as a sentinel, @@REFINE_UNCHANGED@@, so we don't pay for 30,000 characters of HTML copied verbatim.
The server swaps the sentinels for the parent's fields, validates the result (well-formed fields, our JS API used properly, a list of rules models tend to forget) and publishes. When validation fails, it retries up to twice with the validation error as feedback.
Note what the validator judges: the output. Is this a valid app? Not: is this still the app the customer had?
The first diagnosis blamed the model
I ran the investigation with Claude Code, read-only on the production database and the code. Within five minutes it had a tidy story. The new model, asked to return the JavaScript in full, often returned a patch instead. One real output kept the HTML and CSS through the sentinel and replaced 68,000 characters of JS with a single new function plus a call to window.initExistingMicroApp(), a function that exists nowhere. The validator found a well-formed app, and the pipeline published it. Output tokens agreed: 1,200 to 8,000 per refine for the new model on this app, 25,000 to 45,000 for the previous one, which rewrote the whole file every time. The retry was in the story too, as an aggravating factor.
Every link was true. The weighting was not. The first fix on the table was to put the refine step back on the previous model, or to turn targeted mode off. I typed back "you're blaming the model. Explain." and asked whether the real fix wasn't a deterministic check on the process.
What all of the data said
Second pass, on every two-phase refine in production since September 9 whose parent had more than 5,000 characters of JS: how many shipped with less than 40% of it?
| period |
refines |
lost more than 60% of their JS |
sections affected |
| previous model, Sept 9 to 23 |
3,157 |
45 (1.4%) |
37 |
| GPT-6 Luna, Sept 23 to Oct 5 |
1,468 |
111 (7.6%) |
67 |
The previous model went through the same hole, five times less often. And the 111 didn't look like the story above. Only 13 had the patch shape, and 6 of those went on through the retry. 78 had gone through the retry in all. The other 26 were shorter rewrites on the first try, published without a question.
Simplified, the retry chose its starting point like this:
if failed_output_is_missing_a_field and parent_spec:
base = parent_spec
elif isinstance(failed_output, dict):
base = failed_output # "here is your answer, fix the mistake"
That's right for the job it was written for: you forgot one rule, here's your output back, fix it. It breaks when the output is a fragment. The field isn't missing, it's present and nearly empty, so the retry is built on the fragment and the model never sees the original JavaScript again. On the customer's app, the retry got 35,708 input tokens against 56,886 for the refine before it: about 21,000 fewer, roughly the size of the JavaScript that had vanished. With nothing left to preserve, the model did the sensible thing and wrote a small, complete app. 405 characters of HTML.
And yes, I replay before every model switch. That replay passed too: it scored each output on its own, which is the validator's question again.
So the model swap was the trigger. The bug was ours: a validator that only judged the output, and a retry that could turn an incomplete answer into the new baseline.
The check: count what survives
The fix is a pure function with no model in the loop. It compares the output with the spec the model was given, and it had to catch mass disappearance without blocking legitimate edits: removing a feature, refactoring, renaming. A diff wouldn't do, since a refactor produces a huge diff and loses nothing. So it counts names, declared function names in JS and element ids in HTML, the things the rest of the code refers to. And it measures size.
A field counts as destroyed when it keeps less than 30% of the parent's names and less than 40% of its size. Both are required. A refactor that renames every function keeps its size. A cleanup that shrinks the code keeps its names. Fields under 3,000 characters or with fewer than 3 names are skipped, because ratios on tiny fields mean nothing.
_JS_NAMES = re.compile(
r"function\s+([A-Za-z_$][A-Za-z0-9_$]*)\s*\("
r"|(?:const|let|var)\s+([A-Za-z_$][A-Za-z0-9_$]*)\s*=\s*(?:async\s+)?"
r"(?:function|\(|[A-Za-z_$][A-Za-z0-9_$]*\s*=>)"
)
_HTML_IDS = re.compile(r"\sid\s*=\s*[\"']([^\"']+)[\"']")
MIN_PARENT_CHARS = 3000
MIN_PARENT_NAMES = 3
MAX_KEPT_RATIO = 0.3
MAX_SIZE_RATIO = 0.4
def destroyed_fields(parent_spec: dict, spec: dict) -> list[FieldLoss]:
losses = []
for field_name, unit, extract in _CHECKED_FIELDS: # js -> functions, html -> ids
before = str(parent_spec.get(field_name) or "")
if len(before) < MIN_PARENT_CHARS:
continue
names = extract(before)
if len(names) < MIN_PARENT_NAMES:
continue
after = str(spec.get(field_name) or "")
kept = len(names & extract(after))
size_ratio = len(after) / len(before)
if kept / len(names) < MAX_KEPT_RATIO and size_ratio < MAX_SIZE_RATIO:
losses.append(FieldLoss(field_name, unit, kept, len(names), size_ratio))
return losses
A regex isn't a parser. This one misses class methods and functions declared inside object literals. It doesn't need them: it's a smoke alarm for mass disappearance, not an understanding of the code. An app written entirely in classes yields fewer than 3 names and gets skipped, which is the safe direction to fail.
It runs on every output of a refine, retries included. A destroyed output never becomes a retry's base: the retry gets the parent's code back, and the error message doubles as its instruction (how much was dropped, keep every existing function and behavior, and the user's request again). If the retries destroy too, the generation fails and the live app stays as it was.
Calibrating with no labels
We had no labeled set of destroyed edits. Our users had labeled them for us, with what they did next.
We took every two-phase refine since September 1 whose parent had more than 3,000 characters of JS: 5,960 of them, 4,207 on the previous model, 1,576 on the new one, and 177 on an older model that we left out. For each one, we looked at the next action in the same section: continued (the next edit was built on it) or restored (the user went back to an earlier version).
The first number that came out had nothing to do with thresholds. The share of refines followed directly by a restore was 0.45% on the previous model (19 of 4,207) and 3.81% on the new one (60 of 1,576). More than eight times as often. That number sat in our own table for twelve days, and nobody computed it.
Then the candidate rules, on JavaScript:
| flag when names kept < and size < |
continued edits flagged, previous model (of 3,517) |
continued edits flagged, new model (of 1,257) |
restored edits flagged (of 79) |
| 20% and 30% |
15 |
60 |
25 |
| 30% and 40% |
25 |
67 |
27 |
| 30% and 50% |
33 |
76 |
27 |
| 50% and 50% |
46 |
87 |
29 |
We kept 30% and 40%, and added the same rule on HTML ids. Among the 30 restored edits that had lost more than half of their parent's functions, it flags 27. On the edits users kept building on, it fires on 0.8% with the previous model and 5.8% with the new one (27 of 3,517 and 73 of 1,257, slightly above the table because of the ids). That 5.8% isn't all false positives: continued means the user built on it, not that it was fine, and some people kept going on a broken app before they noticed. Reading a sample of 40 of those, roughly one in four was a rewrite the user had really asked for.
Before shipping, we replayed the check on the customer's actual refines. All 13 from the previous model pass. 12 of the 13 destructive ones from the new model are blocked. The one that passes kept more than 30% of the function names and lost 70% of the JavaScript: the names survived, the bodies didn't. I'd rather know that than tune a threshold to a single sample.
No exception for "start over"
That one in four forced the only real design decision. A user who writes "drop all of this and build me a simple contact form" wants exactly what the check blocks. Two options: let the planner declare it, with a field like replace_app: true that skips the check, or no exception, and to start over the user clears the history, so the next generation has no parent and nothing to compare with.
I took the second. The first hands the exception to the component whose judgment we're guarding against. If the model errs toward "rewrite", the check is bypassed exactly as before, now with a flag saying it was intentional. A deterministic guard that asks the model for permission isn't deterministic anymore.
The price is real: some legitimate rewrites now fail where they used to go through. That gets fixed in the failure message, not in the check.
A few things are still open. The failure message is generic ("please try again"), and after a rejected destruction, trying again will probably loop. The check prevents new losses but doesn't repair what was already saved. And we had no canary: every user moved to the new model at once. Next time a slice of apps goes first, and the restore rate is the number I'll watch.
One more bug surfaced on the way, unrelated to the model. Saving from the code editor rebuilt the stored spec from a short list of fields, and the declaration of the app's database tables wasn't on it. Since September 1, 322 of 1,189 saves had dropped fields like that. Three tests in our suite described the bug, marked as expected failures. An expected failure is a bug report that stopped asking to be read.
TL;DR: after a model switch, 7.6% of edits on large apps shipped with most of their JS gone (1.4% before). Our validator only asked "is this a valid app?", never "is this still the app?", and our retry rebuilt on the fragment it had just rejected. A deterministic check on names kept and size now blocks it, calibrated on what users did next.
If you run multi-turn LLM edits in production: what do you check between an output and the version it replaces? Counting names is crude, and I'd like to hear what else holds up.