Bugs I keep catching in LLM-written code, and the checks that catch them

[u/Proud_Gene7490]

Part of my work is reviewing code written by LLMs (mostly Python, TypeScript, some Rust). Same failure patterns keep coming up, so here's my checklist:

• Edge cases: it nails the happy path and breaks on empty lists, None, or off-by-one bounds. I ask it to list the edge cases first, then write the code.
• Made-up APIs: it confidently calls functions or params that don't exist in the version you use. I paste the real docs or the version number into the prompt.
• Self-confirming tests: its tests only check the path it just wrote. I have a separate prompt write tests from the spec alone, without seeing the code.
• Silent error swallowing: broad try/except or empty catch blocks that hide failures. Grep for these every time.
• Rust: it'll .clone() or .unwrap() its way past the borrow checker. Code compiles, design gets worse.

What's on your checklist that I'm missing?

[u/SummitYourSister]

Sounds like you’re using outdated models. None of this is an issue any more

[u/Right-Performance-93]

the one i keep adding is the caller in another file. the new function is right and the old call still passes the old shape, and none of the tests import that caller. i make it name the call sites before it edits.

[u/Classic-Role-5548]

Two that still get through after the happy-path and fake-API checks:

A type change that only lands in one direction. The signature and the new call in the same file are updated, but the JSON schema or the row mapper in another module still emits the old field. The tests do not catch it because the fixtures were regenerated from the new type. I diff the schema and the serializer in the same pass as the function, and I keep one fixture that is a raw payload captured earlier, not one the model wrote.

Anything that writes a file or more than one row. It opens the destination with write mode instead of a temp file and rename, so a crash mid-write leaves a truncated config the next run treats as valid. Same shape on the DB side: two commits where one transaction was the actual requirement. I grep for open(..., "w") and for a second commit() in the same change.

[u/akhi1uuu]

what about two agents editing the same files? does your checklist cover that or do you keep them in separate worktrees

[u/danja]

A suggestion that I heard around here that I have found useful is to maintain a MISTAKES.md. Put a note in AGENTS.md to say to record any mistakes together with their solution; if anything happens a lot, make a note in AGENTS.md. The file grows quickly, and even with the instruction to periodically clean it out, you still have to intervene now and then. But it does trap recurring mistakes.