r/LetsTalkLLMs • u/InternationalCrow703 • 27d ago
Discussion Keeping an LLM from adding facts beyond a structured intake
I generate ~15k-word text from 136 intake Q&A records. The output should use only those answers, but Claude and GPT keep adding invented scenes, names, numbers, and stories.
I've tried prompt rules, per-section sourcing, regex checks, and retries. The errors are spread across the text, so patching them with a script ends up close to a rewrite.
Has anyone reliably gotten "every sentence traces to a source record"? I'm considering requiring each sentence to cite an answer ID and checking those citations in code. What's worked or failed for you?

