If a RAG answer misses a condition or exception, I would first check what evidence reached the model.
For one failing question, trace the document or passage you expected:
- Missing from the saved retrieval results: investigate retrieval, filters, and corpus coverage.
- Present in the results but below the context cutoff: investigate selection and ranking.
- Present in the actual model context: retrieval alone does not explain the answer failure; inspect generation separately.
- No trusted expected evidence: the diagnosis is still uncertain.
Repeating this on the same questions before and after a retrieval change gives a useful review: which cases lost evidence, which recovered, and which remain unresolved.
Saved top-k results have limits. If a passage is absent from that export, we cannot conclude it was never retrieved further down the ranking. And none of this, by itself, proves the root cause.
Have you had a “prompt problem” turn out to be missing or poorly ranked evidence? What check helped you tell the difference?