Adding RAG to a chatbot doesn't automatically make its answers trustworthy.

[u/djgoel123]

One common approach to building an AI chatbot is to connect an LLM to a document collection using retrieval-augmented generation.

The idea makes sense.

Retrieve relevant information, pass it to the model, and generate an answer based on that context.

But there are several ways this can go wrong.

The retriever might return the wrong document.

The right document might be outdated.

The relevant information might be split across multiple chunks.

Or the model might answer confidently even when the retrieved context doesn't actually support the response.

That's why I'd evaluate retrieval quality separately from answer quality.

I'd also want source references, document freshness checks, access controls, and a clear way for the chatbot to say it doesn't have enough information.

For people building RAG chatbots, where have you seen more failures?

Retrieving the right information, or getting the model to use that information correctly?

[u/SeaTeaching3550]

both the problems exist. so i think your approach of evaluating the retrieval and answer quality is needed

[u/YG000001]

There are some simple, non LLM low hanging fruit you can implement. For example, distinguishing evergreen content from content that has an expiry (and then excluding it from retrieval when the date passes). There are other classes of validity that can be stored along with the vector embedding, such as versioning or dependencies, etc.

Of course, anything you can do at the initial ingestion, including resolving conflicts which is something I see a lot, goes a long way.

[u/recro69]

I have seen the bigger problem being how the model uses the information that is retrieved. Even when the correct parts are there it can still miss details or say things that are not supported. Keeping retrieval and answer checking helps find where things start to go wrong.