Look at retrieval before the answer

Separate finding from answering

Retrieval and response generation are connected, but they are different parts of the system. Retrieval decides which pieces of information reach the model. Generation decides what to do with those pieces.

Inspecting them separately makes debugging more concrete. If the source passage is missing, a more emphatic instruction to answer accurately will not put it into the context. If the passage is present, the next investigation can focus on how the response used it.

Write questions before tuning the index

A small collection of realistic questions gives the team something to test. Use the language that users are likely to write, including incomplete phrasing and terms that differ from the source documents.

For each question, identify the material that should support the answer. This provides a reference for reviewing retrieval results without having to judge them only by the final response.

Give the source material useful structure

Documents vary in layout and purpose. A short policy, a long technical guide and a structured database record should not automatically be treated as the same kind of content.

Chunk boundaries and metadata can preserve information about the source, its section and its intended use. Filters can then narrow the search when the application already knows something about the user’s task or permitted scope.

• Keep source identifiers with retrieved passages.
• Preserve useful headings and document context.
• Decide how changed material replaces old material.
• Apply access boundaries before returning context.

Review what came back

A practical review looks at the retrieved passages themselves. Are they relevant to the question? Do they contain the needed detail? Are multiple similar passages crowding out a more useful source?

This is where changes to search, filtering or reranking can be tested deliberately. Each change should address an observed retrieval problem rather than simply adding another component to the pipeline.

Let the assistant stop at the evidence

Some questions will not have a supported answer in the available material. The product needs a way to acknowledge that, ask for clarification or direct the user to another step.

Source references help users inspect an answer and help engineers inspect the system. A knowledge assistant becomes easier to trust when its relationship to the source material is visible.