RAG Retrieval
Finding the right piece of your own material to answer from. This is where answer quality is mostly decided, and it is the part vendors talk about least.
Finding the right piece of your own material to answer from. This is where answer quality is mostly decided, and it is the part vendors talk about least.
What it does
Retrieval is the step that takes a question and finds the relevant passage in your approved material. Everything the assistant says afterwards is built from what retrieval returned, which makes it the single largest determinant of whether an answer is right.
It is also the mechanism that makes grounding possible at all. An assistant that answers from retrieval is answering from documents you control and can point to. One that answers from a model's training is producing something plausible with no source behind it, and the difference is invisible in the reply itself.
Where it shows up in a conversation
Every grounded answer
The retrieval step sits behind every reply drawn from your material.
Policy questions
Finding the specific clause that answers a question rather than the general page.
Product information
Matching a customer's phrasing to the right product content.
Procedure lookup
Returning the current version of a process rather than an older one.
Document questions
Locating the relevant passage in a file a customer just uploaded.
Citation
Being able to say where an answer came from, which is what makes review possible.
Why it is harder in two languages
Retrieval in two languages is not two retrieval problems solved separately — it is one problem that has to work across both. A customer asking in Arabic about content written in English must still find it, because organisations rarely maintain perfectly parallel material.
The harder issue is phrasing. Retrieval matches a question against material, and the way people ask differs enormously from the way documents are written. A policy headed "Termination provisions" has to be found by someone asking "how do I cancel", in either language, informally, possibly misspelt.
Arabic widens that gap. Formal written Arabic and the Arabic people type are meaningfully different, so the distance between a document's phrasing and a customer's phrasing is larger than in English. Closing it requires work on the Arabic side specifically rather than a general retrieval improvement.
What to look for when you evaluate it
| Retrieval quality in Arabic | Measured separately from English. An overall figure hides the gap that matters, and the gap is usually substantial. |
|---|---|
| Informal phrasing | Whether questions asked the way customers ask them find the right material. |
| Cross-language retrieval | Whether an Arabic question can find relevant English material and vice versa, since parallel content is rare. |
| Version handling | Whether the current version is returned rather than a superseded one. |
| Citation | Whether an answer can be traced to its source passage, which is what makes audit possible. |
| Behaviour when nothing matches | Whether it declines or returns the nearest thing and answers from it anyway. |
Where it should not be trusted alone
Retrieval should not be assumed to have succeeded. Returning the nearest passage is not the same as returning a relevant one, and a system that always returns something will always produce an answer — including for questions your material does not cover.
This is the mechanism behind most confident wrong answers. The retrieval returned something tangentially related, the reply was composed from it fluently, and nothing in the output signals that the source did not actually address the question.
The defence is a threshold and a check: below a relevance threshold, decline rather than answer, and verify the finished reply against what was retrieved before sending. Both are deterministic steps rather than instructions to be careful.
How this fits the whole
No single capability answers a question on its own. A conversation passes through several in sequence — understanding what was asked, finding the relevant material, composing a reply, checking it before it is sent — and the quality of the result is set by the weakest link rather than the strongest.
That has a practical consequence for evaluation. Vendors tend to demonstrate their strongest component, and a system with excellent retrieval and weak language handling will demo beautifully and disappoint in production, because the demo was conducted in the language and phrasing the system handles best.
It also explains why grounding matters more than raw capability. A more capable model that answers from its own training rather than from your material produces confident, fluent, unsupported answers — which is a worse outcome than a less capable one that retrieves correctly and declines when it cannot. Capability without grounding increases the cost of being wrong rather than reducing it.
The pieces are described separately here because that is how they are evaluated and configured. They are not experienced separately: a customer asking a question encounters one system, and judges it as one.
Where this sits in a conversation
- Something arrivesA typed message, a spoken sentence, an uploaded document. The first job is establishing what was actually received — which language, which script, whether it is legible, whether it is one question or several.
- It is interpretedWorking out what is being asked rather than what words were used. The same question arrives phrased twenty different ways, in two languages, often mixed, frequently misspelt, sometimes as a fragment.
- Material is retrievedThe relevant part of your own approved content is found. This is where answer quality is largely decided: a correct answer assembled from the wrong source is still wrong.
- A reply is composedIn the language of the question, in the configured tone, from the retrieved material and nothing else. Where the material is silent, the reply says so rather than filling the gap.
- It is checkedBefore sending, the reply is examined for claims the retrieved material does not support and for personal data that should not be repeated back. This runs on every reply.
- It is deliveredAs streamed text, as speech, or as a structured response — and the conversation is recorded so that what happened can be reviewed later.
Why this is built rather than bought in
Every capability described here could, in principle, be obtained by calling somebody else's service. That is how most assistants in this market are assembled, and it is why so many of them cannot answer the question that decides regulated procurements: where does the data go.
When a component is a call to an external provider, every question that passes through it leaves the organisation. That may be perfectly acceptable for a retailer answering questions about opening hours. It is usually not acceptable for a bank explaining a transaction, a hospital confirming an appointment, or a public body handling a citizen case — and the constraint is architectural, so it cannot be configured away later.
The second consequence is control over behaviour. A capability obtained as a service behaves the way the provider decides, changes when the provider changes it, and cannot be tuned for the specific way your customers speak or write. A capability that runs inside your own estate can be adjusted for dialect, for your terminology, and for the particular failure modes your material produces.
The third is continuity. External services are deprecated, repriced and withdrawn. A deployment whose core capability depends on somebody else's roadmap inherits that roadmap, including the parts nobody announced in advance.
Measuring this component honestly
| Measured in which language? | Nearly every published benchmark in this field was developed against English. An overall figure that averages across languages hides exactly the gap that matters in this market. |
|---|---|
| Measured on what input? | Clean or realistic. Read speech or spontaneous speech. Typed text or phone-typed text with autocorrect. The difference between these is usually larger than the difference between vendors. |
| Measured by whom? | Self-reported figures with an undescribed method are marketing. Ask for the test set and the procedure. |
| What counts as correct? | A surprising number of published figures count a partially correct answer as correct. Ask what the scoring rule was. |
| How does it fail? | A component that degrades gracefully is worth more than one with a higher headline figure that fails silently and confidently. |
| Does it report uncertainty? | A component that surfaces low confidence lets the system confirm rather than guess. One that hides it produces confident errors downstream. |
What to ask about this specifically
Ask for the Arabic figure
Not the overall figure. Nearly every published benchmark in this field was developed against English, and the gap between the two languages is where deployments disappoint.
Ask how it was measured
Against what test set, by whom, on what kind of input. A number without a described method is marketing rather than measurement.
Ask what it does when uncertain
Whether uncertainty is surfaced or hidden. A component that hides low confidence produces confident errors downstream.
Ask about real conditions
Noisy audio, poor scans, dialect, informal phrasing, mixed languages. Laboratory conditions are not predictive of contact-centre reality.
Ask where processing happens
For any component touching personal data, this is a procurement question rather than a technical curiosity in regulated sectors.
Ask what is retained
What data this component keeps, for how long, and whether it is used to improve anything.
Common questions
This does. The quality and organisation of your own material, and whether retrieval finds the right part of it, matter more than any other single factor.
Cross-language retrieval means an Arabic question can find English material. It is better to have parallel content, but rare in practice, and the system should not require it.
Matching is built around how people actually ask rather than how documents are written — which is a larger gap in Arabic than in English.
It should decline rather than answer from the nearest passage. A system that always returns something will always produce an answer, including for questions you never covered.
Yes, and this is what makes review possible. Being able to point at the source passage is the difference between an auditable answer and an assertion.
Whenever it changes, with a named owner. Retrieval returning a superseded policy confidently is one of the quieter ways a deployment goes wrong.