Speech-to-Text
Turning what somebody said into text they would recognise as what they said. Everything downstream depends on getting this right, and in Arabic it is considerably harder than the English case.
Turning what somebody said into text they would recognise as what they said. Everything downstream depends on getting this right, and in Arabic it is considerably harder than the English case.
What it does
Speech recognition converts spoken audio into text. It is the first step in any voice interaction and the one where errors compound worst: a misheard word changes the apparent question, and the assistant then answers that different question fluently and confidently.
The difficulty is that people do not speak the way text is written. They hesitate, restart, run words together, use dialect, and do it in cars, shops and streets rather than in quiet rooms. A system evaluated on read speech in studio conditions will report figures that do not survive contact with real callers.
Where it shows up in a conversation
Voice conversations
The foundation of any spoken interaction, on the phone or in the widget.
Voice notes
Where customers send audio rather than typing, which is common on messaging channels.
Call analysis
Turning recorded conversations into text that can be reviewed and measured.
Accessibility
Serving people for whom typing is impractical or impossible.
Hands-busy contexts
Drivers, field engineers, warehouse and clinical staff.
Dictated input
Where speaking is simply faster than typing on a phone.
Why it is harder in two languages
Arabic speech recognition is structurally harder than English for reasons that are not about effort. The language is diglossic: the formal written variety and the spoken varieties differ substantially in vocabulary, structure and rhythm. A model trained predominantly on broadcast-standard Arabic will transcribe a Saudi caller speaking naturally with materially worse accuracy.
The Kingdom compounds this. Najdi, Hijazi and Gulf varieties differ from each other, and a large expatriate workforce brings Egyptian and Levantine Arabic into the same contact centre. A system tuned to one variety will underperform on the others.
Code-switching is the third factor and the one most often overlooked. Saudi speakers routinely embed English terms — product names, technical vocabulary, numbers — inside Arabic sentences. A pipeline that must commit to one language per utterance will mis-transcribe most real speech.
What to look for when you evaluate it
| Accuracy on dialect | The only figure that matters is accuracy on how your customers actually speak. Ask for it specifically; studio figures on read speech are not predictive. |
|---|---|
| Noise robustness | Performance in a car, a shop, a street. This is where calls happen. |
| Code-switching | Whether a sentence containing both Arabic and English survives transcription. |
| Latency | How quickly text is available, since everything downstream waits on it. |
| Confidence reporting | Whether the system indicates uncertainty, which is what allows the assistant to confirm rather than guess. |
| Handling of names and numbers | Account numbers, references and proper nouns are where mis-transcription costs most. |
Where it should not be trusted alone
Recognition should not be trusted silently where the consequence is significant. A misheard account number or reference produces a confident answer about the wrong record, and the caller has no way to know that is what happened.
The mitigation is confirmation rather than better recognition. Reading back the understanding before acting costs two seconds and prevents the entire category. Systems that skip it to sound faster fail more expensively and less visibly.
It should also not be used as an authentication signal. A voice is not a credential, and any disclosure decision should rest on identity established properly rather than on the assistant believing it recognises someone.
How this fits the whole
No single capability answers a question on its own. A conversation passes through several in sequence — understanding what was asked, finding the relevant material, composing a reply, checking it before it is sent — and the quality of the result is set by the weakest link rather than the strongest.
That has a practical consequence for evaluation. Vendors tend to demonstrate their strongest component, and a system with excellent retrieval and weak language handling will demo beautifully and disappoint in production, because the demo was conducted in the language and phrasing the system handles best.
It also explains why grounding matters more than raw capability. A more capable model that answers from its own training rather than from your material produces confident, fluent, unsupported answers — which is a worse outcome than a less capable one that retrieves correctly and declines when it cannot. Capability without grounding increases the cost of being wrong rather than reducing it.
The pieces are described separately here because that is how they are evaluated and configured. They are not experienced separately: a customer asking a question encounters one system, and judges it as one.
Where this sits in a conversation
- Something arrivesA typed message, a spoken sentence, an uploaded document. The first job is establishing what was actually received — which language, which script, whether it is legible, whether it is one question or several.
- It is interpretedWorking out what is being asked rather than what words were used. The same question arrives phrased twenty different ways, in two languages, often mixed, frequently misspelt, sometimes as a fragment.
- Material is retrievedThe relevant part of your own approved content is found. This is where answer quality is largely decided: a correct answer assembled from the wrong source is still wrong.
- A reply is composedIn the language of the question, in the configured tone, from the retrieved material and nothing else. Where the material is silent, the reply says so rather than filling the gap.
- It is checkedBefore sending, the reply is examined for claims the retrieved material does not support and for personal data that should not be repeated back. This runs on every reply.
- It is deliveredAs streamed text, as speech, or as a structured response — and the conversation is recorded so that what happened can be reviewed later.
Why this is built rather than bought in
Every capability described here could, in principle, be obtained by calling somebody else's service. That is how most assistants in this market are assembled, and it is why so many of them cannot answer the question that decides regulated procurements: where does the data go.
When a component is a call to an external provider, every question that passes through it leaves the organisation. That may be perfectly acceptable for a retailer answering questions about opening hours. It is usually not acceptable for a bank explaining a transaction, a hospital confirming an appointment, or a public body handling a citizen case — and the constraint is architectural, so it cannot be configured away later.
The second consequence is control over behaviour. A capability obtained as a service behaves the way the provider decides, changes when the provider changes it, and cannot be tuned for the specific way your customers speak or write. A capability that runs inside your own estate can be adjusted for dialect, for your terminology, and for the particular failure modes your material produces.
The third is continuity. External services are deprecated, repriced and withdrawn. A deployment whose core capability depends on somebody else's roadmap inherits that roadmap, including the parts nobody announced in advance.
Measuring this component honestly
| Measured in which language? | Nearly every published benchmark in this field was developed against English. An overall figure that averages across languages hides exactly the gap that matters in this market. |
|---|---|
| Measured on what input? | Clean or realistic. Read speech or spontaneous speech. Typed text or phone-typed text with autocorrect. The difference between these is usually larger than the difference between vendors. |
| Measured by whom? | Self-reported figures with an undescribed method are marketing. Ask for the test set and the procedure. |
| What counts as correct? | A surprising number of published figures count a partially correct answer as correct. Ask what the scoring rule was. |
| How does it fail? | A component that degrades gracefully is worth more than one with a higher headline figure that fails silently and confidently. |
| Does it report uncertainty? | A component that surfaces low confidence lets the system confirm rather than guess. One that hides it produces confident errors downstream. |
What to ask about this specifically
Ask for the Arabic figure
Not the overall figure. Nearly every published benchmark in this field was developed against English, and the gap between the two languages is where deployments disappoint.
Ask how it was measured
Against what test set, by whom, on what kind of input. A number without a described method is marketing rather than measurement.
Ask what it does when uncertain
Whether uncertainty is surfaced or hidden. A component that hides low confidence produces confident errors downstream.
Ask about real conditions
Noisy audio, poor scans, dialect, informal phrasing, mixed languages. Laboratory conditions are not predictive of contact-centre reality.
Ask where processing happens
For any component touching personal data, this is a procurement question rather than a technical curiosity in regulated sectors.
Ask what is retained
What data this component keeps, for how long, and whether it is used to improve anything.
Common questions
That is the requirement rather than an enhancement. Callers do not speak Modern Standard Arabic, and a system tuned only to broadcast speech will transcribe them poorly and then answer the wrong question confidently.
Real calls happen in cars, shops and streets. Performance under those conditions is what matters, and it is not predicted by figures from quiet-room testing.
Yes, and it must — a typical Saudi utterance contains both. Pipelines that commit to one language per utterance mis-transcribe most real speech.
Where the request carries consequence, the understanding is confirmed before anything is acted on. That is the defence, not better recognition.
Only as your retention policy allows, and voice is treated as sensitive personal data with a short default retention.
Voice is not treated as a credential. Identity for disclosure purposes comes from proper authentication, not from the system believing it recognises a voice.