Sentiment Analysis
Noticing how somebody feels, not just what they asked. Mostly useful as an escalation signal, and frequently over-claimed as an analytics feature.
Noticing how somebody feels, not just what they asked. Mostly useful as an escalation signal, and frequently over-claimed as an analytics feature.
What it does
Sentiment analysis assesses the emotional tone of a message or a conversation. In an assistant its most valuable use is operational rather than analytical: recognising that somebody is frustrated and escalating before the conversation deteriorates further.
Its second use is aggregate — understanding how conversations are going across a service. That is genuinely useful, and it is also where sentiment is most often over-interpreted, because a single score compresses a great deal of nuance into a number.
Where it shows up in a conversation
Escalation
Recognising frustration early and routing to a person before it becomes a complaint.
Prioritisation
Ordering a queue by how badly a conversation is going rather than by arrival time.
Quality review
Finding conversations worth listening to among thousands that were fine.
Trend analysis
Whether service is improving, at the aggregate level rather than per conversation.
Agent support
Flagging to a human agent that an incoming conversation is already difficult.
Early warning
Detecting that something has gone wrong operationally from a shift in tone at scale.
Why it is harder in two languages
Sentiment is culturally and linguistically specific in ways that make naive cross-language analysis unreliable. Arabic business communication carries higher baseline formality and more elaborate courtesy than English, and a direct English-style complaint can appear in Arabic wrapped in polite framing that a system tuned to English reads as satisfaction.
The reverse also happens. Arabic expressions of emphasis can appear stronger than intended when assessed by English norms, producing false escalations from customers who were merely being direct.
Dialect adds another layer. Colloquial expressions of frustration differ substantially between Gulf, Egyptian and Levantine Arabic, and a system tuned to one will misread the others — which matters given how mixed a typical Saudi customer base is.
What to look for when you evaluate it
| Arabic accuracy separately | Sentiment is where cross-language transfer fails most quietly. |
|---|---|
| Politeness handling | Whether formal courtesy is distinguished from genuine satisfaction. |
| Dialect coverage | Whether colloquial frustration is recognised across varieties. |
| Escalation usefulness | Whether flagged conversations genuinely needed a person, measured after the fact. |
| False positive rate | Over-escalation wastes human capacity and trains agents to ignore the signal. |
| Aggregate stability | Whether trends are meaningful over time or noisy enough to be unusable. |
Where it should not be trusted alone
Sentiment should not be used to make decisions about individuals. Judging that a customer is difficult, or that an employee is disengaged, on the basis of an automated tone score is both unreliable and, depending on context, a serious fairness problem.
It should not be reported as a precise measurement. A sentiment score is a rough signal with meaningful error, and presenting it as a percentage to two decimal places implies a certainty that does not exist.
And it should not be the only escalation mechanism. It is a useful early signal, but a customer who asks for a person must reach one regardless of what the tone analysis concluded.
How this fits the whole
No single capability answers a question on its own. A conversation passes through several in sequence — understanding what was asked, finding the relevant material, composing a reply, checking it before it is sent — and the quality of the result is set by the weakest link rather than the strongest.
That has a practical consequence for evaluation. Vendors tend to demonstrate their strongest component, and a system with excellent retrieval and weak language handling will demo beautifully and disappoint in production, because the demo was conducted in the language and phrasing the system handles best.
It also explains why grounding matters more than raw capability. A more capable model that answers from its own training rather than from your material produces confident, fluent, unsupported answers — which is a worse outcome than a less capable one that retrieves correctly and declines when it cannot. Capability without grounding increases the cost of being wrong rather than reducing it.
The pieces are described separately here because that is how they are evaluated and configured. They are not experienced separately: a customer asking a question encounters one system, and judges it as one.
Where this sits in a conversation
- Something arrivesA typed message, a spoken sentence, an uploaded document. The first job is establishing what was actually received — which language, which script, whether it is legible, whether it is one question or several.
- It is interpretedWorking out what is being asked rather than what words were used. The same question arrives phrased twenty different ways, in two languages, often mixed, frequently misspelt, sometimes as a fragment.
- Material is retrievedThe relevant part of your own approved content is found. This is where answer quality is largely decided: a correct answer assembled from the wrong source is still wrong.
- A reply is composedIn the language of the question, in the configured tone, from the retrieved material and nothing else. Where the material is silent, the reply says so rather than filling the gap.
- It is checkedBefore sending, the reply is examined for claims the retrieved material does not support and for personal data that should not be repeated back. This runs on every reply.
- It is deliveredAs streamed text, as speech, or as a structured response — and the conversation is recorded so that what happened can be reviewed later.
Why this is built rather than bought in
Every capability described here could, in principle, be obtained by calling somebody else's service. That is how most assistants in this market are assembled, and it is why so many of them cannot answer the question that decides regulated procurements: where does the data go.
When a component is a call to an external provider, every question that passes through it leaves the organisation. That may be perfectly acceptable for a retailer answering questions about opening hours. It is usually not acceptable for a bank explaining a transaction, a hospital confirming an appointment, or a public body handling a citizen case — and the constraint is architectural, so it cannot be configured away later.
The second consequence is control over behaviour. A capability obtained as a service behaves the way the provider decides, changes when the provider changes it, and cannot be tuned for the specific way your customers speak or write. A capability that runs inside your own estate can be adjusted for dialect, for your terminology, and for the particular failure modes your material produces.
The third is continuity. External services are deprecated, repriced and withdrawn. A deployment whose core capability depends on somebody else's roadmap inherits that roadmap, including the parts nobody announced in advance.
Measuring this component honestly
| Measured in which language? | Nearly every published benchmark in this field was developed against English. An overall figure that averages across languages hides exactly the gap that matters in this market. |
|---|---|
| Measured on what input? | Clean or realistic. Read speech or spontaneous speech. Typed text or phone-typed text with autocorrect. The difference between these is usually larger than the difference between vendors. |
| Measured by whom? | Self-reported figures with an undescribed method are marketing. Ask for the test set and the procedure. |
| What counts as correct? | A surprising number of published figures count a partially correct answer as correct. Ask what the scoring rule was. |
| How does it fail? | A component that degrades gracefully is worth more than one with a higher headline figure that fails silently and confidently. |
| Does it report uncertainty? | A component that surfaces low confidence lets the system confirm rather than guess. One that hides it produces confident errors downstream. |
What to ask about this specifically
Ask for the Arabic figure
Not the overall figure. Nearly every published benchmark in this field was developed against English, and the gap between the two languages is where deployments disappoint.
Ask how it was measured
Against what test set, by whom, on what kind of input. A number without a described method is marketing rather than measurement.
Ask what it does when uncertain
Whether uncertainty is surfaced or hidden. A component that hides low confidence produces confident errors downstream.
Ask about real conditions
Noisy audio, poor scans, dialect, informal phrasing, mixed languages. Laboratory conditions are not predictive of contact-centre reality.
Ask where processing happens
For any component touching personal data, this is a procurement question rather than a technical curiosity in regulated sectors.
Ask what is retained
What data this component keeps, for how long, and whether it is used to improve anything.
Common questions
Escalating early and prioritising a queue. Those are operational uses with clear value. Fine-grained analytical claims about individual conversations are where it is over-sold.
It has to be built for Arabic rather than transferred from English. Politeness conventions differ enough that an English-tuned model reads courteous complaints as satisfaction.
It should not be used that way. Judging individuals on an automated tone score is unreliable and, in many contexts, a fairness problem.
Accurate enough to be a useful signal, not accurate enough to be a measurement. Anyone quoting a precise figure for sentiment accuracy is over-claiming.
Language indicating distress or emergency is a related and more important signal, and in sectors like healthcare it should be configured explicitly rather than left to general sentiment.
That is a decision for you, and one worth thinking about carefully. Aggregate trends are defensible; individual scoring on tone is not.