The Best Arabic LLM in 2026: Why the Leaderboard Will Mislead You, and the One-Day Evaluation That Will Not
The ranking depends on properties of your data that no leaderboard measures. The four reasons Arabic is genuinely harder, the tokenisation cost nobody quotes, the normalisation that beats a model upgrade, and an evaluation you can run this week.
“Which is the best Arabic LLM?” is asked constantly and answered badly, because the honest answer is that it depends on a property of your data that no public leaderboard measures. A model that tops an Arabic benchmark can still be the wrong choice for a Saudi insurer whose customers write in Najdi dialect mixed with English product names.
This article does two things. It describes the Arabic model landscape as of 2026 — ALLaM, Jais, and the multilingual frontier models — without pretending to a benchmark we have not run. Then it gives you an evaluation you can run yourself in an afternoon, on your own data, which is the only comparison that will predict your outcome.
Why Arabic is genuinely harder, in four specific ways
Arabic is not merely “another language” to a model. Four structural properties make it measurably harder than the languages most models were optimised for, and each one shows up as a distinct failure mode.
| Property | What it means | How it fails in production |
|---|---|---|
| Rich morphology | One root generates dozens of surface forms through prefixes, infixes and suffixes | Retrieval misses documents that use a different form of the same word |
| Optional diacritics | Short vowels are usually omitted, so one written form has several readings | The model resolves an ambiguity confidently and silently, in the wrong direction |
| Diglossia | Formal written Arabic and spoken dialect differ substantially | Trained on formal text, the model handles the way customers actually write badly |
| Orthographic variation | أ إ آ ا written interchangeably; ة and ه; ى and ي | Exact-match search fails on text a human would call identical |
None of these is exotic. All four appear in the first hundred real customer messages.
Tokenisation: the cost multiplier nobody quotes you
Models process text as tokens, and tokenisers built primarily on English text fragment Arabic far more aggressively. The same sentence in Arabic frequently costs substantially more tokens than its English translation. Three consequences follow, and all three are financial.
Directional. The ratio varies by tokeniser and by text; models with Arabic-aware tokenisers narrow all four gaps. Measure it on your own text rather than trusting a general figure.
This is the strongest practical argument for an Arabic-first model that is rarely made in Arabic-first marketing. A model with a tokeniser that handles Arabic efficiently is cheaper per answer and fits more retrieved context into the same window — before any question of output quality arises.
Illustrative of the pattern we consistently see: normalisation moves retrieval further than a model change does. Your numbers will differ — the point is that none of these steps required a different model.
The landscape in 2026
Three families are worth knowing about. We describe their published positioning; we have not run a head-to-head benchmark and do not publish comparative claims we have not measured.
| Origin | Positioning | Deployment | |
|---|---|---|---|
| ALLaM | Saudi Arabia, under the national AI programme | Arabic as a first-class training language, not a translated add-on | Distributed through platforms including Microsoft Azure AI and IBM watsonx |
| Jais | UAE — Inception / G42 with partners | An Arabic-centric family released in several sizes | Open weights for several variants, plus hosted access |
| Multilingual frontier models | US, Europe, China | Very strong general capability with Arabic among many languages | Mostly API; some open-weight families can be self-hosted |
Positioning as published by the respective programmes. Treat all of it as a starting point for your own testing, not a result.
The distribution column matters more than buyers expect. An open-weight model can be run on your own hardware with no outbound call. A model available only through a hosted API cannot, whatever its nationality. If in-Kingdom residency is a requirement, this column filters your options before quality does.
Why public benchmarks will mislead you
Benchmark scores are useful for model researchers and close to useless for a procurement decision, for three reasons that compound.
- 1The distribution is wrongBenchmarks use curated formal Arabic. Your customers write dialect, transliteration, and Arabic mixed with English product names, in the same sentence.
- 2Contamination is unmeasurablePopular benchmarks leak into training corpora over time. A rising score can reflect exposure rather than capability, and you cannot tell from outside.
- 3The wrong component is being measuredA benchmark tests the model. In a retrieval assistant, most failures are in retrieval, prompt construction and escalation — none of which the benchmark touches.
An evaluation you can actually run this week
This is deliberately unglamorous and takes about a day. It will tell you more than a month of vendor demos, because a demo is a curated question and this is not.
- Step 1Take 200 real messagesFrom your own logs, unedited — including the misspelt, the dialectal, the half-English and the rude. Do not clean them.
- Step 2Write the right answer for eachBy a person who actually knows. This is the slow part and it is the part that creates the value; the set is reusable forever.
- Step 3Run each candidate end to endNot the raw model — the whole assembly, with your retrieval and your prompt, exactly as it would run.
- Step 4Score three things separatelyDid it retrieve the right source? Was the answer correct? Was the Arabic natural? A system can pass one and fail the others.
- Step 5Count the confident wrong answersSeparately and prominently. One confident wrong answer about a policy costs more than twenty honest “I don’t know”s.
- Step 6Re-run on every changeModel swap, prompt edit, new documents. Without this you are guessing about regressions.
Relative impact as we have observed it. Directional, not a controlled study — but the ordering has been consistent, and the top item is the one teams reach for last.
Arabic normalisation: the unglamorous fix that beats a model upgrade
Before changing model, normalise. Most Arabic retrieval failures we see are solved by text processing that takes a day to implement and costs nothing to run.
| Step | What it does | Why it matters |
|---|---|---|
| Alef unification | أ إ آ ا treated as one | Users type all four for the same word |
| Ta marbuta and ha | ة and ه treated as one | A near-universal variation in informal writing |
| Alef maqsura and ya | ى and ي treated as one | Same, and it silently breaks exact match |
| Diacritic stripping | Remove tashkeel before indexing | Present in source documents, absent in user questions |
| Tatweel removal | Strip the kashida elongation character | Common in decorative headings inside PDFs |
| Digit folding | Eastern and Western numerals treated alike | ٢٠٢٦ and 2026 must match |
Apply the same normalisation to the query and to the index. Applying it to one side only makes retrieval worse, not better.
Dialect: the gap between the demo and the inbox
Vendors demonstrate in Modern Standard Arabic because it is what models handle best and what a room of executives finds legible. Customers do not write that way. A Saudi retail inbox contains Najdi and Hijazi forms, Gulf vocabulary, English product names in Latin script, Arabizi transliteration, and voice-note transcriptions with their own error profile.
| Input type | Typical demo coverage | What it needs |
|---|---|---|
| Modern Standard Arabic | Fully covered | Nothing extra |
| Gulf and Najdi dialect | Rarely shown | Dialect examples in your evaluation set |
| Arabic mixed with English terms | Rarely shown | Product-name handling in retrieval, not in the model |
| Arabizi — Arabic in Latin letters | Almost never shown | An explicit transliteration step before retrieval |
| Voice-note transcripts | Almost never shown | Tolerance for transcription errors in the matching stage |
Ask any vendor to run their demo on the last fifty messages from your own inbox instead of their script. The result is the product.
So which one should you pick
A defensible decision procedure, in order. Residency first, because it eliminates options regardless of quality. Then tokenisation, because it sets your cost floor. Then your own evaluation. Public benchmarks come last, as a tiebreaker between candidates that already passed the first three.
For a Saudi organisation handling personal data, that ordering frequently lands on an open-weight model — Arabic-capable, self-hosted, with normalisation and retrieval done properly — rather than on whichever model currently leads a leaderboard. Not because the leading model is worse, but because it cannot be deployed where the data has to stay.
Honest limits
We have not run a comparative benchmark of ALLaM against Jais or against frontier multilingual models, and we do not publish comparison tables we have not measured. Everything above describing a model family is its published positioning.
The impact figures are our own observations across deployments we have worked on. That is a small, self-selecting sample. We report the ordering because it has been consistent, not because it is a controlled result.
What we can state precisely: Elbi is model-agnostic and runs Arabic-capable open-weight models on hardware you control inside the Kingdom, with Arabic normalisation applied to both the index and the query, and no external model call in the answer path. If you run the evaluation above and a different model wins on your data, that is a configuration change, not a migration.
Common questions
There is no single answer, because the ranking depends on properties of your own data that public leaderboards do not measure. ALLaM is Saudi Arabia’s Arabic-first model developed under the national AI programme; Jais is an Arabic-centric family from the UAE with open weights for several variants; and multilingual frontier models are strong generally with Arabic among many languages. The decision procedure that works is: filter by deployment and residency first, then by Arabic tokenisation efficiency, then by an evaluation on two hundred of your own real customer messages.
Four structural properties. Rich morphology means one root generates dozens of surface forms, so retrieval misses documents using a different form. Diacritics are usually omitted, so one written form has several readings the model resolves silently. Diglossia means formal written Arabic differs substantially from how customers actually write. And orthographic variation — أ إ آ ا, ة and ه, ى and ي written interchangeably — breaks exact matching on text a human would call identical.
Usually yes, because tokenisers built primarily on English fragment Arabic more aggressively, so the same sentence costs more tokens. That raises cost per answer on per-token pricing, reduces how much retrieved context fits in the same window, and increases latency. Models with Arabic-aware tokenisers narrow all three gaps. Measure the ratio on your own text rather than relying on a general figure.
Only as a tiebreaker. Benchmarks use curated formal Arabic while your customers write dialect, transliteration and Arabic mixed with English product names. Popular benchmarks also leak into training data over time, so a rising score may reflect exposure rather than capability. Most importantly, a benchmark tests the model, while in a retrieval assistant most failures occur in retrieval, prompt construction and escalation.
In our experience, fixing retrieval — chunking and Arabic normalisation — produces the largest single gain, followed by cleaning contradictory source documents, then prompt and escalation design. Switching base models and fine-tuning matter less than teams expect. Normalisation specifically means unifying alef forms, ta marbuta with ha, alef maqsura with ya, stripping diacritics and tatweel, and folding Eastern and Western digits — applied identically to both the index and the query.
Keep reading
Arabic NLP: why understanding is harder than generating
Modern models write beautiful Arabic. That is the problem. Fluent output is the easy half, and it hides the hard half so completely that a system can fail on most of your traffic while every reply looks correct.
Read Article
Arabic AI chatbots for Saudi customers
Almost every assistant sold in the Kingdom supports Arabic. Far fewer understand the Arabic customers actually type — which is not the Arabic of official documents, and is where most deployments quietly fail.
Read Article
HUMAIN, ALLaM and the sovereign AI stack
A Saudi model served over a foreign cloud is still a foreign network call. Here is what HUMAIN provides, where ALLaM actually runs, and the six questions that separate a real residency claim from a hosting address.
Read Article