How is AI “intelligence” measured? Inside the Intelligence Index
The single score that ranks language models on our AI Models page — what it is made of, how it is calculated, and how to read it without over-trusting it.
Executive summary
The Intelligence Index is a single 0–100 score that lets you rank AI language models at a glance — higher means more capable. It is not one exam but a weighted average of many independent benchmarks spanning reasoning, knowledge, mathematics, coding and agentic tool-use, published and run by Artificial Analysis. For decision-makers, three things matter: it is a relative ranking aid, not an absolute measure of business value; it is text-only and English-language, run identically across every model so the comparison is fair; and it is versioned and periodically reweighted as older tests saturate — so only compare models measured under the same index version. Always read it next to price and speed: the most capable model is rarely the cheapest or fastest, and our AI Models dashboard plots exactly that trade-off.
What the Intelligence Index actually is
Every leading lab claims its model is the smartest. The Intelligence Index exists to replace marketing with a single, comparable number produced by running each model through the same battery of hard tests, under the same conditions, and combining the results. A model scoring in the high 50s today is genuinely at the frontier; one in the teens is a small or older model. The value is relative: it tells you how models stack up against each other, not whether a model will succeed at your specific task.
The benchmarks behind the number
The index is built from roughly a dozen public evaluations, each targeting a different capability. The exact set evolves, but the families are stable:
| Benchmark | What it probes | Capability area |
|---|---|---|
| MMLU-Pro | Expert knowledge across 50+ subjects (a harder MMLU) | Knowledge |
| GPQA Diamond | “Google-proof” graduate questions in biology, physics, chemistry | Scientific reasoning |
| Humanity’s Last Exam (HLE) | Frontier academic questions across many disciplines | Scientific reasoning |
| AIME | Olympiad-level competition mathematics | Mathematics |
| LiveCodeBench | Recent, contamination-resistant programming problems | Coding |
| SciCode | Writing code for real scientific-computing tasks | Coding |
| Terminal-Bench | End-to-end software & sysadmin work in a real shell | Coding / agents |
| IFBench | Following precise, multi-part instructions | General |
| AA-LCR | Reasoning over long (~100k-token) documents | General |
| τ-Banking (agentic) | Multi-step customer-support tasks using tools | Agents |
Each test is chosen because it is hard, discriminating, and resistant to being gamed — several use fresh or held-out questions specifically to limit memorisation from training data.
How a dozen tests become one number
The individual benchmarks are first normalised onto comparable scales (most are scored as pass@1 — the model must be right on its first attempt; some, like agentic tasks, are scored by Elo-style comparison and then rescaled). They are then grouped into a few capability categories — broadly reasoning & knowledge, mathematics, coding, and agentic/tool-use — and the index is the weighted average across those categories. The weighting deliberately rewards the skills that matter most for real work, and it is revised with each index version as the field moves. Because the recipe is fully published, the number is reproducible rather than a black box — the methodology page lists the current evaluations and weights.
How the tests are run
Comparability comes from running everything the same way:
- Zero-shot prompting — clear instructions, no worked examples, so the test measures the model rather than the prompt engineering.
- Fixed sampling — temperature 0 for standard models and a low fixed temperature (around 0.6) for reasoning models, per each lab’s guidance.
- Repeats and averaging — most questions are run several times (typically 3–5) and aggregated, so a lucky or unlucky single run does not distort the score.
- Automated, audited grading — multiple-choice answers are extracted with robust pattern matching; open-ended and maths answers are checked by a grader (symbolic math checks plus an “equality-checker” model) rather than brittle string matching; code is executed against test suites in a sandbox.
Reasoning models and “effort”
Modern models can “think” before answering, and many expose an effort or reasoning setting (low → high). More effort usually means higher scores — and far more output tokens, latency and cost. That is why you will see the same model family listed several times on our dashboard at different effort levels: each configuration is a genuinely different point on the capability-versus-cost curve, so we keep them distinct.
What the score does not tell you
A single number invites over-trust, so read it with these caveats:
- It is text-only and English-language. Image understanding, speech and multilingual ability are measured separately and are not in this score.
- It is not your use case. A model that tops the index can still trail a cheaper one on your documents, tone or domain. Treat it as a shortlist tool, then test.
- Benchmark contamination is real. Some datasets were curated against specific earlier models; the methodology flags where direct comparisons are discouraged.
- It moves over time. As tests saturate they are replaced and reweighted, so a score is only meaningful within its index version and date.
How to read it on this page
On the AI Models dashboard the Intelligence Index is the vertical axis of the headline chart and a sortable column in the table. The useful move is not “pick the highest number” but find the frontier: the models that deliver the most intelligence for an acceptable price and speed. The data refreshes daily, and full credit for the underlying measurements goes to Artificial Analysis.