activeGeneral
MMLU PRO
Metric: Accuracy (higher is better)Introduced: 2024
Enhanced MMLU with 10-option questions (vs 4 in original), harder problems filtered through expert review, and reduced shortcuts. Maintains coverage of 57 academic subjects.
Models ranked
119
tracked on this benchmark
Score band
91.8 – 20.0
best → lowest tracked
Snapshot trend
-8.50
Oct 9 → Oct 17 · 1 models
Leaderboard
Tracked models ranked by Accuracy (higher is better).
#Model variant and provenanceRelative to leaderScore
191.8
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Dec 11, 2025Confidence: Not recordedSource
291.0
Version: MMLU-Pro (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
389.6
Version: MMLU-Pro (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
489.1
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 19, 2026Confidence: Not recordedSource
588.9
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
688.6
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 19, 2026Confidence: Not recordedSource
788.5
Version: MMLU-Pro (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
888.5
Version: MMLU-Pro (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
988.4
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Jun 17, 2025Confidence: Not recordedSource
1088.1
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 29, 2026Confidence: Not recordedSource
1187.8
Version: From official HuggingFace model card (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
1287.7
Version: MMLU-Pro (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
1387.7
Version: From third-party comparison citing ByteDance official figures (accuracy%)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
1487.5
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 19, 2026Confidence: Not recordedSource
1587.5
Version: Think Max mode (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
1687.3
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 19, 2026Confidence: Not recordedSource
1787.1
Version: Thinking mode (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
1887.0
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
1987.0
Version: From ByteDance official launch materials (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
2087.0
Version: From official ERNIE 5 (accuracy%)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
2186.8
Version: From official NVIDIA HuggingFace model cardHarness: Not recordedEvaluator: Not recordedObserved: Oct 7, 2026Confidence: Not recordedSource
2286.7
Version: From official HuggingFace model card (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
2386.4
Version: MMLU-Pro EM, V4-Flash Think HighHarness: Not recordedEvaluator: Not recordedObserved: Jul 3, 2026Confidence: Not recordedSource
2486.2
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
2586.2
Version: From official HuggingFace model card (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
Showing the top 25 of 119 tracked models. Browse all models.
How to read this benchmark
This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
Related benchmarks
Last reviewed: Apr 15, 2026