activeReasoning

Humanity's Last Exam

Metric: Score (higher is better)

Multi-domain expert-level exam covering science, math, humanities, and more

Models ranked

47

tracked on this benchmark

Score band

64.5 – 6.5

best → lowest tracked

Snapshot trend

-1.48

Oct 17 → Oct 26 · 4 models

Leaderboard

Tracked models ranked by Score (higher is better).

Compare candidates
#Model variant and provenanceScore
1
Version: HLE (with tools)Harness: Not recordedEvaluator: Not recordedObserved: Jun 9, 2026Confidence: Not recordedSource

Notes: Official Anthropic launch table starred the 64.5% HLE with-tools score as Claude Mythos 5. The 59.0% no-tools variant is recorded but not seeded because modelBenchmark rows are unique by modelSlug and benchmarkSlug.

64.5
2
Version: HLE with toolsHarness: Not recordedEvaluator: Not recordedObserved: Aug 26, 2026Confidence: Not recordedSource
55.3
3
Version: HLE with tools (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
54.7
4
Version: HLE with tools enabled (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
53.0
5
Version: HLE with tools (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
51.4
6
Version: HLE-Full without tools; max reasoning effortHarness: Not recordedEvaluator: Not recordedObserved: Oct 23, 2026Confidence: Not recordedSource

Notes: Moonshot self-reported launch table, row 'HLE-Full', column 'Kimi K3.1 Pro (max)'. Settings: reasoning effort max, temperature 1.0, top-p 0.95 (blog footnote). Full 2,500-question set, no tools.

51.1
7
Version: HLE-Full with tools (agentic) (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
50.2
8
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: May 29, 2026Confidence: Not recordedSource
47.2
9
Version: HLE-Full without tools; max reasoning effortHarness: Not recordedEvaluator: Not recordedObserved: Jul 16, 2026Confidence: Not recordedSource

Notes: Model variant: Kimi K3 (max reasoning effort). Benchmark variant: HLE-Full without tools. Harness/evaluator not disclosed by Moonshot. Source posture: Moonshot/Kimi self-reported launch table; confidence: medium because the value is vendor-reported and not independently reproduced. Recommended seed value: 43.5.

43.5
10
Version: Humanity's Last Exam; no tools; reasoning-only; thinking auto; 1M total-token cap; Claude Opus 4.6 graderHarness: Not recordedEvaluator: Not recordedObserved: Jun 30, 2026Confidence: Not recordedSource
43.2
11

Configuration: DeepSeek-V4-Pro-0813 (max effort)

Version: HLE w/o tools; max reasoning effortHarness: No toolsEvaluator: DeepSeekObserved: Oct 26, 2026Confidence: self-reportedSource

Notes: DeepSeek first-party HF card DeepSeek-V4-Pro-0813, benchmark table (column 1 = DeepSeek-V4-Pro-0813). Note 1: code-agent tasks run with DeepSeek Harness minimal mode, max effort, temperature 1.0, top_p 0.95. Row 'HLE (wo / w tools)': 42.7 / 60.0. Filed under hle (not hle-no-tools) to match the DeepSeek-family precedent (deepseek-v4.1-flash, -1017 rows). Retrieved 2026-10-26.

42.7
12
Version: HLE w/o tools; full set (text-only subset 44.0)Harness: Not recordedEvaluator: Not recordedObserved: Oct 17, 2026Confidence: Not recordedSource
42.6
13
Version: HLE (xhigh setting) (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
41.6
14
Version: HLE no tools (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
41.4
15
Version: HLE without tools (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
41.4
16
Version: HLE (accuracy, text-only assumed)Harness: Not recordedEvaluator: Not recordedObserved: Jun 13, 2026Confidence: Not recordedSource
40.5
17
Version: HLE (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
40.2
18
Version: Humanity's Last Exam (accuracy%)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
39.0
19

Configuration: DeepSeek-V4-Flash-0731 (max effort)

Version: HLE w/o tools; max reasoning effortHarness: No toolsEvaluator: DeepSeekObserved: Oct 26, 2026Confidence: self-reportedSource

Notes: DeepSeek first-party HF card DeepSeek-V4-Pro-0813, benchmark table column 2 = DeepSeek-V4-Flash-0731, row 'HLE (wo / w tools)': 37.8 / 51.5. The Flash-0731 card itself has no HLE row. Filed under hle per DeepSeek-family precedent. Retrieved 2026-10-26.

37.8
20
Version: HLE (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
37.7
21
Version: HLE (accuracy)Harness: Not recordedEvaluator: Not recordedObserved: Oct 9, 2026Confidence: Not recordedSource
37.0
22
Version: HLE Pass@1; full set; no tools; max reasoning effortHarness: Not recordedEvaluator: Not recordedObserved: Oct 17, 2026Confidence: Not recordedSource

Notes: Full HLE set, no tools. DeepSeek also reports 39.1 on the HLE text-only subset (base HF card † footnote; changelog 2026-09-10). No separate score row for the text-only subset: the benchmark catalog has hle, hle-no-tools, hle-with-tools, and hle-with-tools-pro only. Rows are unique by modelSlug and benchmarkSlug. Relabeled 2026-10-17.

36.8
23

Configuration: tool setting not stated; GPT-4o judge

Version: HLE (tool setting not stated; GPT-4o judge)Harness: Not recordedEvaluator: Qwen (Alibaba)Observed: Oct 26, 2026Confidence: self-reportedSource

Notes: Qwen Team tech report 'Qwen3.8-Omni: Towards Native Omni-Modal Agents' (arXiv 2609.25611v1, 2026-09-22), Table 2 column 'Qwen3.8-Omni-Flash', row 'HLE': 36.5. Tool setting not stated, so filed under generic hle. Retrieved 2026-10-26.

36.5
24

Configuration: tool setting not stated; GPT-4o judge

Version: HLE (tool setting not stated; GPT-4o judge)Harness: Not recordedEvaluator: Qwen (Alibaba)Observed: Oct 26, 2026Confidence: self-reportedSource

Notes: Qwen Team tech report 'Qwen3.8-Omni: Towards Native Omni-Modal Agents' (arXiv 2609.25611v1, 2026-09-22), Table 2 column 'Qwen3.8-Flash', row 'HLE': 35.9. Tool setting not stated, so filed under generic hle. Identity: the report's 'Qwen3.8-Flash' column and the QwenCloud qwen3.8-flash docs page carry the same values as the Qwen3.8-Flash-Next blog/HF card; Qwen's blog says Qwen3.8-Flash is the production served version (1M context, built-in tools) while these evals used a 256K context. Retrieved 2026-10-26.

35.9
25
Version: Humanity's Last Exam pass@1, no tools, V4-Flash Think MaxHarness: Not recordedEvaluator: Not recordedObserved: Jul 3, 2026Confidence: Not recordedSource
34.8
Showing the top 25 of 47 tracked models. Browse all models.

How to read this benchmark

This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.

Trust this score when

  • There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
  • The model list covers the same version family you can actually deploy today.
  • Top candidates overlap with your required routing and feature requirements.

Be cautious when

  • There is only one benchmark snapshot or the dataset appears stale.
  • The benchmark metric direction is opposite of your decision objective.
  • The score difference between options is narrow and likely within implementation variance.

Related benchmarks

Last reviewed: May 20, 2026

Resources