activeCoding
HumanEval
Metric: Pass@1 (higher is better)Introduced: 2021
164 Python coding problems measuring functional correctness of code generation via pass@k metric. Released by OpenAI in 2021; HumanEval+ provides a more rigorous extension.
Models ranked
97
tracked on this benchmark
Score band
98.0 – 28.1
best → lowest tracked
Snapshot trend
-19.12
Jun 7 → Jul 3 · 1 models
Leaderboard
Tracked models ranked by Pass@1 (higher is better).
#Model variant and provenanceRelative to leaderScore
1
Claude Sonnet 4.6
98.0Version: HumanEval (pass@1)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
2
o3
96.7Version: 2025-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 9, 2026Confidence: Not recordedSource
3
Claude Opus 4.6
95.0Version: HumanEval (pass@1)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
4
Grok-3
94.5Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Feb 17, 2025Confidence: Not recordedSource
5
GPT-5.5
94.2Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 29, 2026Confidence: Not recordedSource
6
Gemini 3.1 Pro Preview
94.0Version: HumanEval (pass@1)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
7
Gemini 2.5 Pro
93.1Version: 2025-03Harness: Not recordedEvaluator: Not recordedObserved: Apr 9, 2026Confidence: Not recordedSource
8
Claude 3.7 Sonnet
93.0Version: 2025-02Harness: Not recordedEvaluator: Not recordedObserved: Apr 9, 2026Confidence: Not recordedSource
9
GPT-4.1
92.9Version: 2025-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 9, 2026Confidence: Not recordedSource
10
Qwen2.5-Coder-32B-Instruct
92.7Version: 2024-11Harness: Not recordedEvaluator: Not recordedObserved: Apr 9, 2026Confidence: Not recordedSource
11
Qwen3-235B-A22B
92.7Version: 2025-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 9, 2026Confidence: Not recordedSource
12
Claude 3.5 Sonnet
92.0Version: pass@1Harness: Not recordedEvaluator: Not recordedObserved: Mar 6, 2026Confidence: Not recordedSource
13
Kimi K2.6
92.0Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 29, 2026Confidence: Not recordedSource
14
Mistral Large 3 675B Instruct
92.0Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Dec 1, 2025Confidence: Not recordedSource
15
Gemini 3.5 Flash
92.0Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: May 25, 2026Confidence: Not recordedSource
16
OLMo 3 32B Think
91.4Version: HumanEval (pass@1)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
17
GPT-4o (05-13)
90.2Version: pass@1Harness: Not recordedEvaluator: Not recordedObserved: Mar 6, 2026Confidence: Not recordedSource
18
Gemini 2.5 Flash
90.1Version: 2025-05Harness: Not recordedEvaluator: Not recordedObserved: Apr 9, 2026Confidence: Not recordedSource
19
DeepSeek R1
89.9Version: 2025-01Harness: Not recordedEvaluator: Not recordedObserved: Apr 9, 2026Confidence: Not recordedSource
20
Granite 4.1 30B
89.6Version: pass@1, instruct model (pass@1)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
21
Llama 3.1 405B
89.0Version: pass@1Harness: Not recordedEvaluator: Not recordedObserved: Mar 6, 2026Confidence: Not recordedSource
22
Grok-2
88.4Version: pass@1Harness: Not recordedEvaluator: Not recordedObserved: Mar 6, 2026Confidence: Not recordedSource
23
Qwen2.5-32B-Instruct
88.4Version: pass@1Harness: Not recordedEvaluator: Not recordedObserved: Mar 6, 2026Confidence: Not recordedSource
24
Granite 4.1 8B
87.2Version: pass@1, instruct model (pass@1)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
25
OLMo 3.1 32B Instruct
86.7Version: HumanEval (pass@1)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
Showing the top 25 of 97 tracked models. Browse all models.
How to read this benchmark
This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
Related benchmarks
Last reviewed: Apr 15, 2026