activeCodingAgents

SWE-bench Verified

Metric: % ResolvedIntroduced: 2024

500 human-validated GitHub issue resolution tasks from SWE-bench, created with OpenAI in August 2024. The standard evaluation for agentic coding systems. Top performers (2026) exceed 78% resolved.

Models ranked

104

tracked on this benchmark

Score band

96.0 – 23.6

best → lowest tracked

Snapshot trend

-10.25

Oct 21 → Nov 6 · 2 models

Leaderboard

Tracked models ranked by % Resolved (higher is better).

Compare candidates
#Model variant and provenanceScore
1
Version: Vals.ai harness (weighted avg by difficulty tier)Harness: Not recordedEvaluator: Not recordedObserved: Jun 9, 2026Confidence: Not recordedSource

Notes: Independent Vals.ai SWE-bench Verified result; keep separate from Anthropic's SWE-bench Pro launch row.

96.0
2

Configuration: Adaptive thinking at maximum effort

Version: Verified, 500-problem solvable subsetHarness: Anthropic system-card standard configuration; average of five trialsEvaluator: AnthropicObserved: Jul 24, 2026Confidence: confirmedSource

Notes: Vendor-reported. Source: p.149, Table 8.1.A. Preserve the five-trial setting; do not treat single-trial rows as equivalent.

96.0
3
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: May 1, 2026Confidence: Not recordedSource
93.9
4
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: May 28, 2026Confidence: Not recordedSource

Notes: Official Anthropic launch benchmark.

88.6
5
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource
87.6
6
Version: SWE-bench Verified; Anthropic standard configuration, adaptive thinking at max effort, default sampling, average over 5 trialsHarness: Not recordedEvaluator: Not recordedObserved: Jun 30, 2026Confidence: Not recordedSource
85.2
7
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource
85.0
8

Configuration: Ornith-1.5-122B

Version: SWE-bench Verified (non-hack) (self-reported, official model card)Harness: OpenHands, temperature 1.0, top_p 0.95, 256K context; reward-hacking solutions marked incorrectEvaluator: OrnithObserved: Nov 6, 2026Confidence: self-reportedSource

Notes: Self-reported by Ornith. Ornith first-party HF model card Ornith-1.5-122B, benchmark table (column 'Ornith-1.5-122B'). Retrieved 2026-11-06. SWE-bench Verified (non-hack): 84.5.

84.5
9
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Apr 30, 2026Confidence: Not recordedSource

Notes: source reconciliation: vals.ai standardized harness reports GPT-5.5 at 82.6; do not use the non-comparable 88.7 self-reported row for /best/coding.

82.6
10
Version: Vals.ai independent SWE-bench Verified; standard GPT-5.5 weightsHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource

Notes: Medium confidence: GPT-5.5 Pro uses the same underlying weights as standard GPT-5.5; no Pro-specific SWE-bench Verified score was found.

82.6
11
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Oct 16, 2026Confidence: Not recordedSource

Notes: Reflection first-party final model card (HF https://huggingface.co/reflection/Beam-1.0-501B-A23B, BF16 column), retrieved 2026-10-16. Replaces 2026-10-05 preview-blog score 80.9. Card label 'SWEBench Verified'. Lab self-reported.

81.6
12
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Nov 1, 2025Confidence: Not recordedSource

Notes: Confidence: medium. May 12 /best/ refresh; benchlm.ai May 11 snapshot showed Claude Opus 4.5 at 80.9%.

80.9
13
Version: 2026-02Harness: Not recordedEvaluator: Not recordedObserved: Apr 9, 2026Confidence: Not recordedSource
80.8
14
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource
80.6
15
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource
80.6
16
Version: Rank 8 of 49 (resolved%)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
80.5
17
Version: Rank 9 of 97 (resolved%)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
80.4
18
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource
80.2
19
Version: M2 (resolved%)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
80.2
20
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource

Notes: Verified via third-party source; official OpenAI benchmark may differ. Use directionally.

80.0
21
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Feb 17, 2026Confidence: Not recordedSource

Notes: Confidence: high. Anthropic self-reported this score and identified Composer 2.5's prior 80.8 SWE-bench Verified row as a Claude Opus 4.6 misattribution.

79.6
22
Version: SWE-bench Verified resolved, V4-Flash Think MaxHarness: Not recordedEvaluator: Not recordedObserved: Jul 3, 2026Confidence: Not recordedSource
79.0
23
Version: SWE-Bench Verified (Resolved)Harness: Not recordedEvaluator: Not recordedObserved: Apr 28, 2026Confidence: Not recordedSource
78.9
24
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource
78.8
25
Version: SWE-bench VerifiedHarness: Not recordedEvaluator: Not recordedObserved: May 12, 2026Confidence: Not recordedSource

Notes: Confidence: high. May 12 /best/ refresh; benchlm.ai and llm-stats both reported Qwen3.6 Plus at 78.8%.

78.8
Showing the top 25 of 104 tracked models. Browse all models.

How to read this benchmark

This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.

Trust this score when

  • There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
  • The model list covers the same version family you can actually deploy today.
  • Top candidates overlap with your required routing and feature requirements.

Be cautious when

  • There is only one benchmark snapshot or the dataset appears stale.
  • The benchmark metric direction is opposite of your decision objective.
  • The score difference between options is narrow and likely within implementation variance.

Related benchmarks

Last reviewed: Apr 15, 2026