DeepSWE 1.1
DeepSWE 1.1 is Datacurve's June 2026 revised execution and grading setup for the same long-horizon DeepSWE tasks, grading committed patches in a clean isolated verifier environment and using mini-swe-agent for the public leaderboard. Model scores must preserve the mini-swe-agent harness/evaluator context. High benchmark score alone doesn't make a model the right pick — weigh it against pricing, API availability, and release date.
Models ranked
3
tracked on this benchmark
Score band
72.7 – 53.0
best → lowest tracked
Snapshot trend
+15.80
Jul 8 → Jul 24 · 1 models
Leaderboard
Tracked models ranked by Pass@1 (higher is better).
Notes: Official GPT-5.6 GA launch benchmark row.
Configuration: Maximum effort
Notes: Vendor-reported. Source: pp.149-150, Table 8.1.A and Figure 8.2. The effort sweep ranges from 57.7 at low to 68.8 at maximum.
Notes: xAI launch blog DeepSWE 1.1 chart reports Grok 4.5 at 53% Pass@1. Evaluator/source: Datacurve with mini-swe-agent harness per xAI first-party claim. Variant: Grok 4.5 base API model. Harness status: mini-swe-agent (DeepSWE 1.1 public leaderboard setup). Confidence: medium for vendor-reported chart value, medium for harness identifiability. Recommended seed value: 53.
How to read this benchmark
This benchmark scores models where higher is better. Scores are useful for directional filtering and shortlisting — not for universal quality ranking. Prefer benchmarks closest to your workload, then validate the linked model pages for pricing, context window, and provider availability.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
FAQ
What does the DeepSWE 1.1 benchmark measure?
DeepSWE 1.1 is Datacurve's June 2026 revised execution and grading setup for the same long-horizon DeepSWE tasks, grading committed patches in a clean isolated verifier environment and using mini-swe-agent for the public leaderboard. Model scores must preserve the mini-swe-agent harness/evaluator context. On this page it lists 3 tracked model variants where higher is better.
Is a higher DeepSWE 1.1 score always better?
For this benchmark, higher is better. A high score helps you shortlist, but confirm pricing, context window, and provider availability on each model page before committing — the top scorer is not always the right pick for your workload or budget.
How current is this DeepSWE 1.1 data?
This benchmark was last reviewed on Jul 8, 2026. The tracked score average moved +15.80 points across the last 3 snapshots.
Related benchmarks
Last reviewed: Jul 8, 2026