activeCoding

DeepSWE 1.1

Metric: Pass@1 (higher is better)Introduced: 2026

DeepSWE 1.1 is Datacurve's June 2026 revised execution and grading setup for the same long-horizon DeepSWE tasks, grading committed patches in a clean isolated verifier environment and using mini-swe-agent for the public leaderboard. Model scores must preserve the mini-swe-agent harness/evaluator context.

Models ranked

32

tracked on this benchmark

Score band

79.0 – 42.2

best → lowest tracked

Snapshot trend

-6.05

Oct 23 → Nov 9 · 2 models

Leaderboard

Tracked models ranked by Pass@1 (higher is better).

Compare candidates
#Model variant and provenanceScore
1

Configuration: Muse Wonder Full (max)

Version: Not recordedHarness: Not recordedEvaluator: MetaObserved: Oct 12, 2026Confidence: self-reportedSource

Notes: Meta first-party benchmark table on the Muse Wonder model page; methodology in the Muse Wonder model card.

79.0
2

Configuration: Muse Wonder (max)

Version: Not recordedHarness: Not recordedEvaluator: MetaObserved: Oct 12, 2026Confidence: self-reportedSource

Notes: Meta first-party benchmark table on the Muse Wonder model page; methodology in the Muse Wonder model card.

78.8
3

Configuration: Gemini 4 Argon (highest thinking)

Version: DeepSWE v1.1Harness: mini-swe-agentEvaluator: Google DeepMindObserved: Oct 17, 2026Confidence: self-reportedSource

Notes: Blog benchmark table (gemini-4-argon_table_blog.gif); evals methodology page https://deepmind.google/models/evals-methodology/gemini-4-argon: self-computed with mini-swe-agent harness, highest thinking level. Retrieved 2026-10-17.

77.9
4
Version: DeepSWE v1.1 Pass@1; mini-swe-v2 harness; max reasoning effortHarness: Not recordedEvaluator: Not recordedObserved: Oct 17, 2026Confidence: Not recordedSource
77.0
5
Version: DeepSWE v1.1; Claude-Code harness; max reasoning effortHarness: Not recordedEvaluator: Not recordedObserved: Oct 23, 2026Confidence: Not recordedSource

Notes: Moonshot self-reported launch table, row 'DeepSWE v1.1', column 'Kimi K3.1 Pro (max)'. Settings: reasoning effort max, temperature 1.0, top-p 0.95 (blog footnote). Moonshot-run with the Claude-Code harness, not the Datacurve mini-swe-agent board.

75.5
6

Configuration: claude-fable-5-2 (mean of 5 trials)

Version: DeepSWE v1.1Harness: Anthropic system-card standard configuration (scaffold not named); average of five trialsEvaluator: AnthropicObserved: Nov 9, 2026Confidence: self-reportedSource

Notes: Anthropic self-reported (lab-published). System card, retrieved 2026-11-09. Does not move /best rankings (self-reported / raon rule). System card §8.3: DeepSWE v1.1, 113 long-horizon tasks, 74.3% averaged over five trials. Evaluator is Anthropic (lab run), not the public Datacurve/mini-swe-agent leaderboard. Do not treat as a board restamp.

74.3
7
Version: DeepSWE 1.1 Resolved; mini-SWE harness; max reasoning effortHarness: Not recordedEvaluator: Not recordedObserved: Sep 10, 2026Confidence: Not recordedSource
74.2
8

Configuration: gpt-6-astra (Extra High)

Version: DeepSWE 1.1Harness: mini-swe-agent (Datacurve DeepSWE 1.1)Evaluator: DatacurveObserved: Sep 22, 2026Confidence: confirmedSource

Notes: Datacurve DeepSWE 1.1 leaderboard (113 tasks, mini-swe-agent, Pass@1 over 4 runs). Verified on https://deepswe.datacurve.ai/ (page generated_at 2026-09-22; page shows integer %%). Precise value from Epoch AI mirror deepswe_external.csv (Source column = deepswe.datacurve.ai). Highest-scoring published effort selected.

74.1
9

Configuration: gemini-3.8-flash (High)

Version: DeepSWE 1.1Harness: mini-swe-agent (Datacurve DeepSWE 1.1)Evaluator: DatacurveObserved: Sep 22, 2026Confidence: confirmedSource

Notes: Datacurve DeepSWE 1.1 leaderboard (113 tasks, mini-swe-agent, Pass@1 over 4 runs). Verified on https://deepswe.datacurve.ai/ (page generated_at 2026-09-22; page shows integer %%). Precise value from Epoch AI mirror deepswe_external.csv (Source column = deepswe.datacurve.ai). Highest-scoring published effort selected.

73.8
10

Configuration: MiniMax M3.1 Flash

Version: DeepSWE v1.1Harness: mini-SWE-agent 2.4.5 (512K context)Evaluator: MiniMaxObserved: Oct 19, 2026Confidence: self-reportedSource

Notes: MiniMax M3.1 Flash launch blog (2026-10-19) main benchmark table image m31-flash-en-2 (webp converted to png; values transcribed by eye from table) incl. footnotes. Footnote: mini-SWE-agent 2.4.5, max context 512K. Blog text also says 73.7%. Not on Datacurve DeepSWE board as of retrieval. Retrieved 2026-10-19.

73.7
11
Version: DeepSWE 1.1; OpenAI GA launch tableHarness: Not recordedEvaluator: Not recordedObserved: Jul 9, 2026Confidence: Not recordedSource

Notes: Official GPT-5.6 GA launch benchmark row.

72.7
12
Version: DeepSWE v1.1Harness: Not recordedEvaluator: Not recordedObserved: Oct 9, 2026Confidence: Not recordedSource
71.9
13

Configuration: Maximum effort

Version: DeepSWE v1.1Harness: 113 long-horizon agentic software-engineering tasks; five-trial averageEvaluator: AnthropicObserved: Jul 24, 2026Confidence: confirmedSource

Notes: Vendor-reported. Source: pp.149-150, Table 8.1.A and Figure 8.2. The effort sweep ranges from 57.7 at low to 68.8 at maximum.

68.8
14

Configuration: kimi-k3 (Max)

Version: DeepSWE 1.1Harness: mini-swe-agent (Datacurve DeepSWE 1.1)Evaluator: DatacurveObserved: Sep 22, 2026Confidence: confirmedSource

Notes: Datacurve DeepSWE 1.1 leaderboard (113 tasks, mini-swe-agent, Pass@1 over 4 runs). Verified on https://deepswe.datacurve.ai/ (page generated_at 2026-09-22; page shows integer %%). Precise value from Epoch AI mirror deepswe_external.csv (Source column = deepswe.datacurve.ai). Highest-scoring published effort selected.

68.5
15
Version: DeepSWE v1.1Harness: Not recordedEvaluator: Not recordedObserved: Oct 9, 2026Confidence: Not recordedSource
67.9
16
Version: DeepSWE v1.1Harness: Not recordedEvaluator: Not recordedObserved: Aug 14, 2026Confidence: Not recordedSource
66.9
17
Version: DeepSWE v1.1, Grok 4.6 High (xAI first-party evals table)Harness: Not recordedEvaluator: Not recordedObserved: Aug 12, 2026Confidence: Not recordedSource

Notes: xAI launch post evals table reports Grok 4.6 High at 65.9% on DeepSWE v1.1. Variant: Grok 4.6 High (same model ID grok-4.6; high is the API default reasoning_effort).

65.9
18

Configuration: Nex-N2.5-Max

Version: DeepSWE v1.1Harness: NexAU harnessEvaluator: Nex-AGIObserved: Oct 19, 2026Confidence: self-reportedSource

Notes: Nex-AGI HF model card benchmark table (column 3 = Nex-N2.5-Max). Sampling temperature 0.7, top_p 0.95, top_k 40 (note 2). Note 3: NexAU harness. Not the official mini-swe-agent setup; comparability caveat (10-17 Nex-N2.5-Pro precedent). Retrieved 2026-10-19.

65.6
19

Configuration: high thinking

Version: DeepSWE v1.1Harness: Not recordedEvaluator: Not recordedObserved: Aug 13, 2026Confidence: Not recordedSource
65.3
20

Configuration: LongCat-2.5 (thinking on)

Version: DeepSWE v1.1, average of 3 runsHarness: mini-swe-agent, temperature 1.0, top_p 0.95, max_seq_length 1MEvaluator: MeituanObserved: Nov 9, 2026Confidence: self-reportedSource

Notes: Self-reported by Meituan. Meituan LongCat first-party HF model card meituan-longcat/LongCat-2.5 (README sha 6fc4844c, lastModified 2026-11-09), benchmark table column 'LongCat-2.5'. Card default: enable_thinking=on, temperature 1.0, top_p 0.95 unless a footnote says otherwise. Footnote 2: 'DeepSWE v1.1: mini-swe-agent at temperature = 1.0, top_p = 0.95 and max_seq_length = 1M; results are averaged over three independent runs.' Row 'DeepSWE (v1.1)': 64.6. Not on Datacurve's board (deepswe.datacurve.ai leaderboard-live.json checked 2026-11-09). Retrieved 2026-11-09. Card caveat (verbatim): 'unmarked scores are measured in-house and are indicative only.' This LongCat-2.5 value is unmarked (no '*').

64.6
21
Version: DeepSWE v1.1Harness: Not recordedEvaluator: Not recordedObserved: Aug 26, 2026Confidence: Not recordedSource
63.4
22
Version: DeepSWE 1.1, preferred harness; xhigh reasoning effortHarness: Not recordedEvaluator: Not recordedObserved: Oct 15, 2026Confidence: Not recordedSource

Notes: Meta HF model card Benchmarks table, xhigh reasoning effort.

60.0
23
Version: DeepSWE 1.1Harness: Not recordedEvaluator: Not recordedObserved: Aug 26, 2026Confidence: Not recordedSource
58.7
24

Configuration: Qwen3.8-Flash

Version: DeepSWE 1.1 (best of Claude Code / mini-SWE-agent; Qwen3.8-Flash best on mini-SWE-agent)Harness: mini-SWE-agent (temperature 1.0, top_p 0.95, 256K context)Evaluator: Qwen (Alibaba)Observed: Oct 26, 2026Confidence: self-reportedSource

Notes: Qwen Team tech report 'Qwen3.8-Omni: Towards Native Omni-Modal Agents' (arXiv 2609.25611v1, 2026-09-22), Table 2 column 'Qwen3.8-Flash', row 'DeepSWE 1.1': 58.7. Footnote 1: highest of Claude Code and mini-SWE-agent runs; 'notably, Qwen3.8-Flash performs best on mini-SWE-agent', so this value is the mini-SWE-agent run. QwenCloud docs: 'DeepSWE 1.1 at 58.7'. Identity: the report's 'Qwen3.8-Flash' column and the QwenCloud qwen3.8-flash docs page carry the same values as the Qwen3.8-Flash-Next blog/HF card; Qwen's blog says Qwen3.8-Flash is the production served version (1M context, built-in tools) while these evals used a 256K context. Retrieved 2026-10-26.

58.7
25
Version: DeepSWE 1.1Harness: Not recordedEvaluator: Not recordedObserved: Aug 12, 2026Confidence: Not recordedSource
56.6
Showing the top 25 of 32 tracked models. Browse all models.

How to read this benchmark

This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.

Trust this score when

  • There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
  • The model list covers the same version family you can actually deploy today.
  • Top candidates overlap with your required routing and feature requirements.

Be cautious when

  • There is only one benchmark snapshot or the dataset appears stale.
  • The benchmark metric direction is opposite of your decision objective.
  • The score difference between options is narrow and likely within implementation variance.

Related benchmarks

Last reviewed: Jul 8, 2026