activeCoding

SWE-bench Multilingual

Metric: % Resolved (higher is better)Introduced: 2026

Multilingual SWE-bench suite evaluating software engineering issue resolution across repositories and languages beyond the original Python-heavy SWE-bench tasks.

Models ranked

8

tracked on this benchmark

Score band

89.5 – 73.3

best → lowest tracked

Snapshot trend

+11.69

Jun 30 → Jul 24 · 1 models

Leaderboard

Tracked models ranked by % Resolved (higher is better).

Compare candidates
#Model variant and provenanceScore
1
Claude Opus 5

Configuration: Claude Opus 5 standard configuration

Version: SWE-bench MultilingualHarness: Anthropic system-card standard configuration; average of five trialsEvaluator: AnthropicObserved: Jul 24, 2026Confidence: confirmedSource

Notes: Vendor-reported. Source: p.149, Table 8.1.A. This multilingual suite is not the English SWE-bench Verified row.

89.5
2
Composer 2.5
Version: SWE-bench MultilingualHarness: Not recordedEvaluator: Not recordedObserved: May 21, 2026Confidence: Not recordedSource 1Source 2

Notes: Confidence: medium. DAT-4756 secondary-source score for Cursor Composer 2.5; compare directionally because Composer is a productized IDE agent.

79.8
3
Claude Sonnet 5
Version: SWE-bench Multilingual; Anthropic standard configuration, adaptive thinking at max effort, default sampling, average over 5 trialsHarness: Not recordedEvaluator: Not recordedObserved: Jun 30, 2026Confidence: Not recordedSource
78.3
4
LongCat-2.0
Version: SWE-bench Multilingual; Code Agent chart; LongCat launch benchmarkHarness: Not recordedEvaluator: Not recordedObserved: Jun 30, 2026Confidence: Not recordedSource

Notes: Official LongCat-2.0 launch chart reports 77.33. Evaluator/harness details are not fully reproducible from public text. Confidence: confirmed for reported value, medium for reproducibility.

77.3
5
Kimi K2.6
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 20, 2026Confidence: Not recordedSource
76.7
6
DeepSeek V4 Pro
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource
76.2
7
Claude Sonnet 4.6
Version: SWE-bench MultilingualHarness: Not recordedEvaluator: Not recordedObserved: May 21, 2026Confidence: Not recordedSource

Notes: Confidence: medium. DAT-4756 comparison research; preserve harness caveat when comparing against Composer 2.5.

75.9
8
DeepSeek V4 Flash
Version: SWE-bench Multilingual resolved, V4-Flash Think MaxHarness: Not recordedEvaluator: Not recordedObserved: Jul 3, 2026Confidence: Not recordedSource
73.3

How to read this benchmark

This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.

Trust this score when

  • There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
  • The model list covers the same version family you can actually deploy today.
  • Top candidates overlap with your required routing and feature requirements.

Be cautious when

  • There is only one benchmark snapshot or the dataset appears stale.
  • The benchmark metric direction is opposite of your decision objective.
  • The score difference between options is narrow and likely within implementation variance.

Related benchmarks

HumanEval
MBPP
Mostly Basic Programming Problems
HumanEval+
MBPP+
Mostly Basic Programming Problems+
Spider
ToolTalk

Last reviewed: May 21, 2026

Resources

WebsiteGitHubarXiv Paper