activeCoding

SWE-bench Multilingual

Metric: % Resolved (higher is better)Introduced: 2026

Multilingual SWE-bench suite evaluating software engineering issue resolution across repositories and languages beyond the original Python-heavy SWE-bench tasks.

Models ranked

15

tracked on this benchmark

Score band

96.6 – 30.8

best → lowest tracked

Snapshot trend

+18.30

Oct 16 → Nov 9 · 1 models

Leaderboard

Tracked models ranked by % Resolved (higher is better).

Compare candidates
#Model variant and provenanceScore
1

Configuration: claude-fable-5-2 (max effort, mean of 5 trials)

Version: SWE-bench MultilingualHarness: Anthropic system-card standard configuration (scaffold not named); average of five trialsEvaluator: AnthropicObserved: Nov 9, 2026Confidence: self-reportedSource

Notes: Anthropic self-reported (lab-published). System card, retrieved 2026-11-09. Does not move /best rankings (self-reported / raon rule). System card §8.2: 300 problems across nine programming languages; 96.6%; average of five trials, max effort.

96.6
2

Configuration: Claude Opus 5 standard configuration

Version: SWE-bench MultilingualHarness: Anthropic system-card standard configuration; average of five trialsEvaluator: AnthropicObserved: Jul 24, 2026Confidence: confirmedSource

Notes: Vendor-reported. Source: p.149, Table 8.1.A. This multilingual suite is not the English SWE-bench Verified row.

89.5
3

Configuration: Qwen3.8-Flash

Version: SWE-bench MultilingualHarness: mini-SWE-agent (temperature 1.0, top_p 0.95, 256K context)Evaluator: Qwen (Alibaba)Observed: Oct 26, 2026Confidence: self-reportedSource

Notes: Qwen Team tech report 'Qwen3.8-Omni: Towards Native Omni-Modal Agents' (arXiv 2609.25611v1, 2026-09-22), Table 2 column 'Qwen3.8-Flash', row 'SWE-bench Multilingual': 81.0. Identity: the report's 'Qwen3.8-Flash' column and the QwenCloud qwen3.8-flash docs page carry the same values as the Qwen3.8-Flash-Next blog/HF card; Qwen's blog says Qwen3.8-Flash is the production served version (1M context, built-in tools) while these evals used a 256K context. Retrieved 2026-10-26.

81.0
4

Configuration: Qwen3.8-Omni-Flash

Version: SWE-bench MultilingualHarness: mini-SWE-agent (temperature 1.0, top_p 0.95, 256K context)Evaluator: Qwen (Alibaba)Observed: Oct 26, 2026Confidence: self-reportedSource

Notes: Qwen Team tech report 'Qwen3.8-Omni: Towards Native Omni-Modal Agents' (arXiv 2609.25611v1, 2026-09-22), Table 2 column 'Qwen3.8-Omni-Flash', row 'SWE-bench Multilingual': 80.5. Retrieved 2026-10-26.

80.5
5
Version: SWE-bench MultilingualHarness: Not recordedEvaluator: Not recordedObserved: May 21, 2026Confidence: Not recordedSource 1Source 2

Notes: Confidence: medium. secondary-source score for Cursor Composer 2.5; compare directionally because Composer is a productized IDE agent.

79.8
6
Version: SWE-bench Multilingual; Anthropic standard configuration, adaptive thinking at max effort, default sampling, average over 5 trialsHarness: Not recordedEvaluator: Not recordedObserved: Jun 30, 2026Confidence: Not recordedSource
78.3
7
Version: SWE-bench MultilingualHarness: Not recordedEvaluator: Not recordedObserved: Oct 16, 2026Confidence: Not recordedSource

Notes: Reflection first-party final model card (HF https://huggingface.co/reflection/Beam-1.0-501B-A23B, BF16 column), retrieved 2026-10-16. Replaces 2026-10-05 preview-blog score 78.0. Card label 'SWEBench Multilingual'. Lab self-reported.

78.3
8
Version: SWE-bench Multilingual; Code Agent chart; LongCat launch benchmarkHarness: Not recordedEvaluator: Not recordedObserved: Jun 30, 2026Confidence: Not recordedSource

Notes: Official LongCat-2.0 launch chart reports 77.33. Evaluator/harness details are not fully reproducible from public text. Confidence: confirmed for reported value, medium for reproducibility.

77.3
9
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 20, 2026Confidence: Not recordedSource
76.7
10
Version: Not recordedHarness: Not recordedEvaluator: Not recordedObserved: Apr 24, 2026Confidence: Not recordedSource
76.2
11
Version: SWE-bench MultilingualHarness: Not recordedEvaluator: Not recordedObserved: May 21, 2026Confidence: Not recordedSource

Notes: Confidence: medium. Preserve harness caveat when comparing against Composer 2.5.

75.9
12
Version: SWE-bench Multilingual resolved, V4-Flash Think MaxHarness: Not recordedEvaluator: Not recordedObserved: Jul 3, 2026Confidence: Not recordedSource
73.3
13
Version: SWE-bench Multilingual (official NVIDIA HF model card)Harness: Not recordedEvaluator: Not recordedObserved: Oct 7, 2026Confidence: Not recordedSource
67.7
14
Version: SWE-bench Multilingual (official IBM HF model card)Harness: Not recordedEvaluator: Not recordedObserved: Oct 7, 2026Confidence: Not recordedSource
41.9
15
Version: SWE-bench Multilingual (official IBM HF model card)Harness: Not recordedEvaluator: Not recordedObserved: Oct 7, 2026Confidence: Not recordedSource
30.8

How to read this benchmark

This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.

Trust this score when

  • There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
  • The model list covers the same version family you can actually deploy today.
  • Top candidates overlap with your required routing and feature requirements.

Be cautious when

  • There is only one benchmark snapshot or the dataset appears stale.
  • The benchmark metric direction is opposite of your decision objective.
  • The score difference between options is narrow and likely within implementation variance.

Related benchmarks

Last reviewed: May 21, 2026