SWE-bench Multilingual
Multilingual SWE-bench suite evaluating software engineering issue resolution across repositories and languages beyond the original Python-heavy SWE-bench tasks.
Models ranked
15
tracked on this benchmark
Score band
96.6 – 30.8
best → lowest tracked
Snapshot trend
+18.30
Oct 16 → Nov 9 · 1 models
Leaderboard
Tracked models ranked by % Resolved (higher is better).
Configuration: claude-fable-5-2 (max effort, mean of 5 trials)
Notes: Anthropic self-reported (lab-published). System card, retrieved 2026-11-09. Does not move /best rankings (self-reported / raon rule). System card §8.2: 300 problems across nine programming languages; 96.6%; average of five trials, max effort.
Configuration: Claude Opus 5 standard configuration
Notes: Vendor-reported. Source: p.149, Table 8.1.A. This multilingual suite is not the English SWE-bench Verified row.
Configuration: Qwen3.8-Flash
Notes: Qwen Team tech report 'Qwen3.8-Omni: Towards Native Omni-Modal Agents' (arXiv 2609.25611v1, 2026-09-22), Table 2 column 'Qwen3.8-Flash', row 'SWE-bench Multilingual': 81.0. Identity: the report's 'Qwen3.8-Flash' column and the QwenCloud qwen3.8-flash docs page carry the same values as the Qwen3.8-Flash-Next blog/HF card; Qwen's blog says Qwen3.8-Flash is the production served version (1M context, built-in tools) while these evals used a 256K context. Retrieved 2026-10-26.
Configuration: Qwen3.8-Omni-Flash
Notes: Qwen Team tech report 'Qwen3.8-Omni: Towards Native Omni-Modal Agents' (arXiv 2609.25611v1, 2026-09-22), Table 2 column 'Qwen3.8-Omni-Flash', row 'SWE-bench Multilingual': 80.5. Retrieved 2026-10-26.
Notes: Confidence: medium. secondary-source score for Cursor Composer 2.5; compare directionally because Composer is a productized IDE agent.
Notes: Reflection first-party final model card (HF https://huggingface.co/reflection/Beam-1.0-501B-A23B, BF16 column), retrieved 2026-10-16. Replaces 2026-10-05 preview-blog score 78.0. Card label 'SWEBench Multilingual'. Lab self-reported.
Notes: Official LongCat-2.0 launch chart reports 77.33. Evaluator/harness details are not fully reproducible from public text. Confidence: confirmed for reported value, medium for reproducibility.
Notes: Confidence: medium. Preserve harness caveat when comparing against Composer 2.5.
How to read this benchmark
This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
Related benchmarks
Last reviewed: May 21, 2026