Code editing across 8 programming languagesactiveCoding
Code editing across 8 programming languages: Aider Polyglot
Metric: % Exercises CompletedIntroduced: 2024
Real-world code editing benchmark measuring a model's ability to apply changes to existing codebases across 8 programming languages using Exercism platform exercises.
Models ranked
33
tracked on this benchmark
Score band
88.0 – 3.6
best → lowest tracked
Snapshot trend
-3.50
May 29 → Jul 1 · 1 models
Leaderboard
Tracked models ranked by % Exercises Completed (higher is better).
#Model variant and provenanceRelative to leaderScore
1
GPT-5
88.0Version: Aider PolyglotHarness: Not recordedEvaluator: Not recordedObserved: May 5, 2026Confidence: Not recordedSource
Notes: Confidence: high. DAT-3780 weekly best-of refresh: uses the highest GPT-5 thinking-level score reported by aider.chat.
2
GPT-5.5
88.0Version: Listed as 'gpt-5 (high)' = 88 (percent_correct)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
3
o3-pro
84.9Version: 2026-04 (high)Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
4
Gemini 2.5 Pro
83.1Version: preview-06-05 (32k think)Harness: Not recordedEvaluator: Not recordedObserved: May 29, 2026Confidence: Not recordedSource
5
o3
81.3Version: 2026-04 (high)Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
6
Grok 4
79.6Version: Aider polyglot, high reasoning, 225 testsHarness: Not recordedEvaluator: Not recordedObserved: Jul 1, 2026Confidence: Not recordedSource
7
Grok 4.20
79.6Version: Listed as 'Grok-4 (high)' = 79 (percent_correct)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
8
Gemini 2.5 Pro Preview 06-05
79.1Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
9
DeepSeek V3.2 Exp
74.2Version: 2026-04 (Reasoner variant)Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
10
Claude 4 Opus
72.0Version: 2026-04 (32k thinking tokens)Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
11
o4-mini
72.0Version: 2026-04 (high)Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
12
Claude Opus 4.5
72.0Version: 2026-05 (32k thinking)Harness: Not recordedEvaluator: Not recordedObserved: May 12, 2026Confidence: Not recordedSource
Notes: Confidence: high. DAT-4172 May 12 /best/ refresh; aider leaderboard listed claude-opus-4-20250514 32k thinking at 72.0%.
13
Claude Opus 4.6
72.0Version: Listed as 'Claude Opus 4 (32k thinking)' = 72 (percent_correct)Harness: Not recordedEvaluator: Not recordedObserved: Jun 7, 2026Confidence: Not recordedSource
14
DeepSeek R1 0528
71.4Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
15
Claude 3.7 Sonnet
64.9Version: 2026-04 (32k thinking tokens)Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
16
Claude 4 Sonnet
61.3Version: 2026-04 (32k thinking tokens)Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
17
o3 Mini
60.4Version: 2026-04 (high)Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
18
Qwen3-235B-A22B
59.6Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
19
DeepSeek R1
56.9Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
20
Gemini 2.5 Flash
55.1Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
21
DeepSeek V3 0324
55.1Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
22
Grok-3
53.3Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
23
GPT-4.1
52.4Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
24
Claude 3.5 Sonnet
51.6Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
25
DeepSeek V3
48.4Version: 2026-04Harness: Not recordedEvaluator: Not recordedObserved: Apr 14, 2026Confidence: Not recordedSource
Showing the top 25 of 33 tracked models. Browse all models.
How to read this benchmark
This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
Related benchmarks
Last reviewed: Apr 15, 2026