MCP-Atlas
Benchmark for evaluating model performance on Model Context Protocol tool use tasks
Models ranked
27
tracked on this benchmark
Score band
89.2 – 29.5
best → lowest tracked
Snapshot trend
-33.62
Oct 26 → Nov 6 · 2 models
Leaderboard
Tracked models ranked by Score (higher is better).
Notes: Meta HF model card Benchmarks table, xhigh reasoning effort.
Configuration: Muse Spark 1.1 (xhigh)
Notes: Muse Spark 1.1 model page benchmark table (raw HTML parsed). Methodology: evaluation report https://ai.meta.com/static-resource/muse-spark-1-1-evaluation-report/ (2026-07-09). All Muse Spark 1.1 results run via Meta Model API at xhigh reasoning effort (report §5.1, p.100-101). Report §5.2.1: 'Results are sourced from Scale AI' (benchmark owner). Retrieved 2026-10-17.
Configuration: Fable 5.1
Notes: Scale AI MCP-Atlas leaderboard (headless-Chrome capture 2026-10-26). Row: rank 2 'Fable 5.1 NEW', 87.20 ± 2.05. The page's methodology stamp reads 'Updated April 8, 2026', which predates Fable 5.1; the row itself carries the board's NEW badge. Effort level is not shown on the board. Scale runs the evaluation with its own MCP-Atlas harness. Retrieved 2026-10-26.
Notes: Model variant: Kimi K3 (max reasoning effort). Benchmark variant: MCP-Atlas 500-task public subset with a 100-turn limit. Harness/evaluator: Gemini 3.1 Pro judge; remaining harness/evaluator details not disclosed by Moonshot. Source posture: Moonshot/Kimi self-reported launch table; confidence: medium because the value is vendor-reported and not independently reproduced. Recommended seed value: 84.2.
Notes: Low-confidence secondary-source row; useful as supporting Anthropic MCP-native evidence.
Notes: Reflection first-party final model card (HF https://huggingface.co/reflection/Beam-1.0-501B-A23B, BF16 column), retrieved 2026-10-16. Replaces 2026-10-05 preview-blog score 78.7. Card label 'MCP Atlas'. Lab self-reported.
Notes: Moonshot-run MCP tool-use benchmark. Compare with other MCP Atlas rows cautiously when source harnesses differ.
Configuration: High Reasoning
Notes: Score confirmed by the Muse Glimmer technical report (Oct 19, 2026), Table 2 (Tool Use), PDF p.4, Muse Glimmer (high) = 75.5; internal run (App. A).
Notes: Low-confidence secondary-source row; useful as supporting tool-use evidence.
Configuration: Ornith-1.5-122B
Notes: Self-reported by Ornith. Ornith first-party HF model card Ornith-1.5-122B, benchmark table (column 'Ornith-1.5-122B'). Retrieved 2026-11-06. MCP-Atlas, 500-task public subset, thinking mode, Claude 4.8 Opus judge: 74.6.
Configuration: Solar Pro 4
Notes: Upstage Solar Pro 4 launch blog (2026-08-11), comparison table (Solar Open 2 / Solar Pro 4 / Gap). Asterisked rows 'evaluated in-house under the same internal evaluation environment'. Row 'MCP-Atlas*': Solar Pro 4 61.4. Retrieved 2026-10-29.
How to read this benchmark
This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
Related benchmarks
Last reviewed: May 20, 2026