tau-bench: τ-bench
Multi-turn tool-use benchmark for agent behavior in realistic retail and airline customer-service tasks, measuring whether models complete tasks through APIs while following policy constraints.
Models ranked
32
tracked on this benchmark
Score band
95.3 – 61.1
best → lowest tracked
Snapshot trend
+25.40
Aug 7 → Oct 19 · 1 models
Leaderboard
Tracked models ranked by % Task Success (higher is better).
Notes: Aleph Alpha first-party HF model card Aleph-Alpha/Kolibri-1 (FP8 release checkpoint), Evaluation > Post-training table; same value in launch blog/landing page. Harness: Aleph-Alpha-Research eval-framework (Harbor for TerminalBench/SWE-Bench), Kolibri at reasoning effort high, each model at its documented context window. BF16 repo card reports 94.4. Other splits not seeded (single identity key): Retail 69.9, Airline 76.7, Tau3-Bench Banking 38.1. Self-reported by the lab.
Notes: Confidence: high. weekly best-of refresh: benchlm.ai TAU-bench snapshot lists Claude Mythos Preview at 89.2%.
Configuration: High Reasoning
Notes: Muse Glimmer technical report (Oct 19, 2026), Table 5 (reasoning-strength scorecard, Agentic & Coding), PDF p.24; Muse Glimmer (high) column. Retail domain, 110 tasks, mean of 4 runs (App. A.1, p.26).
Notes: source reconciliation: benchlm.ai tau-bench snapshot lists Claude Opus 4.6 at 84.8.
Notes: source reconciliation: benchlm.ai tau-bench snapshot lists Grok 4.20 at 78.9.
How to read this benchmark
This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
Related benchmarks
Last reviewed: Apr 26, 2026