ClaimBench
ClaimBench is a Vals AI benchmark for decision models, released October 20, 2026. It has 1,960 claims, each paired with an excerpt from a company's 10-K filing; the system must say whether the excerpt supports the claim, contradicts it, or says nothing about it (claim kinds: direct support, subtle contradiction, numeric discrepancy, missing evidence, non-supporting citation). Vals AI runs the evaluation and publishes the leaderboard; the headline metric is accuracy over all claims.
Models ranked
8
tracked on this benchmark
Score band
99.7 – 87.7
best → lowest tracked
Snapshot trend
—
need ≥2 snapshots
Leaderboard
Tracked models ranked by Accuracy (%) (higher is better).
Notes: Vals AI ClaimBench leaderboard (vals.ai/benchmarks/claimbench, Decision models tab, updated 10/20/2026), rank 1 of 14, 'Decider V1.2' accuracy 99.69%; 1,960 claims on 10-K excerpts. Third-party run by Vals AI.
Notes: Vals AI ClaimBench leaderboard (vals.ai/benchmarks/claimbench, Decision models tab, updated 10/20/2026), rank 3 of 14, 'Decider V1.1 27B' accuracy 98.78%; 1,960 claims on 10-K excerpts. Third-party run by Vals AI.
Notes: Vals AI ClaimBench leaderboard (vals.ai/benchmarks/claimbench, Decision models tab, updated 10/20/2026), rank 4 of 14, 'd1' accuracy 98.67%; 1,960 claims on 10-K excerpts. Third-party run by Vals AI.
Notes: Vals AI ClaimBench leaderboard (vals.ai/benchmarks/claimbench, Decision models tab, updated 10/20/2026), rank 5 of 14, 'Microsoft-Decision-1' accuracy 98.21%; 1,960 claims on 10-K excerpts. Third-party run by Vals AI.
Notes: Vals AI ClaimBench leaderboard (vals.ai/benchmarks/claimbench, Decision models tab, updated 10/20/2026), rank 8 of 14, 'Jev 1.13.0' accuracy 97.65%; 1,960 claims on 10-K excerpts. Third-party run by Vals AI. Listed entry: Jev 1.13.0.
Notes: Vals AI ClaimBench leaderboard (vals.ai/benchmarks/claimbench, Decision models tab, updated 10/20/2026), rank 10 of 14, 'GPT-6 Luna Decisions' accuracy 96.89%; 1,960 claims on 10-K excerpts. Third-party run by Vals AI.
Notes: Vals AI ClaimBench leaderboard (vals.ai/benchmarks/claimbench, Decision models tab, updated 10/20/2026), rank 11 of 14, 'Decider Flash V1.2' accuracy 96.53%; 1,960 claims on 10-K excerpts. Third-party run by Vals AI.
Notes: Vals AI ClaimBench leaderboard (vals.ai/benchmarks/claimbench, Decision models tab, updated 10/20/2026), rank 13 of 14, 'Clef Omni' accuracy 87.70%; 1,960 claims on 10-K excerpts. Third-party run by Vals AI.
How to read this benchmark
This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
Last reviewed: Oct 20, 2026