activeAgentsAudio

EVA-Bench Accuracy (EVA-A pass@1)

Metric: EVA-A pass@1 (%) (higher is better)Introduced: 2026

ServiceNow's open-source end-to-end benchmark for enterprise voice agents: simulated callers (ElevenLabs user simulator) run full spoken conversations across airline, ITSM and HR/medical scenarios. EVA-A pass@1 is the share of trials where every accuracy metric (task completion, speech fidelity, faithfulness) clears its threshold, averaged over scenarios with 5 trials each. Pooled across the three domains, clean audio. Kept separate from the EVA-X experience axis.

Models ranked

1

tracked on this benchmark

Score band

73.7 – 73.7

best → lowest tracked

Snapshot trend

—

need ≥2 snapshots

Leaderboard

Tracked models ranked by EVA-A pass@1 (%) (higher is better).

Compare candidates
#Model variant and provenanceScore
1
Grok Voice 3.0

Configuration: Grok Voice 3.0 (speech-to-speech)

Version: EVA-Bench EVA-A pass@1, clean, pooled (airline, ITSM, HR)Harness: Not recordedEvaluator: ServiceNow (EVA-Bench)Observed: Nov 9, 2026Confidence: confirmedSource

Notes: Benchmark-owner board: ServiceNow EVA-Bench results site (servicenow.github.io/eva, data file website/src/data/leaderboardStats.json in github.com/ServiceNow/eva at main 8e101906; Grok Voice 3.0 added in commit e5772533 'add grok results', 2026-10-13). System 'Grok Voice 3.0' (id grok-voice-3-0, type s2s). Clean condition, pooled over airline (n=50), ITSM (n=80) and medical_hr (n=83) scenarios, 5 trials each. EVA-A_pass pooled point 0.7371519410977241 (95% CI 0.6906-0.7805), stored as percent rounded to 2 decimals. Retrieved 2026-11-09.

73.7

How to read this benchmark

This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.

Trust this score when

  • There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
  • The model list covers the same version family you can actually deploy today.
  • Top candidates overlap with your required routing and feature requirements.

Be cautious when

  • There is only one benchmark snapshot or the dataset appears stale.
  • The benchmark metric direction is opposite of your decision objective.
  • The score difference between options is narrow and likely within implementation variance.

Related benchmarks

AA Coding Agent Index
Artificial Analysis Coding Agent Index
Agents' Last Exam
Android Daily
AndroidWorld
AutomationBench
BFCL v3
Berkeley Function Calling Leaderboard v3

Last reviewed: Nov 9, 2026

Resources

WebsiteGitHubarXiv PaperHuggingFace