activeAgents

OSWorld 2.1 offline set (partial score)

Metric: Partial score (higher is better)Introduced: 2026

OSWorld 2.1 official offline set: 82 of 108 long-horizon computer-use tasks on a live Ubuntu VM with no internet access, default 1080p and a 500-action step cap. The partial score is mean per-task credit across checkpoints, reported as Pass@1 averaged over five independent attempts. Distinct from osworld, osworld-verified, and the strict pass rate on the same 2.1 offline set.

Models ranked

tracked on this benchmark

Score band

89.3 – 89.3

best → lowest tracked

Snapshot trend

—

need ≥2 snapshots

Leaderboard

Tracked models ranked by Partial score (higher is better).

Compare candidates

#Model variant and provenanceRelative to leaderScore

Claude Fable 5.2

Configuration: claude-fable-5-2 (max effort, Pass@1 mean of 5, 82-task official offline set)

Version: OSWorld 2.1 offline set (partial)Harness: OSWorld 2.1 official offline set (82 of 108 tasks, no internet); default 1080p / 500-step settings; Pass@1 averaged over five attempts; Opus 4.8 model grader where neededEvaluator: AnthropicObserved: Nov 9, 2026Confidence: self-reportedSource

Notes: Anthropic self-reported (lab-published). System card + launch table, retrieved 2026-11-09. Does not move /best rankings (self-reported / raon rule). System card §8.13.3 and launch table: 89.3% partial score on OSWorld 2.1 official offline set (82 of 108 tasks, no internet), max effort, Pass@1 averaged over five attempts, default 1080p / 500-step settings, Opus 4.8 as model grader where needed. Strict pass rate was 54.6% (not this row). Do NOT write onto osworld or osworld-verified — this is a 2.1 offline-set partial metric.

89.3

How to read this benchmark

This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.

Trust this score when

There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
The model list covers the same version family you can actually deploy today.
Top candidates overlap with your required routing and feature requirements.

Be cautious when

There is only one benchmark snapshot or the dataset appears stale.
The benchmark metric direction is opposite of your decision objective.
The score difference between options is narrow and likely within implementation variance.

Related benchmarks

AA Coding Agent Index

Artificial Analysis Coding Agent Index

Berkeley Function Calling Leaderboard v3

Last reviewed: Nov 9, 2026

Resources

Website