OpenWorldEngineer Mean@8
Mercor’s 26-task benchmark of long-horizon engineering and research projects. Each task’s raw score is rescaled against its baseline and ceiling; Mean@8 averages eight attempts per task, then tasks equally. Best@8 and Elo are separate metrics.
Models ranked
5
tracked on this benchmark
Score band
67.2 – 41.2
best → lowest tracked
Snapshot trend
—
need ≥2 snapshots
Leaderboard
Tracked models ranked by Mean@8 (%) (higher is better).
Configuration: GPT-6.2 Astra Max
Notes: Official benchmark-owner results, independently evaluated. Mean@8 percentage, not Best@8 or Elo. Timestamp is retrieval date; page has no per-row run date. Reported 95% intervals are in the captured table.
Configuration: Fable 5.2 Max
Notes: Official benchmark-owner results, independently evaluated. Mean@8 percentage, not Best@8 or Elo. Timestamp is retrieval date; page has no per-row run date. Reported 95% intervals are in the captured table.
Configuration: GPT-6 Astra Max
Notes: Official benchmark-owner results, independently evaluated. Mean@8 percentage, not Best@8 or Elo. Timestamp is retrieval date; page has no per-row run date. Reported 95% intervals are in the captured table.
Configuration: Opus 5.5 Max
Notes: Official benchmark-owner results, independently evaluated. Mean@8 percentage, not Best@8 or Elo. Timestamp is retrieval date; page has no per-row run date. Reported 95% intervals are in the captured table.
Configuration: GPT-6.1 Sol Max
Notes: Official benchmark-owner results, independently evaluated. Mean@8 percentage, not Best@8 or Elo. Timestamp is retrieval date; page has no per-row run date. Reported 95% intervals are in the captured table.
How to read this benchmark
This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
Related benchmarks
Last reviewed: Nov 18, 2026