JEV Decision Index (0.4)
Community Decision Index for typed decision models (choice, yes/no, and score answers with probabilities), run and published on the Hugging Face Space multimodalart/jev-decision-index by Apolinário (multimodalart). Edition 0.4 (2026-10-19; point release 0.4.1, 2026-10-21) computes the headline Full score from private tests only: 0.60 × private tests of the same skills + 0.40 × private tasks from new domains, both equated onto the public scale using 21 untuned stock models. Public benchmarks are still run and shown but no longer count.
Models ranked
2
tracked on this benchmark
Score band
68.9 – 60.3
best → lowest tracked
Snapshot trend
—
need ≥2 snapshots
Leaderboard
Tracked models ranked by Decision Index 0.4 (Full score) (higher is better).
Notes: Independent board multimodalart/jev-decision-index, Decision Index 0.4.1, entrant 'mammoth-1 (27B)' (engine mammoth-1, weights_repo baselabs/mammoth-1). Full score 68.87 (= 0.60 same-skill private + 0.40 new-domain private, equated). Parts: same-skill 66.56, new-domain 56.78; public index 75.18 (not counted). Rank 2 by Full score (tie group #1, tied); public rank 1. Board data generated 2026-10-21T20:30:52Z (2026-10-21 13:30 PDT), edition v0.4 dated 2026-10-19, v0.4.1 dated 2026-10-21; Space sha 1f4e4904cc57. Retrieved 2026-10-21. Vendor claim '#1 ... 71.24' is the edition 0.4 value (board data 2026-10-20T14:13:56Z, rank 1 untied); 0.4.1 went live 2026-10-21T15:13Z, before the 17:13Z launch. Board note: 'Scored on the weights submitted for evaluation (revision b9f2f2c). The repository's main branch has since been updated with different weights.' (HF main now 8f9f861 weights, 2026-10-20 18:14Z.) Board values used, not vendor values.
Notes: Independent board multimodalart/jev-decision-index, Decision Index 0.4.1, entrant 'elephant-1 (9B)' (engine elephant-1-baselabs, weights_repo baselabs/elephant-1). Full score 60.26 (= 0.60 same-skill private + 0.40 new-domain private, equated). Parts: same-skill 62.44, new-domain 44.41; public index 72.24 (not counted). Rank 38 by Full score (tie group #8, tied); public rank 4. Board data generated 2026-10-21T20:30:52Z (2026-10-21 13:30 PDT), edition v0.4 dated 2026-10-19, v0.4.1 dated 2026-10-21; Space sha 1f4e4904cc57. Retrieved 2026-10-21. Vendor 62.65 (report) / 62.6 (library) is the edition 0.4 value (board 2026-10-20, rank 25, tie group 7); blog says 'tied for #8', report 'tied 7th'. Scored checkpoint 14537f0 = current HF main weights commit. Board values used, not vendor values.
How to read this benchmark
This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.
Trust this score when
- There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
- The model list covers the same version family you can actually deploy today.
- Top candidates overlap with your required routing and feature requirements.
Be cautious when
- There is only one benchmark snapshot or the dataset appears stale.
- The benchmark metric direction is opposite of your decision objective.
- The score difference between options is narrow and likely within implementation variance.
Last reviewed: Oct 21, 2026