Decision Index (multimodalart)active

Decision Index (multimodalart): JEV Decision Index (0.2.1)

Metric: Decision Index (balanced skill) (higher is better)Introduced: 2026

Community Decision Index for typed decision models (choice, yes/no, and score answers with probabilities), run and published on the Hugging Face Space multimodalart/jev-decision-index by Apolinário (multimodalart). Edition 0.2.1 covers a fixed public panel across five areas (Knowledge, Language, Retrieval, Tools, Arts); the headline balanced_skill score chance-corrects each benchmark so random guessing is 0 and perfect is 100. Live Space edition as of 2026-10-15 is 0.3.2; vendor scores in this packet cite 0.2.1.

Models ranked

4

tracked on this benchmark

Score band

61.7 – 48.6

best → lowest tracked

Snapshot trend

—

need ≥2 snapshots

Leaderboard

Tracked models ranked by Decision Index (balanced skill) (higher is better).

Compare candidates
#Model variant and provenanceScore
1
SubQ Snap
Version: 0.2.1Harness: Not recordedEvaluator: Not recordedObserved: Oct 15, 2026Confidence: Not recordedSource

Notes: Vendor-reported. Subquadratic launch post: Snap scored 61.70 vs Jev 57.90 on Decision Index 0.2.1. Self-run; Snap is not on the Space 0.2.1 board file.

61.7
2
pplx-decider-v1.1-27b
Version: 0.2.1Harness: Not recordedEvaluator: Not recordedObserved: Oct 15, 2026Confidence: Not recordedSource

Notes: Vendor-reported. Perplexity HF card Evaluation table: Overall Decision Index 61.56 (Knowledge 48.18, Language 69.45, Retrieval 61.26, Tools 78.88, Arts 44.66). Card links https://huggingface.co/spaces/multimodalart/jev-decision-index; edition not printed, but v1 baseline 56.4 matches the Space 0.2.1 board row for pplx-decider-v1-27b, so version 0.2.1 inferred.

61.6
3
Surogate Rune 26B-A4B
Version: 0.2.1Harness: Not recordedEvaluator: Not recordedObserved: Oct 15, 2026Confidence: Not recordedSource

Notes: Vendor-reported. Surogate page: 57.44 Decision Index 0.2.1, #2 on the board as published 2026-09-26; maintainers reran open weights bit-for-bit. Cross-check: Space data/index-v0.2.1.json engine rune-26b-a4b-v3 balanced_skill 57.44.

57.4
4
d1-3B
Version: 0.2.1Harness: Not recordedEvaluator: Not recordedObserved: Oct 15, 2026Confidence: Not recordedSource

Notes: Vendor-reported. Liquid AI HF card Decision Index 0.2.1 table: d1-3B 48.57 (Knowledge 23.8, Language 56.4, Retrieval 52.8, Tools 74.5, Arts 36.3); scored with the official scorer (not a leaderboard submission). Same 48.57 on liquid.ai/blog/d1-open (2026-10-07), public split.

48.6

How to read this benchmark

This benchmark scores models where higher is better. Use scores for directional filtering and shortlisting, not universal quality ranking; then validate pricing, context window, provider availability, and fit for your workload.

Trust this score when

  • There is a fresh timestamped snapshot (or multiple snapshots) for this benchmark.
  • The model list covers the same version family you can actually deploy today.
  • Top candidates overlap with your required routing and feature requirements.

Be cautious when

  • There is only one benchmark snapshot or the dataset appears stale.
  • The benchmark metric direction is opposite of your decision objective.
  • The score difference between options is narrow and likely within implementation variance.

Last reviewed: Oct 15, 2026

Resources

Website