LLMReference — Data Quality Dashboard

Pulse summarizes weekly release velocity, freshness, cadence, and current frontier pricing. Changelog is the month-by-month archive; this page covers raw field completeness for maintainers.

Headline Coverage

2,371
Total model rows tracked
Full seed catalog, including embeddings and deprecated rows
2,320
Public models incl. deprecated
233 deprecated public rows remain available behind the toggle
53
Models added this month
Released in November 2026
1,502
Open-weights models
Self-hostable or open-license rows
177
API providers covered
Provider catalogs and routes
311
Labs and creators
Research labs, companies, and builders
203
Benchmarks covered
Evaluation sources in the seed data

Database Totals

2,087
Public models
676
Families
177
Providers
311
Researchers
203
Benchmarks
69
Concepts
2,612
Provider Links

License-Derived Openness

894
Open source
43% of public models
543
Open weights
26% of public models
650
Proprietary
31% of public models
0
Unknown
0% of public models
License coverageKnownMissing / unknown
Public models2,087 (100%)0 (0%)
Families676 (100%)0 (0%)
OSI classificationCountShare
OSI-approved effective license894 / 2,08743%
Non-OSI effective license1,193 / 2,08757%
Missing / unknown effective license0 / 2,0870%
Commercial useModelsShare
Permitted977 / 2,08747%
Restricted1,031 / 2,08749%
Not permitted79 / 2,0874%
Unknown0 / 2,0870%
Verification backlogCountHealth
License records marked verify:true0 / 740 (0%)
License records with unknown openness1 / 741 (1%)
License records with unknown commercial use1 / 741 (1%)
Models using verify:true licenses0 / 2,0870 (0%)
Models missing/unknown effective license0 / 2,0870 (0%)
Families missing/unknown license0 / 6760 (0%)
Model rows use the effective license resolver: model license first, then family license. Missing licenses and licenses classified as unknown remain visible as backlog instead of being inferred from legacy strings.

Data Freshness

EntityTotalPlaceholder 2026-01-01Stale >90dStale >180d
Public models2,0870 (0%)1,180 (57%)861 (41%)
Families6760 (0%)353 (52%)263 (39%)
Providers1770 (0%)85 (48%)6 (3%)
Benchmarks2030 (0%)157 (77%)85 (42%)
Missing or invalid lastResearched dates are counted as stale so research queues do not hide unverified rows.

Benchmark Coverage

Source URL coverage
99%
2,753 / 2,791 scores start with http.
Benchmarks with stale score sets
2
  • gaokao: 11 rows, latest 2023-05-20
  • gaokao-mm: 7 rows, latest 2024-02-23
MetricCountCoverage / Health
Public models with >=1 benchmark score564 / 2,08727%
Scores with verified URL source2,753 / 2,79199%
Benchmarks with zero scores56 / 20372%
Benchmark scores older than 1 year105 / 2,79196%
Benchmark scores older than 2 years29 / 2,79199%
Benchmarks with all scores older than 2 years2 / 14799%

Model Field Completeness

FieldFilledCoverage
parameters1,480 / 2,08771%
context1,480 / 2,08771%
knowledgeCutoff531 / 2,08725%
description2,087 / 2,087100%
release2,052 / 2,08798%
familySlug2,087 / 2,087100%
vision471 / 2,08723%
reasoning337 / 2,08716%
functionCalling403 / 2,08719%
toolUse393 / 2,08719%
structuredOutputs475 / 2,08723%
codeExecution91 / 2,0874%
promptCaching109 / 2,0875%
batchApi56 / 2,0873%
audio213 / 2,08710%
fineTuning22 / 2,0871%
multimodal620 / 2,08730%
maxOutputTokens143 / 2,0877%
licenseSlug942 / 2,08745%
Capability rows count truthy capability flags; empty, 0, and false mean the capability is not currently recorded as present.

Family and Concept Completeness

FieldFilledCoverage
family.description676 / 676100%
family.researcherSlug674 / 676100%
concept.lastResearched69 / 69100%

Pricing Coverage

MetricCountCoverage / Health
Provider links with pricing1,828 / 2,18284%
Excludes non-token-priced rows (per-image, per-second, gpu-hour, etc.).
Models with ≥1 pricing record854 / 2,08741%
Provider links with model ID (providerModelId)2,118 / 2,61281%
Non-token links missing all prices501 / 2,61281%
Provider links with $0 token price102 / 2,61296%
Missing-price rows have no token, image, video, audio, query, or gpu-hour price recorded. $0 token rows may include true free-tier entries and should be reviewed before changing seed data.

Data Quality

IndicatorCount
Duplicate model slugs0
Duplicate model+provider pairs0
Models without family0
Families missing description0
Concepts missing lastResearched0
Broken family references (model→family)0
Broken researcher references (family→researcher)0
Broken provider references (link→provider)0
Broken model references (link→model)0

Provider Link Completeness

FieldFilledCoverage
docs155 / 17788%
pricing139 / 17779%
portal138 / 17778%
tier146 / 17782%
providerType161 / 17791%

Provider Coverage (top 25 by model count)

ProviderModelsPricedPricing %
OpenRouter36634594%
Fireworks AI22219286%
Vercel AI Gateway20219597%
Together AI11510692%
AWS Bedrock11310492%
Microsoft Foundry1138676%
GCP Vertex AI997475%
Replicate API978487%
Novita AI9696100%
DeepInfra575393%
Alibaba Cloud PAI-EAS492551%
OpenAI API444295%
Google AI Studio433172%
Cloudflare Workers AI372773%
IBM watsonx322784%
Featherless2828100%
Mistral AI Studio221986%
Anthropic2121100%
Baseten API18528%
Bitdeer AI1818100%
Z.ai1515100%
Prodia1500%
Zhipu AI GLM API14321%
Cohere API13538%
Databricks Foundation Model Serving13538%

Coverage

2,001 active models

Denominators use active (non-deprecated) models from the full seed catalog via lib/stats-eligibility.ts. Benchmark eligibility follows the researcher baseline rules (modality-matched), including embed, image_gen, image_edit (interim: scored on lmarena-image-edit only), and video_gen benchmark types. Active excludes deprecated rows, commercialAvailability announced or pre-release, and rows with replacedBySlug. As-of 2026-11-15.

1. Benchmark coverage

TypeActive−5 Announced / pre-release · −131 Replaced by newer model0 scores1–23+
Text LLM1,5161,133171212
Decision3212173
Image153112410
Video4230120
Audio/voice144132111
Embed/rerank6449150
Other505000
Total2,0011,518267216

Zero-score models whose type has no benchmark in the catalog (other): 50.

BenchmarkTypeStatusScored / eligible%Active scored (any)
lmarena-text-creative-writingtextactive159/151610.5%160
gpqatextactive145/15169.6%148
chatbot-arenatextactive115/15167.6%116
mmlu-protextactive72/15164.7%76
mmlutextactive68/15164.5%71
humanevaltextactive66/15164.4%66
swe-bench-verifiedtextactive55/15163.6%55
livecodebenchtextactive50/15163.3%50
lmarena-visiontext_visionactive47/28416.5%50
bfcltextactive43/15162.8%43
terminal-bench-2-1textactive43/15162.8%43
mmmu-protext_visionactive41/28414.4%43
lmarena-text-to-imageimage_genactive38/14526.2%38
swe-bench-protextactive38/15162.5%38
hellaswagtextactive32/15162.1%35
aime-2025textactive28/15161.8%28
rulertextactive26/15161.7%26
hletextactive25/15161.6%25
deepswe-1-1textactive24/15161.6%24
browsecomptextactive22/15161.5%22
frontiermath-tier4textactive21/15161.4%21
aime-2026textactive18/15161.2%18
arc-agi-2textactive18/15161.2%18
hle-no-toolstextactive17/15161.1%17
hle-with-toolstextactive16/15161.1%16
ifevaltextactive16/15161.1%16
mmmutext_visionactive16/2845.6%20
tau-benchtextactive16/15161.1%16
terminal-bench-2-0textactive16/15161.1%16
mcp-atlastextactive15/15161%15
automation-benchtextactive14/15160.9%14
terminal-bench-4-0textactive14/15160.9%14
frontiercode-maintextactive13/15160.9%13
math-500textactive13/15160.9%13
bfcl-v3textsuperseded12/15160.8%12
decision-index-0-3decisionactive12/3237.5%15
lmarena-image-editimage_editactive12/12100%12
lmarena-text-to-videovideo_genactive12/4228.6%14
aider-polyglottextactive11/15160.7%11
cursorbenchtextactive11/15160.7%11
gsm8ktextactive11/15160.7%14
lmarena-image-to-videovideo_genactive11/4226.2%12
osworld-verifiedtext_visionactive11/2843.9%11
swe-bench-multilingualtextactive11/15160.7%11
genebench-protextactive10/15160.7%10
mteb-eng-v2embedactive10/4721.3%10
wmt26-general-mt-humantextactive10/15160.7%10
agents-last-examtextactive9/15160.6%9
multichallengetextactive9/15160.6%9
aime-2024textactive8/15160.5%8
gaokaotextactive8/15160.5%8
nl2repo-benchtextactive8/15160.5%8
mteb-multilingual-v2embedactive7/4714.9%7
toolathlontextactive7/15160.5%7
androidworldtext_visionactive6/2842.1%6
charxivtext_visionactive6/2842.1%6
osworld-gtext_visionactive6/2842.1%6
arctextactive5/15160.3%8
cybergymtextactive5/15160.3%5
decision-indexdecisionactive5/3215.6%5
deepsearchqatextactive5/15160.3%5
gdp-pdftext_visionactive5/2841.8%5
hmmt-feb-2026textactive5/15160.3%5
humaneval-plustextactive5/15160.3%5
program-benchtextactive5/15160.3%5
screenspot-protext_visionactive5/2841.8%5
babyvision-with-toolstext_visionactive4/2841.4%4
bigcodebenchtextactive4/15160.3%4
exploitgymtextactive4/15160.3%4
finance-agent-v2textactive4/15160.3%4
legal-agent-benchtextactive4/15160.3%4
mathvistatext_visionactive4/2841.4%4
mbpp-plustextactive4/15160.3%4
osworldtext_visionactive4/2841.4%4
swe-rebenchtextactive4/15160.3%4
webvoyagertext_visionactive4/2841.4%4
wer-librispeechaudio_sttactive4/478.5%4
worldvqatext_visionactive4/2841.4%4
chartography-with-toolstext_visionactive3/2841.1%3
codeforcestextactive3/15160.2%3
jevbench-public-231decisionactive3/329.4%3
sec-bench-protextactive3/15160.2%3
webclicktext_visionactive3/2841.1%3
blueprint-bench-2text_visionactive2/2840.7%2
decision-index-0-4decisionactive2/326.3%3
frontiercode-2-0textactive2/15160.1%2
jevbenchdecisionactive2/326.3%7
mathtextactive2/15160.1%2
screenspot-v2text_visionactive2/2840.7%2
tau-voiceaudio_s2sactive2/385.3%2
terminal-bench-science-0-1textactive2/15160.1%2
wer-open-asraudio_sttactive2/474.3%2
wer-open-universal-arabic-asraudio_sttactive2/474.3%2
android-dailytext_visionactive1/2840.4%1
arc-agi-2-hightext_visionactive1/2840.4%1
arc-agi-3-standardtextactive1/15160.1%1
bbhtextactive1/15160.1%1
benchcad-2-1-with-toolstext_visionactive1/2840.4%1
browsecomp-pro-computetextactive1/15160.1%1
browsecomp-singletextactive1/15160.1%1
chartography-no-toolstext_visionactive1/2840.4%1
claimbenchtextactive1/15160.1%7
claweval-1-1textactive1/15160.1%1
docvqatext_visionactive1/2840.4%1
eva-bench-accuracyaudio_s2sactive1/382.6%1
eva-bench-experienceaudio_s2sactive1/382.6%1
frontierswe-v2textactive1/15160.1%1
gdpvaltextactive1/15160.1%1
hle-with-tools-protextactive1/15160.1%1
hr-bench-4ktext_visionactive1/2840.4%1
infinitebench-en-mctextactive1/15160.1%1
kimi-code-bench-v2textactive1/15160.1%1
lambadatextactive1/15160.1%1
lmarena-webdevtextactive1/15160.1%1
mathversetext_visionactive1/2840.4%1
mbpptextactive1/15160.1%1
mgsmtextactive1/15160.1%1
omnidocbenchtext_visionactive1/2840.4%1
osworld-2-1-offline-partialtextactive1/15160.1%1
program-bench-almost-at-1textactive1/15160.1%1
racetextactive1/15160.1%1
remote-labor-indextextactive1/15160.1%1
researchrubricstextactive1/15160.1%1
simplevqa-search-tooltext_visionactive1/2840.4%1
swe-bench-multimodaltext_visionactive1/2840.4%1
terminal-benchtextsuperseded1/15160.1%1
ui-localization-averagetext_visionactive1/2840.4%1
v-star-pythontext_visionactive1/2840.4%1
wandr-relaxedtextactive1/15160.1%1
wer-open-asr-v1.15audio_sttactive1/472.1%1
winograndetextactive1/15160.1%4
zerobench-main-with-toolstext_visionactive1/2840.4%1
aci-benchtextactive0/15160%0
adrstextactive0/15160%0
agievaltextactive0/15160%0
anlitextactive0/15160%0
anls-startext_visionactive0/2840%0
arc-etextactive0/15160%3
arc-prizetextactive0/15160%0
arxivmath-june-2026textactive0/15160%0
bbq-ambigtextactive0/15160%0
bbq-disambigtextactive0/15160%0
boolqtextactive0/15160%0
browsecomp-multitextactive0/15160%0
c-evaltextactive0/15160%0
cmmlutextactive0/15160%0
codexgluetextactive0/15160%0
commonsenseqatextactive0/15160%0
copatextactive0/15160%0
crasstextactive0/15160%0
crows-pairstextactive0/15160%0
cruxevaltextactive0/15160%0
deepswe-1-0textactive0/15160%0
droptextactive0/15160%0
dynabenchtextactive0/15160%0
frontierbench-v0-1textactive0/15160%0
frontiercode-extendedtextactive0/15160%0
gaokao-mmtext_visionactive0/2840%4
healthbenchtextactive0/15160%0
healthbench-consensustextactive0/15160%0
healthbench-hardtextactive0/15160%0
healthbench-professionaltextactive0/15160%0
helmtextactive0/15160%0
hhhtextactive0/15160%0
judgelmtextactive0/15160%0
kimi-claw-247-benchtextactive0/15160%0
llm-evaltextactive0/15160%0
llm-judgetextactive0/15160%0
locomotextactive0/15160%0
matharena-apextextactive0/15160%0
mcp-mark-verifiedtextactive0/15160%0
medqatextactive0/15160%0
mls-bench-litetextactive0/15160%0
ms-marcotextactive0/15160%0
mt-benchtextactive0/15160%0
multinlitextactive0/15160%0
naturalqtextactive0/15160%0
nexustextactive0/15160%0
nih-multi-needletextactive0/15160%0
olympiadbenchtextactive0/15160%0
openbookqatextactive0/15160%0
piqatextactive0/15160%0
prometheustextactive0/15160%0
qmsumtextactive0/15160%0
quactextactive0/15160%0
raitextactive0/15160%0
realtalktextactive0/15160%0
realtoxicitytextactive0/15160%0
roundabout-tautext_visionactive0/2840%0
sciqtextactive0/15160%0
social-i-qatextactive0/15160%0
spidertextactive0/15160%0
squadtextactive0/15160%0
terminal-bench-3-0textactive0/15160%0
tooltalktextactive0/15160%0
toxigentextactive0/15160%0
triviaqatextactive0/15160%0
truthfulqatextactive0/15160%0
wildbenchtextactive0/15160%0
winobias-1-2textactive0/15160%0
winobias-2-2textactive0/15160%0
winogendertextactive0/15160%0
zeroscrolls-qualitytextactive0/15160%0

Benchmarks with zero eligible active scores: 71.

2. Capabilities coverage

FieldFilledCoverage
context1,397 / 2,00170%
maxOutputTokens113 / 2,0016%
modalities (input)686 / 2,00134%
modalities (output)1,464 / 2,00173%
toolUse338 / 2,00117%
reasoning291 / 2,00115%
functionCalling345 / 2,00117%
structuredOutputs394 / 2,00120%
codeExecution58 / 2,0013%
promptCaching88 / 2,0014%
batchApi40 / 2,0012%
fineTuning22 / 2,0011%

3. Pricing and host coverage

MetricCountShare
Active API models (≥1 non-deprecated provider join)962—
Subscription-only (excluded from price denominator)2 / 9620%
With maker-endpoint list price200 / 96021%
Active API models with 0 third-party host joins112 / 96212%

4. OpenRouter gap

No committed OpenRouter catalog snapshot in data/seed or data/; only handoff extracts exist under handoffs/llmreference/.

Model Releases by Month

MonthNew Models
2026-1152
2026-10159
2026-0991
2026-0847
2026-0742
2026-0648
2026-0553
2026-0460
2026-0361
2026-0241
2026-0129
2025-1239
2025-1132
2025-1028
2025-0931
2025-0824
2025-0720
2025-0618
2025-0514
2025-0443
2025-0343
2025-0212
2025-0152
2024-1231
2024-1134
2024-1036
2024-0950
2024-0835
2024-0749
2024-0674
2024-0554
2024-0455
2024-0334
2024-0250
2024-0177
2023-1271
2023-1160
2023-1044
2023-0914
2023-0832
2023-0737
2023-0626
2023-0518
2023-0427
2023-0327
2023-022
2023-0112
2022-123
2022-111
2022-1012
2022-091
2022-0713
2022-051
2022-042
2022-035
2022-022
2021-084
2021-053
2021-032
2020-015
2019-111
2019-084
2019-022
2018-102
2018-061

Generated at build time from seed JSON. Last updated: 2026-11-15.