GPT-NeoX Models by EleutherAI
0 models
Last refreshed 2026-10-10. Next refresh: weekly.
About
GPT-NeoX-20B is EleutherAI’s 20-billion-parameter autoregressive language model, announced on February 2, 2022 and trained with EleutherAI’s GPT-NeoX library on GPUs provided by CoreWeave. It was trained on the Pile, using English-language text. Its architecture intentionally resembles GPT-3 and is almost identical to GPT-J-6B, with 44 layers, rotary position embeddings, and a 2,048-token sequence length. EleutherAI built it primarily for research and as a base for fine-tuning: it was not tuned for chat-style use and is not meant to be deployed as-is.
Decision facts
- Best fit
- General model comparison
- Capability starting point
- Not tracked
- Lowest tracked input
- Not tracked
- Closest related family
- GPT-Neo
