GPT-JT
About
GPT-JT is a series of large language models that originate from a fine-tuned version of EleutherAI's GPT-J 6B model. These models utilize a decentralized training algorithm, allowing them to operate efficiently despite using a network with relatively slow interconnect speeds. This novel approach optimizes the use of diverse hardware resources. The training process integrates various open-source methodologies and datasets, including Google Research's UL2 training objective, Chain-of-Thought prompting, and datasets like BigScience's Public Pool of Prompts (P3) and AllenAI's Natural Instructions (NI). As a result, GPT-JT models exhibit strong performance on classification benchmarks and are known to outperform models with significantly larger parameters. Importantly, these models are available as open-source, inviting community participation for further enhancements145.
Specifications(3 models)
| Model | Released | Parameters | Structured Outputs |
|---|---|---|---|
| GPT-JT 6B V0 | 2023-03 | 6B | No |
| GPT-JT 6B V1 | 2023-03 | 6B | No |
| GPT-JT Moderation 6B | 2023-03 | 6B | Yes |
Available From(2 providers)
Pricing
| Model | Provider | Input / 1M | Output / 1M | Type |
|---|---|---|---|---|
| GPT-JT Moderation 6B | Together AI | $0.2 | $0.2 | Serverless |
Frequently Asked Questions
- What is GPT-JT?
- GPT-JT is a series of large language models that originate from a fine-tuned version of EleutherAI's GPT-J 6B model. These models utilize a decentralized training algorithm, allowing them to operate efficiently despite using a network with relatively slow interconnect speeds. This novel approach optimizes the use of diverse hardware resources. The training process integrates various open-source methodologies and datasets, including Google Research's UL2 training objective, Chain-of-Thought prompting, and datasets like BigScience's Public Pool of Prompts (P3) and AllenAI's Natural Instructions (NI). As a result, GPT-JT models exhibit strong performance on classification benchmarks and are known to outperform models with significantly larger parameters. Importantly, these models are available as open-source, inviting community participation for further enhancements145.
- How many models are in the GPT-JT family?
- The GPT-JT family contains 3 models.
- What is the latest GPT-JT model?
- The latest model is GPT-JT 6B V0, released in 2023-03.
- How much does GPT-JT cost?
- GPT-JT models are available at $0.2/1M input tokens through providers like Together AI.
