Llemma
About
Llemma is a family of open-access large language models (LLMs) designed to specialize in mathematical reasoning. Developed by EleutherAI, these models were initialized using Code Llama weights and trained on the Proof-Pile-2 dataset, comprising a vast 55 billion unique tokens of mathematical and scientific documents. This extensive training allows Llemma models to excel in chain-of-thought mathematical reasoning and effectively use computational tools like Python and formal theorem provers. Available in both 7-billion and 34-billion parameter variants, the Llemma models, particularly the larger one, outperform other LLMs of similar size on a range of mathematical benchmarks. The Llemma project's open-source approach facilitates ongoing research and advancements in mathematical reasoning with LLMs 23.
Specifications(2 models)
| Model | Released | Parameters | Structured Outputs |
|---|---|---|---|
| Llemma 34B | 2023-09 | 34B | No |
| Llemma 7B | 2023-09 | 7B | Yes |
Available From(1 provider)
Pricing
| Model | Provider | Input / 1M | Output / 1M | Type |
|---|---|---|---|---|
| Llemma 7B | OpenRouter | $0.8 | $1.2 | Serverless |
Frequently Asked Questions
- What is Llemma?
- Llemma is a family of open-access large language models (LLMs) designed to specialize in mathematical reasoning. Developed by EleutherAI, these models were initialized using Code Llama weights and trained on the Proof-Pile-2 dataset, comprising a vast 55 billion unique tokens of mathematical and scientific documents. This extensive training allows Llemma models to excel in chain-of-thought mathematical reasoning and effectively use computational tools like Python and formal theorem provers. Available in both 7-billion and 34-billion parameter variants, the Llemma models, particularly the larger one, outperform other LLMs of similar size on a range of mathematical benchmarks. The Llemma project's open-source approach facilitates ongoing research and advancements in mathematical reasoning with LLMs 23.
- How many models are in the Llemma family?
- The Llemma family contains 2 models.
- What is the latest Llemma model?
- The latest model is Llemma 34B, released in 2023-09.
- How much does Llemma cost?
- Llemma models are available at $0.8/1M input tokens through providers like OpenRouter.
