Llemma 34B

About

Llemma 34B is a large language model developed by EleutherAI, specifically for mathematical reasoning. It builds on Code Llama architecture and has 34 billion parameters. The model is advanced in solving mathematical problems, leveraging its training on Proof-Pile II, a dataset with 55 billion tokens of mathematical and scientific content. It excels in tool use with Python and theorem provers and showcases strong chain-of-thought reasoning by breaking down complex problems into steps. Despite its capabilities, it faces limitations in generalization outside its training scope and requires substantial computational resources 1.

Capabilities