MetaMath 70B

About

MetaMath-70B is a large language model based on the LLaMA-2 architecture, optimized for mathematical reasoning. It leverages a 4096 context length and requires 138 GB of VRAM for inference. The model's training on the MetaMathQA dataset, which bootstraps mathematical questions, significantly enhances its problem-solving capabilities, achieving an accuracy of 82.3% on benchmarks like GSM8K. Despite its strengths, MetaMath-70B's scalability is limited by computational resource constraints during fine-tuning with QLoRA, and its proficiency is primarily in English. Further research is needed to explore its potential biases and limitations.

Capabilities