SOLAR 0 70B

About

SOLAR-0-70b, developed by Upstage, is an advanced LLM derived from Llama 2, possessing 70 billion parameters and enhanced for processing lengthy sequences over 10,000 tokens using a rope_scaling option. It implements a transformer architecture and is trained on diverse datasets like Orca-style and Alpaca-style, excelling in benchmarks such as ARC-Challenge, HellaSwag, MMLU, and TruthfulQA with high H4 scores. Despite its prowess, it faces limitations, including potential biases and significant computational demands, which may impact accessibility. Various quantized versions, including an 8-bit variant, aim to enhance efficiency, and the model is accessible on platforms like Poe, also offering an API for broader application.

Capabilities

Rankings