Smaug 34B
Smaug 34B is a released long context model with open-source and 200K context; evaluate it while provider pricing coverage matures.
- Family
- Smaug
- Released
- 2023-12-09
- Context
- 200K
- Parameters
- 34B
- Architecture
- Decoder Only
- Specialization
- general
- Openness
- Open source
- License
- Apache 2.0OSI-approvedCommercial use: permitted
- Weights
- Unknown
- Code
- Unknown
- Training
- Fine-tuned
No tracked provider pricing is available yet.
About
Smaug-34B-v0.1 is an advanced large language model developed by Abacus.AI, evolving from the Bagel-34B-v0.2 with 34.4 billion parameters. It employs the LlamaForCausalLM architecture and is accessible via Hugging Face. This model is fine-tuned using DPO-Positive, a novel technique enhancing conventional optimization methods, leading to improved outcomes on tasks with minimal data diversity. It leverages datasets like ARC and HellaSwag, but concerns exist regarding overfitting due to dataset similarities with benchmark tests. Smaug-34B-v0.1 scores an impressive average of 77.29% on several benchmarks, although its performance varies depending on contamination levels, and the absence of DPOP code may hinder reproducibility efforts.
Providers
No active provider routes are tracked for this model yet.
Capabilities
No model capability flags are currently sourced.
Benchmark peer barsfor Long context
No task-mapped benchmark peers are available for this model yet.
Migration checks
No linked migration route is available for this model yet.