Using Dolphin 2.5 Mixtral 8x7B on Together AI
Implementation guide · Dolphin · Cognitive Computations
Quick Start
- 1
- 2Use the Together AI SDK or REST API to call
dolphin-2.5-mixtral-8x7b— see the documentation for request format. - 3
Code Examples
pip install togetherTOGETHER_API_KEYdolphin-2.5-mixtral-8x7bTogether uses "organization/model-name" format, e.g. "meta-llama/Llama-4-Scout-17B-16E-Instruct" or "Qwen/QwQ-32B". See the Together model catalog for the exact ID.
from together import Together
client = Together() # reads TOGETHER_API_KEY from env
response = client.chat.completions.create(
model="dolphin-2.5-mixtral-8x7b",
messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)About Together AI
Platform for running open-source and proprietary LLMs
Together AI is a platform for running open-source and proprietary LLMs with fast serverless and dedicated endpoints at competitive inference pricing.
Pricing on Together AI
| Type | Price (per 1M) |
|---|---|
| Input tokens | $0.60 |
| Output tokens | $0.60 |
Capabilities
No model capability flags are currently sourced.
About Dolphin 2.5 Mixtral 8x7B
The Dolphin 2.5 Mixtral 8x7B is a sophisticated large language model designed primarily for coding tasks, known for its proficiency across diverse programming languages including Kotlin. It utilizes the Mixtral-8x7b architecture and has been fine-tuned on datasets like Dolphin-Coder and MagiCoder, employing qLoRA and Axolotl during training. Featuring a 16k context window for fine-tuning and a base context window of 32k, it offers powerful yet uncensored capabilities, allowing it to handle a wide range of prompts, albeit this introduces ethical considerations. The model is available in various formats on platforms like Hugging Face, catering to different needs with options such as GGUF and GPTQ quantization levels. Despite its strengths, users should be mindful of ethical sensitivities and implement alignment measures when deploying it publicly.