OpenHermes 13B

About

OpenHermes-13B is a large language model (LLM) fine-tuned by Teknium on a dataset primarily containing GPT-4-generated data. Built on the Llama-2-13b-hf base model, it comprises 13 billion parameters and is trained on an open-source dataset with 242,000 entries. The training data includes contributions from sources like GPTeacher, WizardLM, and Airoboros GPT-4. Key features include filtering OpenAI refusals and a context length of 4096 tokens. While demonstrating strong text generation capabilities, especially in benchmarks like GPT4All, it shows limitations in reasoning and logic, as seen in the AGIEval score. Training details are accessible through WandB, highlighting its transparent development process.

Capabilities