Pythia 14M

About

Pythia 14M is a large language model developed by EleutherAI, utilizing the GPT-NeoX architecture. It consists of 14 million parameters and can handle a context length of up to 2048 tokens, which allows it to be efficiently used with just 0.1 GB of VRAM. Designed for generating English text, this model excels at tasks such as sentence generation and completion. However, Pythia 14M is not ideal for tasks that require extensive domain knowledge or involve non-English text due to its smaller size, which can impact performance in comparison to larger models. The model's training incorporated datasets like web-scraped educational content and filtered Common Crawl data.

Capabilities