GatorTron

About

The GatorTron family of large language models (LLMs) includes several models meticulously trained on extensive datasets comprising clinical notes and other textual data. This development is a collaborative endeavor between the University of Florida (UF) and NVIDIA 245. The core models, known as GatorTron, are encoder-only architectures tailored for natural language understanding tasks such as named entity recognition (NER), relation extraction (RE), and machine reading comprehension (MRC), yet they are not designed for generative tasks 6. Versions of these models differ in parameters: GatorTron-base with 345 million, GatorTron-medium with 3.9 billion, and GatorTron-large with 8.9 billion parameters 24. Trained on over 90 billion words, including de-identified clinical notes from UF Health, PubMed articles, and Wikipedia, these models have shown superiority over existing biomedical and clinical transformer models in various NLP tasks 7. Additionally, GatorTronGPT, a generative LLM, utilizes a GPT-3 architecture and is trained with a dataset of 277 billion words, including clinical and general English text, to produce synthetic clinical text for training models like GatorTronS 389. The models are available on Hugging Face, further demonstrating their capabilities and accessibility 24.