Replit Code 1.5

About

Replit Code V1.5 3B is a powerful causal language model comprising 3.3 billion parameters, designed for code completion. It has been trained on an extensive dataset of one trillion tokens spanning thirty programming languages. Utilizing a customized vocabulary of 32,768 tokens, the model enhances compression and vocabulary coverage. Developed on the MosaicML platform with 128 H100-80GB GPUs, it excels in code generation tasks across diverse languages. Despite thorough data cleansing, it can potentially produce offensive content and is limited to a context size of 4096 tokens. The model is optimal for being fine-tuned for specific applications.

Capabilities