pretrained

Pretraining is the initial phase where a model learns general language representations from large unlabeled corpora via self-supervision, such as next-token prediction. It is what a lab does before instruction or chat tuning; the pretraining data and scale set the ceiling on a model's knowledge and capability, which later tuning shapes but rarely exceeds.