Concepts & capability filters
pretrained
Pretraining is the initial phase where a model learns general language representations from large unlabeled corpora via self-supervision, such as next-token prediction.
- Category
- Not classified
- Difficulty
- Not classified
- Aliases
- None tracked
- Last reviewed
- 2026-07-02
Key facts
- It is what a lab does before instruction or chat tuning; the pretraining data and scale set the ceiling on a model's knowledge and capability, which later tuning shapes but rarely exceeds.