Concepts & capability filters
training_technique
Distillation
Distillation transfers knowledge from a large, complex teacher model to a smaller student model by training the student to mimic the teacher's outputs or intermediate representations, creating efficient deployable versions.
- Category
- training_technique
- Difficulty
- Not classified
- Aliases
- None tracked
- Last reviewed
- 2026-07-02
Key facts
- It reduces model size and inference cost while retaining much of the performance.