Parameter-Efficient Fine-Tuning

Definition

PEFT (Parameter-Efficient Fine-Tuning) encompasses methods to adapt pre-trained LLMs for specific tasks by updating a small subset of parameters, avoiding full fine-tuning's compute costs. Approaches like adapters or prompt tuning inject task-specific modules, enabling scalability to massive models.