# Full Fine-Tuning

> Update every model parameter on task-specific data — maximum adaptation capacity, maximum compute and memory cost.

- **Category**: Language Models
- **Subcategory**: Fine-Tuning
- **Canonical URL**: https://designpattern.fyi/patterns/full-fine-tuning/

---

## Description
**Intent**: Fully specialize a pre-trained model to a new domain or behavior by updating every weight — the highest-capacity adaptation method, used when lighter approaches fall short.

**Context**: Pre-trained models encode general knowledge. For significant domain shift (medical imaging reports, legal contracts, financial filings) or deep behavioral change, partial fine-tuning methods may not adapt deep layers sufficiently. Full fine-tuning changes everything.

**Solution**: Initialize from a pre-trained checkpoint. Run standard supervised training on task-specific data with a small learning rate (1e-5 to 5e-5) and linear warmup. All parameters receive gradient updates. Use gradient checkpointing to manage memory (training requires 3-4x the inference memory footprint). Mix in general data to prevent catastrophic forgetting of base capabilities.



## Use Cases
Significant domain shift from pre-training distribution. Tasks requiring deep behavioral change beyond style adaptation. When LoRA or other PEFT methods have been tried and quality is still insufficient.





## Trade-offs


### Advantages

- Maximum adaptation capacity — every parameter can change to fit the new domain

- No architectural constraints — full expressive power of the model is available

- Produces a fully portable standalone checkpoint that doesn't depend on a base model




### Considerations & Drawbacks

- Highest compute cost — requires full training infrastructure and significant GPU-hours

- Catastrophic forgetting — general capabilities degrade without careful data mixing

- Training memory is 3-4x inference memory — requires hardware many teams don't have