# Transfer Learning

> Reuses pretrained model representations for new tasks with less data

- **Category**: Machine Learning
- **Subcategory**: Training Process and Optimization
- **Canonical URL**: https://designpattern.fyi/patterns/transfer-learning/

---

## Description
**Intent**: Leverage models trained on large, related datasets to solve new tasks with much less labeled data by reusing broadly useful representations instead of relearning them.

**Context**: Training a large model from scratch needs large amounts of labeled data and compute that many teams and problems simply don't have. But models already trained on large, related datasets have learned broadly useful representations.

**Solution**: Take a model (or part of one) pretrained on a large source task and reuse it for a related target task with much less data. Use as a fixed feature extractor (freeze pretrained layers, train only new task-specific head) or through fine-tuning (unfreeze some or all layers and continue training at lower learning rate).



## Use Cases
- Computer vision with limited labeled data using ImageNet models
- NLP tasks using pretrained language models
- Specialized domains using related general-purpose models
- Any task with limited data where strong pretrained models exist






## Trade-offs


### Advantages

- Dramatically reduces required labeled data

- Faster training than starting from scratch

- Leverages knowledge from large-scale pretraining

- Often achieves better performance with less data




### Considerations & Drawbacks

- Inherits pretrained architecture's constraints

- Can actively hurt (negative transfer) if domains are too different

- Requires related source and target domains

- Fine-tuning requires careful learning rate management







---
**Reference**: [Original Source](https://github.com/GoogleCloudPlatform/ml-design-patterns)