# Quantization-Aware Training (QAT)

> Simulate quantization noise during training so the model learns weights that survive lower precision — better accuracy than PTQ at the same bit-width, especially below INT8.

- **Category**: Language Models
- **Subcategory**: Quantization
- **Canonical URL**: https://designpattern.fyi/patterns/quantization-aware-training/

---

## Description
**Intent**: Train the model to tolerate the precision reduction it will face at inference time rather than applying quantization as a post-hoc surprise.

**Context**: PTQ quantizes weights that were trained at full precision — the model never saw quantization noise during gradient updates. For aggressive targets (INT4, INT2) this mismatch degrades accuracy significantly. QAT bakes quantization into training.

**Solution**: During forward passes, insert fake quantization nodes — rounding operations that simulate the INT4/INT8 grid — on weights and activations. Gradients still flow through fake-quant nodes using the straight-through estimator (treat rounding as identity for backprop). The model learns parameters that already cluster near quantization grid points. At deployment, real quantization is applied to a model that already expects it.



## Use Cases
Aggressive quantization targets (INT4, INT2) where PTQ accuracy loss is unacceptable. Models destined for edge or mobile deployment with fixed-precision hardware. When you have training compute available and need maximum accuracy at a given bit-width.





## Trade-offs


### Advantages

- Significantly better accuracy than PTQ at the same bit-width — especially at INT4 and below

- Robust to distribution shift — the model was trained expecting quantization noise

- Final weights are optimized for the actual inference precision, not retrofitted to it




### Considerations & Drawbacks

- Requires access to training pipeline and data — not a post-hoc transformation

- Training is slower due to fake quantization operations in every forward pass

- Hyperparameter sensitivity increases with more aggressive quantization targets