# Mixed-Precision Quantization > Assign different precision to different layers based on sensitivity analysis — INT8 where it matters, INT4 where it's tolerant, getting compression without uniform accuracy loss. - **Category**: Language Models - **Subcategory**: Quantization - **Canonical URL**: https://designpattern.fyi/patterns/mixed-precision-quantization/ --- ## Description **Intent**: Blanket INT4 degrades accuracy unevenly — some layers are sensitive, others tolerant. Mixed precision puts bits where they actually matter. **Context**: Sensitivity analysis consistently shows that some layers (typically early attention projections, final output layers) suffer significantly from INT4 quantization while mid-stack FFN layers tolerate it. Uniform quantization wastes precision in tolerant layers and loses accuracy in sensitive ones. **Solution**: Run layer-sensitivity analysis on a calibration dataset — independently quantize each layer to INT4 and measure accuracy impact. Assign INT8 to high-sensitivity layers, INT4 to low-sensitivity ones. The average compression ratio approaches uniform INT4, but accuracy approaches uniform INT8. AutoGPTQ, SpQR, and SqueezeLLM implement automated sensitivity-based mixed-precision assignment. ## Use Cases Maximum compression at a given accuracy target. Production deployments where uniform INT4 degrades key capabilities but uniform INT8 is too memory-expensive. Tuning the compression-accuracy frontier without retraining. ## Trade-offs ### Advantages - Better accuracy than uniform INT4 at comparable average compression ratio - Compression is concentrated in tolerant layers — no wasted precision on insensitive ones - Automated sensitivity analysis removes the need for manual layer inspection ### Considerations & Drawbacks - More complex deployment toolchain than uniform quantization — multiple precision levels to manage - Mixed-precision kernels may not be supported on all target hardware - Sensitivity analysis requires a representative calibration dataset and adds evaluation cost