# Ensemble

> Combines predictions from multiple models to improve accuracy and robustness

- **Category**: Machine Learning
- **Subcategory**: Problem Framing and Model Structure
- **Canonical URL**: https://designpattern.fyi/machine_learning/ensemble/

---

## Description
**Intent**: Overcome the bias and variance limitations of single models by combining multiple models, where uncorrelated errors tend to average out for better overall performance.

**Context**: Any single model carries the limitations of its own algorithm and training run. On complex or noisy problems, one model often isn't as accurate or robust as combining several diverse approaches.

**Solution**: Combine predictions of multiple models using bagging (train many models on bootstrapped samples and average outputs), boosting (train models sequentially, each correcting previous errors), or stacking (train a meta-model on base model outputs). Uncorrelated errors average out, producing more accurate and stable results.



## Use Cases
- Competitions or high-stakes predictions where accuracy matters most
- Reducing variance and overfitting in noisy datasets
- Combining diverse model types (tree-based, neural, linear)
- Systems where robustness matters more than latency






## Trade-offs


### Advantages

- Typically more accurate than single best model

- More robust through error averaging

- Can combine strengths of different algorithm types

- Reduces both bias and variance through different approaches




### Considerations & Drawbacks

- Higher compute cost at training and serving time

- Increased serving complexity and latency

- Harder to explain individual predictions

- Requires validation that ensemble actually helps







---
**Reference**: [Original Source](https://github.com/GoogleCloudPlatform/ml-design-patterns)