# Heuristic Benchmark

> Compares model performance against simple non-ML baselines for meaningful evaluation

- **Category**: Machine Learning
- **Subcategory**: Trust, Evaluation and Responsible AI
- **Canonical URL**: https://designpattern.fyi/patterns/heuristic-benchmark/

---

## Description
**Intent**: Provide a meaningful reference point to know whether an ML system is genuinely worth deploying, or worth its added cost and complexity over the status quo.

**Context**: A raw ML metric like 82% accuracy or an RMSE of 4.2 is hard to interpret in isolation. Both practitioners and stakeholders need to know whether the model is actually good compared to simpler alternatives.

**Solution**: Compare model performance against a simple, easily understood, non-ML baseline that represents what we would do without ML—always predicting historical average, applying current manual business rule, or naive persistence forecast. This gives interpretable reference point so actual value the model adds is clear.



## Use Cases
- Early project evaluation to decide if ML is needed
- Ongoing validation that ML still worth its cost
- Stakeholder communication about model value
- Any situation where ML benefits need justification






## Trade-offs


### Advantages

- Provides interpretable reference point for ML performance

- Makes value assessment clear to technical and non-technical stakeholders

- Essentially no downside—small effort to define and maintain

- Helps avoid deploying ML when simple solutions suffice




### Considerations & Drawbacks

- Requires implementing and maintaining heuristic baseline

- May need periodic re-evaluation as environment changes

- Could discourage ML if baseline is already strong

- Requires careful metric selection for fair comparison







---
**Reference**: [Original Source](https://github.com/GoogleCloudPlatform/ml-design-patterns)