# Two-Phase Predictions

> Splits inference into fast local model and heavier cloud model for efficiency

- **Category**: Machine Learning
- **Subcategory**: Serving and Operational Resilience
- **Canonical URL**: https://designpattern.fyi/patterns/two-phase-predictions/

---

## Description
**Intent**: Handle resource constraints or latency requirements by splitting inference into two stages: a small, fast model handles common cases locally, escalating to a larger model only when needed.

**Context**: The best-performing model is often large and compute-hungry, but use cases need predictions under real resource constraints—offline or on-device—or need very low latency for common cases without paying full model cost on every request.

**Solution**: Split inference into two stages: a small, fast, resource-light model handles common case locally or as trigger, escalating to larger, more capable model (often cloud-hosted) only when situation calls for it. Trades small accuracy on easy cases for responsiveness while reserving full power for hard cases.



## Use Cases
- Voice assistants (local wake-word, cloud recognition)
- Offline-capable mobile applications
- Cost-sensitive high-volume systems
- Edge computing with resource constraints






## Trade-offs


### Advantages

- Responsive performance for common cases

- Offline capability when needed

- Cost-effective by not running large model on every request

- Enables edge deployment with cloud fallback




### Considerations & Drawbacks

- Two models to maintain and version together

- Added system complexity

- Risk of first phase silently under-triggering

- Requires careful trigger threshold tuning







---
**Reference**: [Original Source](https://github.com/GoogleCloudPlatform/ml-design-patterns)