# Batch Serving

> Precomputes predictions asynchronously using distributed data processing

- **Category**: Machine Learning
- **Subcategory**: Serving and Operational Resilience
- **Canonical URL**: https://designpattern.fyi/patterns/batch-serving/

---

## Description
**Intent**: Handle prediction needs that don't require real-time freshness by precomputing predictions asynchronously and serving them via simple lookup, avoiding the overhead of online serving.

**Context**: Not every prediction need is real-time. Scoring an entire customer database overnight doesn't need low-latency per-request serving, and using online serving for bulk workloads wastes overhead without benefit.

**Solution**: Use distributed data processing infrastructure to run trained model inference across large volume of inputs asynchronously, writing predictions to a fast-lookup store. Requests are then served by simple lookup against precomputed results rather than live computation.



## Use Cases
- Large-scale recommendation systems with periodic updates
- Customer database scoring overnight
- Reporting and analytics workloads
- Any latency-tolerant prediction needs






## Trade-offs


### Advantages

- Efficient for large-scale, latency-tolerant workloads

- Leverages existing distributed data processing infrastructure

- Simple serving via fast lookup

- Cost-effective for bulk prediction needs




### Considerations & Drawbacks

- Predictions can go stale between batch runs

- Wrong choice when real-time freshness is required

- Requires scheduling and orchestration infrastructure

- Needs fallback for new entities not in batch







---
**Reference**: [Original Source](https://github.com/GoogleCloudPlatform/ml-design-patterns)