# Keyed Predictions

> Passes identifying keys through serving pipeline to match predictions with inputs

- **Category**: Machine Learning
- **Subcategory**: Serving and Operational Resilience
- **Canonical URL**: https://designpattern.fyi/machine_learning/keyed-predictions/

---

## Description
**Intent**: Keep track of which prediction corresponds to which input when running inference on many inputs at once, especially in batch jobs or highly concurrent asynchronous systems.

**Context**: When running inference on many inputs at once in batch jobs or highly concurrent systems, it's easy to lose track of which prediction corresponds to which input, especially if nothing guarantees strict input/output ordering.

**Solution**: Client passes an identifying key alongside each input, and serving function passes that key through unchanged, attaching it to corresponding output. Predictions can be reliably matched back to inputs regardless of processing order, batching, or parallelism.



## Use Cases
- Batch inference pipelines
- Highly concurrent asynchronous serving systems
- Distributed inference jobs
- Any scenario where input/output ordering isn't guaranteed






## Trade-offs


### Advantages

- Reliable key-to-output matching regardless of processing

- Works across batching, parallelism, and out-of-order processing

- Simple design habit with minimal cost

- Enables proper joining with downstream systems




### Considerations & Drawbacks

- Model interface must explicitly support pass-through field

- Small design overhead rather than genuine downside

- Requires key definition and management







---
**Reference**: [Original Source](https://github.com/GoogleCloudPlatform/ml-design-patterns)