# Adaptive Compute Allocation

> Spend thinking tokens where they matter — skip them where they don't.

- **Category**: Agentic AI
- **Subcategory**: Reasoning
- **Canonical URL**: https://designpattern.fyi/patterns/adaptive_compute_allocation/

---

## Description
**Intent**: Match compute intensity to problem difficulty at runtime — heavy reasoning for complex tasks, lightweight inference for simple ones.

**Context**: Every token spent on chain-of-thought costs money and adds latency. Most agent workloads are a mix of trivial lookups and genuinely hard reasoning. Treating them all the same wastes budget on easy tasks and under-serves hard ones.

**Solution**: Add a difficulty classifier (rule-based or a cheap LLM call) before each reasoning step. Route to a fast, cheap model for low-complexity queries. Route to a slow, expensive reasoning model (o3, Claude with extended thinking) for high-complexity ones. Optionally use a budget parameter to cap max thinking tokens per task type. See also: test-time-compute-scaling, large-reasoning-model-paradigm.



## Use Cases
- Multi-step agents handling both simple lookups and complex planning in the same pipeline.
- Cost-sensitive production deployments where reasoning token spend needs to be justified per call.
- Any system where latency SLAs differ by task type (real-time chat vs. async batch).






## Trade-offs


### Advantages

- Cuts inference cost significantly — easy tasks don't pay the reasoning tax.

- Reduces latency for the majority of calls that don't need deep thinking.

- Scales gracefully as workload complexity grows without budget blowout.




### Considerations & Drawbacks

- Classifier adds an extra hop — miscategorization sends hard problems to weak models.

- Harder to debug when a task lands in the wrong bucket.

- Requires ongoing calibration as task distribution shifts over time.







---
**Reference**: [Original Source](https://www.agentpatternscatalog.org/patterns/adaptive-compute-allocation/)