# Toolformer

> Teach the model when and how to call tools through self-supervised training — no human-annotated tool-use traces required.

- **Category**: Agentic AI
- **Subcategory**: Tool Use & Environment
- **Canonical URL**: https://designpattern.fyi/patterns/toolformer/

---

## Description
**Intent**: Generate tool-calling training data automatically so the model learns when not to call tools as well as when to.

**Context**: You're deploying tool use at scale and prompt-based function-calling underperforms in production. You don't have human-labeled tool-use traces and can't afford to create them at scale.

**Solution**: During training, generate candidate tool calls, insert each into a context, and score whether the resulting completion improves (perplexity drop on the gold continuation). Keep helpful insertions as training data; discard the rest. Fine-tune the model to emit tool calls in those positions. The model learns both when to call and — critically — when not to.


## Use Cases
- Tool use is deployed at scale and prompt-based function-calling underperforms.
- Human-labeled tool-use traces are unavailable or unaffordable.
- Self-supervised data can be generated by inserting candidate tool calls and scoring them.





## Trade-offs


### Advantages

- No human-labeled tool-call data required.

- Model learns when not to call tools, not just when to — that's the hard part.




### Considerations & Drawbacks

- Training pipeline complexity is significant.

- Tool surface drift between training time and serving time breaks the learned behavior.

- Historically significant but largely superseded by RLHF-tuned tool use in frontier models.







---
**Reference**: [Original Source](https://www.agentpatternscatalog.org/patterns/toolformer/)