Tool Loadout Hot-Swap
Mutating the agent's available tools mid-task — invalidating the KV-cache and confusing a model that conditioned on a different tool set.
Intent & Description
🎯 Intent
Dynamically adding or removing tool definitions during a running task to keep the tool set lean as the task evolves.
📋 Context
The team interprets “don’’t expose all tools” as “add tools as needed, remove them when done.” Sounds like good hygiene. In practice, every mutation blows the KV-cache and leaves a model that was conditioning on tools that no longer exist — producing calls to removed tools and contradicted reasoning across the run.
💡 Solution
Define the tool palette once at run start and keep it stable for the entire run. To restrict what the model can call in a given state, mask the tool-name token during decoding — don’’t remove the definition. See tool-loadout (pick the subset at run start, not mid-run), tool-search-lazy-loading, prompt-caching.
Real-world Use Case
- Never. The cache invalidation and contradicted conditioning are not worth the apparent flexibility.
- Pick the tool loadout at run start (tool-loadout) and hold it stable across the entire run.
- Constrain tool availability by masking logits during decoding, not by mutating the registry.
Source
📌 TL;DR
Lock the tool palette at run start — use logit masking to restrict tool availability mid-task, not registry mutations.
Disadvantages
- KV-cache is invalidated on every tool mutation — latency and cost spike for all subsequent turns
- The model emits calls to tools removed mid-run or not yet added at earlier turns
- Earlier conditioning tokens contradict the present tool registry
- Debugging is painful when the apparent tool set changes within a single run