Hierarchical Tool Selection
Organize tools into a category tree so the agent picks a branch first, then a specific tool — keeping selection accurate even with hundreds of tools.
Intent & Description
🎯 Intent
Stop drowning the model in tool lists by structuring the catalog into a navigable tree.
📋 Context
Your agent has access to dozens or hundreds of tools — every API your company exposes across billing, identity, scheduling, search, code, and files. Dumping all of them in the system prompt blows the context window and tanks tool selection accuracy.
💡 Solution
Group tools into named categories. At the top level the agent sees only category names with one-line descriptions. After picking a category, it sees the tools in that branch. Permissions can scope per branch. For very large catalogs, nest the tree further. Cost: one extra decoding step. Saving: full tool descriptions only for the chosen branch.
Real-world Use Case
- Tool catalog exceeds ~30 tools.
- Tools naturally cluster into domain categories.
- Permissions or ownership can scope per category.
Source
📌 TL;DR
Too many tools? Organize them into a tree. Agent picks a category, then picks a tool. Selection stays accurate even at scale.
Advantages
- Token cost stays bounded as the catalog grows — you only pay for the chosen branch.
- Selection accuracy improves because the model picks from few items at each level.
- Permissions and ownership map onto the tree naturally.
Disadvantages
- One extra step per call adds latency and another decoding decision.
- Tools that span two domains need duplication or an awkward compromise.
- Wrong top-level pick is a dead-end — the right tool is in a different branch.