Retrieval-Augmented Tool Selection for Long-Horizon Language Agents: Choosing from thousands of tools without placing them all in the prompt
Lena Hartwell, Tomás Ibarra, Marcus Tan
Language agents that act through external tools are limited by the size of their prompt: listing hundreds of tool descriptions crowds out the task and degrades accuracy. We study tool selection as a retrieval problem and propose a two-stage selector that (i) retrieves a short list of candidate tools using an embedding of the evolving task state and (ii) lets the model re-rank candidates with a single constrained call. Across a synthetic suite of 4,000 tools and three long-horizon task families, the selector keeps task success within 2.1 points of an oracle that sees only the correct tools, while reducing prompt length by 87%. Ablations show that conditioning retrieval on the task state, rather than on the initial instruction alone, accounts for most of the gain. We discuss failure cases, including tools with overlapping descriptions, and release the evaluation harness.