The Shelf · Signal Library
No. 163

Multi-tool retrieval is the quiet failure point in agent stacks

StrongEvidence grade
Nov 2, 2026Shelved
Agentic AISection
sha256:f279cce2a43d…Fingerprint

an agent that needs several tools fails outright if the retriever misses any one of them, yet retrievers are mostly evaluated on single-tool queries. Measured across AppWorld and MCP-Bench, no untrained structure making at most one LLM call per query completes more than 10% of AppWorld queries at k=16; a set-level retriever trained on just 75–90 in-domain queries reaches .33 on AppWorld and .58 on MCP-Bench with zero LLM calls, while an LLM-navigated hierarchy needs a median of 31–38 calls and about five minutes per query to do worse.

Read the source — arXivBrowse the Shelf

Cite as — Sawdust Shelf v2026-11-05.3 · No. 163 · fingerprint sha256:f279cce2a43d…
Share image rendered from the living shelf — the number, the grade and the fingerprint travel with the card.