The Shelf · Signal Library
No. 175

DeliverBench: task skill does not compose into project delivery

StrongEvidence grade
Nov 3, 2026Shelved
Agentic AISection
sha256:2ff4eb89cd9d…Fingerprint

agents are usually benchmarked one task at a time, but professional work pays off only if the whole multi-stage package holds. DeliverBench chains 11 expert-reconstructed projects (financial due diligence, legal drafting, drug safety reporting) through 150 subtasks and 2–15 rounds of new materials and client feedback, with agents inheriting their own earlier errors. Across seven model–harness systems the best scores only 65.0/100 on final delivery and passes 50.0% of subtasks at a 90-point threshold — the failures are mostly failures of carrying work forward coherently.

Read the source — arXivBrowse the Shelf

Cite as — Sawdust Shelf v2026-11-05.3 · No. 175 · fingerprint sha256:2ff4eb89cd9d…
Share image rendered from the living shelf — the number, the grade and the fingerprint travel with the card.