The Shelf · Signal Library
No. 140

rTaste: research agents generate well but rank poorly

StrongEvidence grade
Oct 30, 2026Shelved
Agentic AISection
sha256:bcf29c9ba17b…Fingerprint

six frontier models, working alone or as teams, proposed 415 minimum-viable experiments from seven recent NLP papers; a separate executor ran every one, including the last-ranked. 194 proposals were judged sound, valuable and new, but the agents' first choices were not shown to beat simply running the experiment listed first, their outcome forecasts were no more accurate than a fresh instance of the same model given only the paper, and teams cost more without a clear overall improvement in proposal quality.

Read the source — arXivBrowse the Shelf

Cite as — Sawdust Shelf v2026-11-05.3 · No. 140 · fingerprint sha256:bcf29c9ba17b…
Share image rendered from the living shelf — the number, the grade and the fingerprint travel with the card.