The Shelf · Signal Library
No. 204

Accepted but wrong: silent failures of LLM code repairs after API migrations

StrongEvidence grade
Nov 6, 2026Shelved
Agentic AISection
sha256:2c65179f392d…Fingerprint

a preregistered study of 73 tasks across 60 documented CPython, Node.js, and SQLite changes, executed on the real runtimes against withheld contracts: an all-assertions gate on typical inputs released 0.410 of wrong repairs — code that runs cleanly and returns wrong results. One edge input written from the release note cut that to 0.065. Directly applicable to any fleet release check that only tests typical inputs.

Read the source — arXivBrowse the Shelf

Cite as — Sawdust Shelf v2026-11-06.1 · No. 204 · fingerprint sha256:2c65179f392d…
Share image rendered from the living shelf — the number, the grade and the fingerprint travel with the card.