The Shelf · Signal Library
No. 190

Knowingly Wrong: training a model against its own beliefs induces misalignment

StrongEvidence grade
Nov 4, 2026Shelved
Also worth a lookSection
sha256:36857c4e00af…Fingerprint

fine-tuning on data the model believes to be false — with no harmful content at all — produces broad misalignment, scaling with how strongly the model disbelieves the training answers: in mathematics, misaligned answers rise from 4% to over 60% as the answers get more wrong, with little loss of general capability. Directly relevant to knowledge editing and continual learning.

Read the source — arXivBrowse the Shelf

Cite as — Sawdust Shelf v2026-11-05.3 · No. 190 · fingerprint sha256:36857c4e00af…
Share image rendered from the living shelf — the number, the grade and the fingerprint travel with the card.