From Fact Overwriting to Knowledge Evolution: Causal Editing via On-Policy Self-Distillation

Researchers present CODE, a novel approach to knowledge editing in large language models that replaces fact overwriting with causal reasoning. By embedding causal narratives and on-policy distillation into model parameters, CODE reduces self-refutation rates from 95.6% to 1.8%, enabling LLMs to evolve knowledge coherently rather than storing isolated facts.

Knowledge editing has emerged as a critical capability for maintaining accurate, up-to-date language models without full retraining. Traditional approaches treat LLMs as databases, injecting new facts directly—a method that creates internal contradictions where models simultaneously hold old and new beliefs, leading to self-refutation. This research identifies the root cause: pre-trained logical structures resist isolated fact injection, forcing models to explicitly negate updates.

The work builds on foundational research in model interpretability and causal reasoning, responding to a known limitation in KE methods. As LLMs increasingly serve as knowledge bases for production systems, maintaining consistency becomes commercially and ethically critical. The paper's key innovation couples causal bootstrapping with asymmetric on-policy distillation, essentially teaching models not just facts but the causal logic underlying knowledge transitions.

For practitioners deploying LLMs in high-stakes domains—law, medicine, finance—this addresses a genuine operational challenge. Current knowledge editing methods risk deploying models with internal contradictions that could produce unreliable outputs. The dramatic reduction in self-refutation rates (from 95.6% to 1.8%) suggests CODE could enable safer, more reliable model updates in production environments.

The research focuses on foundational AI capabilities rather than immediate market applications. However, improved knowledge editing directly impacts enterprise adoption of LLMs. Organizations considering large-scale deployments gain a technical pathway for maintaining model accuracy post-deployment, reducing retraining costs and operational friction. Future work should examine computational overhead and scalability across larger model families.

• →Traditional knowledge editing causes 95.6% self-refutation rates due to fractured logical topologies; causal-grounded approaches reduce this to 1.8%
• →CODE couples causal bootstrapping with on-policy distillation to embed knowledge evolution directly into model parameters rather than injecting isolated facts
• →Epistemic dissonance—where models must explicitly negate injected updates—stems from structural design flaws, not algorithmic noise
• →Multi-hop reasoning accuracy reaches 83.5% with CODE, indicating sustained logical consistency across knowledge transitions
• →Improved knowledge editing enables safer, more reliable LLM deployments in production systems without full retraining cycles