DeltaLogic: Minimal Premise Edits Reveal Belief-Revision Failures in Logical Reasoning Models

Researchers introduce DeltaLogic, a new benchmark that tests AI models' ability to revise their logical conclusions when presented with minimal changes to premises. The study reveals that language models like Qwen and Phi-4 struggle with belief revision even when they perform well on initial reasoning tasks, showing concerning inertia patterns where models fail to update conclusions when evidence changes.

• →DeltaLogic benchmark exposes a critical gap between AI models' initial reasoning accuracy and their ability to revise beliefs when premises change.
• →Qwen3-1.7B achieved 66.7% initial accuracy but only 46.7% revision accuracy, demonstrating significant inertial bias.
• →Models exhibit troubling tendency to stick with original conclusions even when new evidence should prompt revision.
• →Phi-4-mini-instruct performed better with 95% initial and 85% revision accuracy but still showed instability issues.
• →The research highlights that logical competence under fixed conditions doesn't guarantee proper belief updating in dynamic environments.