Consistency-Guided Decoding with Proof-Driven Disambiguation for Three-Way Logical Question Answering

Researchers present CGD-PD, a test-time decoding method that improves large language models' performance on three-way logical question answering (True/False/Unknown) by enforcing negation consistency and resolving epistemic uncertainty through targeted entailment probes. The approach achieves up to 16% relative accuracy improvements on the FOLIO benchmark while reducing spurious Unknown predictions.

This research addresses fundamental weaknesses in how large language models handle formal logical reasoning tasks. The two failure modes identified—negation inconsistency and epistemic Unknown—represent systematic vulnerabilities where LLMs fail at tasks requiring deterministic logical constraints. When a model predicts different answers for a hypothesis and its negation, it violates basic logical principles that should be mathematically guaranteed. The epistemic Unknown problem reveals that models often retreat to hedging when uncertain, even when premises logically entail a definitive answer.

The CGD-PD solution operates as a lightweight post-processing layer, making it practically deployable without retraining models. By mechanically negating hypotheses and checking consistency, the method forces logical coherence. The proof-driven disambiguation component is particularly elegant—rather than forcing a binary choice, it strategically deploys entailment probes to gather additional evidence, requiring only 4-5 model calls on average. This efficiency matters for practical deployment in reasoning-heavy applications.

For the AI development community, this work demonstrates that systematic error patterns in LLMs can be addressed through clever inference-time techniques. The improvements across frontier models suggest the approach generalizes beyond any single architecture. This has implications for AI systems deployed in domains requiring formal reasoning—legal analysis, mathematical problem-solving, and knowledge-base querying. The relative 16% accuracy gains are substantial enough to influence model selection decisions for reasoning-critical applications. As enterprises increasingly rely on LLMs for logical tasks, techniques that reliably enforce consistency constraints become essential infrastructure, not optional enhancements.

• →CGD-PD fixes negation inconsistency by querying models on both a hypothesis and its negation, then enforcing logical consistency in post-processing.
• →Proof-driven disambiguation uses targeted entailment probes to resolve Unknown predictions, requiring only 4-5 model calls on average.
• →The method achieves up to 16% relative accuracy improvements on FOLIO's first-order logic benchmarks across multiple frontier LLMs.
• →The approach operates as a test-time layer without requiring model retraining, making it practical for immediate deployment.
• →Systematic logical inconsistencies in LLMs can be addressed through inference-time techniques rather than architectural changes.