ForEx: A Formal Verification Framework for Explainable Reasoning in Logical Fallacy Detection and Annotation

Researchers introduce ForEx, a framework that translates LLM-generated explanations into formal logic (Lean4) to verify whether reasoning actually supports predicted labels on logical fallacy detection tasks. The study reveals a critical gap: while 90% of LLM outputs can be formally verified as logically sound, agreement with human annotations remains around 20%, exposing that formal correctness differs fundamentally from label accuracy.

ForEx addresses a fundamental blind spot in LLM evaluation: the assumption that correct predictions stem from sound reasoning. Traditional benchmarks measure label accuracy alone, overlooking whether models genuinely understand the logical structure of their outputs. This research demonstrates that high formal verification rates can coexist with low human agreement, suggesting LLMs may produce internally consistent but fundamentally misaligned explanations.

The framework emerges from growing concerns about LLM transparency and trustworthiness. As these models are deployed in high-stakes domains—legal reasoning, scientific analysis, policy evaluation—understanding whether their justifications are logically derivable becomes critical. Current evaluation metrics fail to distinguish between lucky predictions and principled reasoning, creating false confidence in model capabilities.

For AI developers and researchers, ForEx provides a new evaluation lens that shifts focus from surface-level accuracy to machine-verifiable reasoning chains. Organizations building explainable AI systems need mechanisms that prove conclusions logically follow from stated premises. The 70-point gap between formal verification (90%) and human agreement (20%) signals that models may be producing technically sound logical chains that don't actually address the original problem, a distinction with serious implications for deployment safety.

Looking ahead, this work likely influences how AI systems are evaluated in regulated industries. Formal verification could become a requirement for high-stakes applications, pushing development toward more rigorous, provably-sound reasoning. The broader trend toward interpretability and formal methods in AI will likely accelerate as organizations recognize that prediction accuracy alone provides insufficient assurance.

• →ForEx framework translates LLM explanations into Lean4 formal logic to verify reasoning validity independently from label correctness.
• →90% of LLM outputs pass formal verification while only 20% align with human annotations, revealing a critical gap between logical soundness and task accuracy.
• →The LLM Argument Verification Matrix separates prediction outcomes from formal reasoning status, exposing limitations of label-only evaluation metrics.
• →Formal verification of AI reasoning chains may become essential for high-stakes applications requiring provably sound decision-making.
• →Research indicates LLMs can generate internally consistent but fundamentally misaligned explanations, a crucial distinction for deployment safety.