DAG-Math: Graph-of-Thought Guided Mathematical Reasoning in LLMs

Researchers introduce DAG-Math, a new framework for evaluating mathematical reasoning in Large Language Models that models Chain-of-Thought as rule-based processes over directed acyclic graphs. The framework includes a 'logical closeness' metric that reveals significant differences in reasoning quality between LLM families, even when final answer accuracy appears comparable.

• →DAG-Math framework models Chain-of-Thought reasoning as rule-based stochastic processes over directed acyclic graphs with intermediate derivation states.
• →New 'logical closeness' metric evaluates how well LLM reasoning adheres to structured mathematical rules beyond simple pass/fail metrics.
• →Analysis reveals statistically significant differences in reasoning fidelity between LLM families even when final answer accuracy is similar.
• →The framework bridges the gap between free-form Chain-of-Thought and formal proof systems for better LLM evaluation.
• →Benchmark and code are publicly available to enable further research in mathematical reasoning evaluation.