Interactive Critique-Revision Training for Reliable Structured LLM Generation

Researchers propose DPA-GRPO, a novel training method for large language models that improves structured decision-making by using a generator-verifier framework where one model produces outputs and another validates them through safety assurance cases. The method demonstrates improved accuracy on tax calculation benchmarks and addresses the challenge of ensuring LLM outputs are locally correct, globally consistent, and auditable.

DPA-GRPO represents a meaningful advance in making language models reliable for high-stakes structured tasks where accuracy is non-negotiable. The core innovation lies in the paired-action framework: instead of having a single model generate outputs or relying on heuristic refinement approaches, the system employs two specialized roles—a generator that proposes and can revise answers, and a verifier that either approves silently or challenges with documented evidence. This mirrors real-world quality assurance processes in compliance and financial domains.

The theoretical contribution matters significantly. By analyzing the underlying game dynamics, the researchers prove that their KL-regularized approach eliminates profitable unilateral deviations, meaning neither the generator nor verifier can improve by deviating from the learned policy unilaterally. This addresses a fundamental weakness in existing LLM refinement methods that lack formal guarantees against gaming behavior.

For practical applications, the experimental results on TaxCalcBench—a domain where computational accuracy directly affects stakeholder liability—show measurable improvements over baseline approaches. The method increases correct silent acceptance (reducing unnecessary interventions) while reducing missed errors and improving calibrated revision behavior. This balance is critical; over-intervention wastes resources while under-intervention misses genuine problems.

The implications extend beyond tax calculation to any structured workflow requiring auditability: compliance checking, maintenance reporting, and regulatory documentation. As enterprises increasingly deploy LLMs in sensitive domains, having a mathematically grounded approach to verification could accelerate adoption in regulated industries. The next phase involves testing scalability across larger models and real-world implementation where downstream consequences of errors are substantial.

• →DPA-GRPO uses a generator-verifier framework where verifiers challenge outputs with documented safety assurance cases rather than simple approval/rejection signals.
• →Theoretical analysis proves the method eliminates profitable unilateral deviations between generator and verifier roles under standard optimization assumptions.
• →Experiments on tax calculation tasks demonstrate improvements in accuracy, reduced missed errors, and better-calibrated revision behavior compared to baseline RL approaches.
• →The paired-action training approach is designed to work across different model sizes, with successful results on Qwen3-4B and Qwen3-8B architectures.
• →The method addresses real-world needs for auditability and compliance in structured decision-making workflows where error consequences are significant.