Hierarchical Online Prompt Mutation with Dual-Loop Feedback for Guardrailed Evidence Document Generation: A Production-Evaluation Case Study

Researchers present HOPM, a hierarchical prompt mutation framework that adaptively optimizes language model outputs for high-stakes document generation in marketplace dispute resolution. Testing on 600 real cases, the system achieved an 11 percentage point improvement in win rate and 19.1 percentage point improvement in amount-weighted outcomes compared to static prompting, combining human feedback with automated evaluation.

HOPM addresses a critical challenge in production AI systems: ensuring language models generate accurate, auditable outputs for high-stakes applications where errors carry legal and financial consequences. The framework innovates by treating prompts as online policies that continuously adapt based on dual feedback loops—combining human expert review with automated judicial assessment. This hybrid approach outperforms single-feedback mechanisms, suggesting that human intuition and algorithmic precision operate synergistically rather than as substitutes.

The research emerges from growing recognition that static prompts cannot handle the complexity of real-world document generation workflows. Traditional approaches either rely on manual iteration (slow and expensive) or automated routing alone (prone to misalignment). HOPM's deterministic guardrails systematically categorize failure modes into mutable prompt-token patterns, enabling targeted improvements rather than broad retuning. The 11 percentage point absolute improvement in baseline win rate and accompanying reduction in issue-flag rates from 15.3% to 5.2% represent substantial operational gains for dispute resolution systems.

For AI practitioners and enterprises deploying language models in regulated domains, HOPM demonstrates the viability of systematic, feedback-driven prompt optimization. The detailed release of evaluation methodology—control setups, confidence intervals, rubric definitions—establishes reproducibility standards often absent in production AI research. Organizations handling marketplace disputes, legal document generation, or other high-accountability workflows can apply these patterns to measure and iteratively improve LLM reliability. The work signals that production AI evaluation requires rigorous ablation studies and transparency about confidence bounds, setting expectations for future enterprise AI deployments.

• →HOPM combines human feedback and automated judging to improve dispute document generation from 34.7% to 45.7% win rate.
• →Deterministic guardrails systematically identify and categorize prompt failure modes for targeted optimization.
• →Dual-loop feedback outperformed single-feedback mechanisms and static prompting across all measured metrics.
• →The framework reduces quality issues from 15.3% to 5.2% flag rate while increasing perceived quality from 3.18 to 4.40 Likert score.
• →Detailed methodology release enables reproducibility and establishes standards for production AI evaluation transparency.