Reinforcement learning to improve large language model-based automated code compliance systems

Researchers introduce P4IR, a two-stage framework combining supervised fine-tuning and Group Relative Policy Optimization to improve LLM accuracy in automated building code compliance systems. The approach reduces errors by up to 38.6% compared to baseline models and outperforms leading LLMs like Claude and GPT in zero-shot settings.

The development of P4IR addresses a critical vulnerability in LLM-based automation: hallucination and incorrect rule generation in high-stakes domains like building compliance. When applied to code compliance, these errors carry real-world consequences for construction safety and regulatory adherence. The framework's two-stage approach—first instilling domain knowledge through supervised fine-tuning, then optimizing output quality via Group Relative Policy Optimization—represents a methodological advance in making LLMs reliable for specialized applications.

This work emerges from growing recognition that generic LLMs, despite their scale and capability, require domain-specific refinement for critical use cases. The building compliance sector has historically relied on manual expert review, making automation particularly valuable but also demanding high accuracy standards. Previous attempts using raw LLMs generated unusable outputs, necessitating techniques that ground models in regulatory frameworks.

The framework's competitive performance against Claude Opus, GPT-5.2, and Qwen-3-Max in zero-shot evaluation demonstrates that targeted optimization can exceed general-purpose models. The 38.6% error reduction in token-level accuracy and reduction in false positives suggest practical deployability. For construction technology companies and regulatory bodies, this enables more efficient compliance verification without sacrificing accuracy.

The broader implication extends beyond building codes—this methodology applies to any regulated domain requiring code generation or rule interpretation: financial compliance, healthcare regulations, and environmental standards. As enterprises increasingly deploy LLMs in risk-sensitive contexts, techniques that combine domain knowledge with reinforcement learning become essential infrastructure.

• →P4IR framework reduces code compliance errors by up to 38.6% compared to baseline LLM approaches through combined SFT and GRPO optimization.
• →The model outperforms leading LLMs including Claude Opus, GPT-5.2, and Qwen-3-Max in zero-shot building code compliance evaluation.
• →Domain-specific reinforcement learning significantly reduces hallucinations and false positives in high-stakes automated compliance systems.
• →Two-stage training approach combining supervised fine-tuning with policy optimization provides replicable methodology for other regulated industries.
• →Framework demonstrates viability of LLM-based automation in sectors requiring high accuracy and safety compliance standards.