PIRS: Physics-Informed Reward Shaping for SAC-Based Building Energy Management

Researchers introduce PIRS (Physics-Informed Reward Shaping), a method that improves deep reinforcement learning controllers for building energy management by replacing ad-hoc comfort metrics with ISO 7730 Predicted Mean Vote (PMV) standards. Tested on CityLearn v2.1.2, PIRS demonstrates competitive performance against manual baselines while substantially outperforming non-physics-grounded approaches in load ramping and peak demand metrics.

PIRS addresses a fundamental challenge in applying deep reinforcement learning to building energy systems: the tension between occupant comfort and grid efficiency requires reward functions grounded in real physical principles rather than arbitrary heuristics. Building climate control represents a critical infrastructure optimization problem affecting energy consumption, grid stability, and human welfare across millions of structures globally. Traditional approaches rely on hand-tuned temperature deviations or comfort proxies disconnected from thermal physics, creating interpretability gaps and limiting transferability across different building types and climates.

The shift toward physics-informed reward shaping reflects a broader maturation in AI systems engineering. Rather than treating reward design as an engineering afterthought, PIRS embeds ISO 7730 PMV formulation—an internationally recognized thermal comfort standard—directly into the learning objective. This grounds the AI system in established domain knowledge, enabling stakeholders to understand and validate comfort trade-offs without requiring deep expertise in reinforcement learning mechanics.

The experimental results reveal both progress and honest limitations. PIRS achieves cost, carbon, and electricity metrics comparable to manually engineered rewards while improving load ramping performance (1.78x versus 2.4x baseline ramping ratios) and daily peak demand management. However, the authors transparently acknowledge that all DRL policies remain above rule-based controllers at the tested compute budget, positioning PIRS as a foundation for iterative improvement rather than claiming immediate superiority. This methodological transparency strengthens credibility within the research community.

The framework establishes a template for physics-informed AI in infrastructure optimization. As building automation, grid modernization, and demand-response programs expand globally, interpretable, standards-aligned control systems become increasingly valuable for regulatory compliance and stakeholder trust.

• →PIRS replaces heuristic comfort metrics with ISO 7730 PMV formulation, improving interpretability and standards alignment in building energy AI
• →Physics-grounded reward shaping substantially outperforms non-physics approaches on load ramping and peak demand while matching manual baselines on cost and carbon metrics
• →The method demonstrates that embedding domain expertise into reward functions enhances both performance and explainability in infrastructure optimization systems
• →Authors candidly acknowledge that DRL policies underperform classical controls at limited compute budgets, positioning PIRS as a foundation rather than a replacement
• →This approach establishes a replicable pattern for physics-informed AI in grid management, smart buildings, and other critical infrastructure domains