Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies

Researchers introduce Safe Flow Q-Learning (SafeFQL), a new offline safe reinforcement learning method that combines Hamilton-Jacobi reachability with flow policies for safety-critical real-time control. The method achieves better safety performance with lower inference latency compared to existing diffusion-based approaches, making it more suitable for real-time deployment.

• →SafeFQL extends Flow Q-Learning to safe offline RL by integrating reachability-based safety value functions with efficient one-step flow policies.
• →The method uses conformal prediction calibration to account for finite-data approximation errors and provide probabilistic safety coverage.
• →SafeFQL trades higher offline training costs for substantially lower inference latency compared to diffusion-style baselines.
• →Testing on boat navigation and Safety Gymnasium MuJoCo tasks showed matching or exceeding prior performance while reducing constraint violations.
• →The approach enables reward-maximizing safe action selection without rejection sampling during deployment.