Visualizing Critic Match Loss Landscapes for Interpretation of Online Reinforcement Learning Control Algorithms

Researchers have developed a new visualization method for analyzing critic neural networks in reinforcement learning algorithms by creating 3D loss landscapes from parameter trajectories. The approach enables both visual and quantitative interpretation of critic optimization behavior in online reinforcement learning, demonstrated on control tasks like cart-pole and spacecraft attitude control.

• →New visualization method projects critic neural network parameters onto low-dimensional subspaces to create interpretable loss landscapes.
• →The approach combines 3D loss surfaces with 2D optimization paths to characterize critic learning behavior in reinforcement learning.
• →Quantitative landscape indices enable structured comparison across different training outcomes beyond visual inspection.
• →Method was demonstrated on Action-Dependent Heuristic Dynamic Programming algorithm for cart-pole and spacecraft control tasks.
• →Framework reveals distinct landscape characteristics associated with stable convergence versus unstable learning patterns.