Attribution-based Explanations for Markov Decision Processes

Researchers have developed attribution techniques that explain decision-making in Markov Decision Processes (MDPs), extending explainability methods beyond static inputs to sequential decision-making systems. The approach assigns importance scores to states and execution paths, enabling more interpretable AI agents in dynamic environments.

Traditional attribution techniques in machine learning explain model outputs by scoring input features at a single moment in time. This methodology breaks down in sequential decision-making contexts where agents must navigate multiple states and make interdependent choices, such as in robotics, trading systems, or game-playing AI. The research addresses this gap by formalizing how attribution should function within MDPs—systems where future states depend probabilistically on current actions.

The significance of this work stems from the growing deployment of reinforcement learning agents in consequential domains. As these systems make sequential decisions affecting real-world outcomes, understanding their reasoning becomes critical for trust and regulatory compliance. Previous explainability methods struggled with non-determinism and temporal dependencies inherent in MDPs, limiting their practical utility.

The researchers leverage strategy synthesis techniques to efficiently compute importance scores despite computational complexity. This enables practitioners to trace which states and decision paths contributed most heavily to an agent's final behavior—crucial for debugging failures, verifying safety properties, and gaining stakeholder confidence in autonomous systems.

The approach was validated across five case studies, demonstrating practical applicability beyond theoretical contribution. For developers and organizations deploying RL-based systems, this work provides tools to generate human-interpretable explanations, addressing a key barrier to responsible AI deployment. As regulation around AI transparency tightens, such explainability methods become essential infrastructure rather than optional features, positioning this research at the intersection of AI safety and practical deployment needs.

• →Attribution techniques now extend to sequential decision-making systems using MDPs, moving beyond single-timepoint explanations.
• →The method assigns importance scores to both individual states and execution paths within probabilistic decision processes.
• →Strategy synthesis techniques enable efficient computation despite the inherent non-determinism in Markov Decision Processes.
• →Five case studies validate the approach's utility for generating interpretable insights into RL agent behavior.
• →Enhanced explainability in sequential decision-making supports regulatory compliance and responsible AI deployment.