Prospect-Theory Behavior from Bellman Optimality in MDPs with Catastrophic States

Researchers demonstrate that optimal control in Markov decision processes with catastrophic failure states naturally produces prospect-theory-like behaviors—including S-shaped value functions and loss aversion—without requiring utility curvature or probability weighting. The mechanism emerges purely from the mathematical structure of Bellman optimality when agents face absorbing failure states, with results validated across 495 configurations and multiple learning paradigms.

This theoretical research reveals a fundamental insight into risk behavior: prospect-theory signatures emerge naturally from optimal decision-making under catastrophic risk, not from cognitive biases or non-linear preferences. The authors prove that standard Bellman optimality produces three characteristic phenomena—S-shaped value profiles, endogenous loss-sensitivity coefficients above unity, and reflection effects—across diverse configurations. The closed-form expression for asymptotic loss-aversion plateaus demonstrates that the boundary contribution (proximity to catastrophic states) dominates payoff asymmetry in generating loss aversion, with median asymmetry contributions of only 4.6% at moderate payoff ratios.

The research bridges behavioral economics and optimal control theory by showing these effects persist under realistic conditions: tabular Q-learning achieves near-perfect correlation with optimal value functions (0.98-1.00), and the mechanism remains robust under stochastic transitions with Gaussian, heavy-tailed, and asymmetric noise distributions up to 50% of step size. This normative foundation for prospect-theory behavior has significant implications for understanding decision-making in systems with tail risks, including financial portfolios, autonomous vehicles near safety boundaries, and reinforcement learning agents navigating danger zones.

For AI and control systems, this suggests that apparent irrationality near catastrophic states may represent genuine optimality rather than bias. The work challenges the framing of loss aversion as purely psychological, demonstrating instead that rational agents naturally exhibit such behavior when facing absorbing failure states. These findings inform algorithm design for safety-critical applications where conservative behavior near boundaries emerges mathematically optimal.

• →Prospect-theory signatures emerge naturally from Bellman optimality with catastrophic states, without cognitive biases or utility curvature
• →The mechanism's loss-aversion coefficient depends primarily on proximity to catastrophe, not payoff asymmetry
• →Results validate robustly across Q-learning, stochastic transitions, and heavy-tailed noise distributions
• →Boundary effects contribute 86-96% of loss aversion in tested configurations, with asymmetry contributing only 4-14%
• →These findings reframe apparent irrationality in high-stakes decisions as mathematically optimal behavior