Failure Modes of Deep Multi-Agent RL in Asynchronous Pricing: Reproducible Triggers, Trace Diagnostics, and a Partial Fix

Researchers identify two critical failure modes in deep multi-agent reinforcement learning applied to continuous pricing markets: tacit collusion between DDPG agents and actor-critic instability at high event rates. While asynchronous pricing and latency reduce collusion by up to 48%, the fix remains partial and breaks down under high-frequency conditions, revealing fundamental limitations in current MARL approaches for market simulation.

This research exposes vulnerabilities in applying deep reinforcement learning to multi-agent market dynamics, a problem increasingly relevant as AI systems model and potentially participate in financial markets. The study demonstrates that synchronized DDPG agents consistently develop tacit cartel behavior with a collusion index of 0.69, approximating coordinated pricing well above competitive Bertrand equilibrium. This finding has profound implications for market design and financial regulation, as it suggests AI systems may naturally gravitate toward anti-competitive outcomes without explicit coordination mechanisms.

The partial fix through asynchronous pricing and observation latency is encouraging but incomplete. The 48% reduction in collusion when removing synchronization indicates market microstructure significantly influences agent behavior, yet the non-monotonic relationship with latency and collapse at high event rates reveals the approach lacks robustness. The emergence of critic divergence at event rates of λ=5 suggests the learning framework itself destabilizes under realistic market conditions, where price updates occur at millisecond scales.

For market participants and regulators, this research clarifies that AI-driven pricing systems require careful constraint design to prevent emergent anti-competitive behavior. The trajectory-level diagnostics showing within-episode signaling collapse provide mechanisms for detecting when agents develop collusive strategies. Financial institutions deploying MARL for trading or market-making should account for potential instability under high-frequency conditions. The study establishes baseline vulnerabilities that future work must address before AI systems can reliably operate in competitive markets without regulatory intervention or architectural safeguards.

• →Synchronized DDPG agents reliably develop tacit collusion with collusion index of 0.69, significantly above competitive pricing
• →Asynchrony and latency reduce collusion by 48% but cannot eliminate it, remaining non-monotonic and unstable at high event rates
• →Actor-critic instability emerges at λ=5 event rate, corrupting the MARL framework under realistic market frequency conditions
• →Trajectory-level diagnostics can detect within-episode signaling collapse and non-recovery patterns indicative of collusive behavior
• →Current deep MARL approaches require architectural constraints to prevent emergent anti-competitive outcomes in pricing markets