QSIM: Mitigating Overestimation in Multi-Agent Reinforcement Learning via Action Similarity Weighted Q-Learning

Researchers propose QSIM, a new framework that addresses systematic Q-value overestimation in multi-agent reinforcement learning by using action similarity weighted Q-learning instead of traditional greedy approaches. The method demonstrates improved performance and stability across various value decomposition algorithms through similarity-weighted target calculations.

• →QSIM addresses systematic Q-value overestimation in multi-agent reinforcement learning through action similarity weighted calculations.
• →The framework replaces greedy joint actions with similarity weighted expectations over structured action spaces.
• →QSIM can be integrated with existing value decomposition methods to improve performance and learning stability.
• →Experimental results show consistent superior performance compared to original algorithms.
• →The approach mitigates issues caused by combinatorial explosion in joint action spaces that lead to unstable learning.