Offline-to-Online Multi-Agent Reinforcement Learning with Offline Value Function Memory and Sequential Exploration

Researchers propose OVMSE, a new framework for Offline-to-Online Multi-Agent Reinforcement Learning that addresses key challenges in transitioning from offline training to online fine-tuning. The framework introduces Offline Value Function Memory and Sequential Exploration strategies to improve sample efficiency and performance in multi-agent environments.

• →OVMSE framework tackles two critical challenges in multi-agent reinforcement learning: unlearning pre-trained Q-values and inefficient exploration in large state-action spaces.
• →The Offline Value Function Memory mechanism preserves knowledge from offline training during the transition to online phases.
• →Sequential Exploration strategy reduces the complexity of joint state-action space exploration by utilizing pre-trained offline policies.
• →Experiments on StarCraft Multi-Agent Challenge demonstrate superior sample efficiency compared to existing baselines.
• →The research advances multi-agent reinforcement learning capabilities for complex decision-making scenarios.