HiMAC: Hierarchical Macro-Micro Learning for Long-Horizon LLM Agents

Researchers introduce HiMAC, a hierarchical reinforcement learning framework that improves LLM agent performance on long-horizon tasks by separating macro-level planning from micro-level execution. The approach demonstrates state-of-the-art results across multiple environments, showing that structured hierarchy is more effective than simply scaling model size for complex agent tasks.

• →HiMAC addresses LLM agents' limitations in long-horizon tasks by implementing hierarchical macro-micro planning structure.
• →The framework uses critic-free hierarchical policy optimization with iterative co-evolution training strategy.
• →Testing on ALFWorld, WebShop, and Sokoban shows consistent outperformance of existing prompting and RL baselines.
• →Results indicate structured hierarchy is more important than model scale increases for robust long-horizon intelligence.
• →The approach achieves improved sample efficiency across both text-based and visually grounded environments.