ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL

Researchers introduce ReSkill, an RL-in-the-loop framework that improves how AI agents create and refine reusable skills during policy learning. The method synchronizes skill evolution with policy optimization, enabling agents to automatically develop, test, and prune strategies that generalize across tasks more effectively than existing approaches.

ReSkill addresses a fundamental limitation in agentic reinforcement learning: while LLM agents can improve from environment feedback, they rarely develop systematic, reusable strategies that transfer across different tasks. Traditional skill-augmented RL methods treat skill creation and policy optimization as separate processes, leading to misalignment where newly created skills may conflict with or fail to support the agent's evolving policy.

The framework builds on Anthropic's Skill Creator concept but integrates it directly into the policy learning loop through group-wise reinforcement policy optimization (GRPO). This integration enables three key mechanisms: assertion-driven diagnosis of failures that triggers skill revisions, controlled rollout sampling within groups to evaluate which skill versions best support learning, and Thompson Sampling with adaptive discounting to balance exploration-exploitation as the policy develops. The overhead remains minimal because these mechanisms exploit GRPO's existing structure rather than adding independent components.

For the AI and reinforcement learning community, ReSkill demonstrates that skill-policy co-evolution produces measurable improvements, particularly on unseen tasks where generalization matters most. The automatic lifecycle of skills—creation, testing, refinement, and pruning—mirrors how human expertise develops iteratively rather than through static knowledge bases. This research advances practical agentic AI by reducing the engineering burden of manually curating skill libraries and enabling agents to discover what capabilities they actually need.

Looking ahead, this work influences how AI systems scale beyond single-task training toward more autonomous, adaptive agents. The framework's success suggests that future LLM-based systems will benefit from tightly coupled learning mechanisms where skills and policies coevolve rather than develop independently.

• →ReSkill synchronizes skill creation with policy learning, reducing conflicts between evolving skills and agent strategies.
• →The framework automates the skill lifecycle—creation, testing, refinement, and pruning—without significant computational overhead.
• →Largest performance gains appear on unseen tasks, indicating improved generalization and transfer learning capabilities.
• →Thompson Sampling with adaptive discounting balances skill exploration and exploitation as agent policies develop.
• →The approach demonstrates that tightly coupled learning mechanisms outperform decoupled skill-creation methods in agentic RL.