MMG2Skill: Can Agents Distill In-the-Wild Guides into Self-Evolving Skills?

Researchers introduce MMG2Skill, a framework that converts unstructured web guides into executable skills for AI agents, with a new benchmark for evaluation. The system improves agent performance by 12.8-25.3 percentage points across multiple domains by structuring knowledge, conditioning vision-language models on refined skills, and iteratively improving them from agent trajectories.

MMG2Skill addresses a fundamental challenge in AI agent development: converting human-oriented procedural knowledge into machine-executable instructions. The framework tackles the heterogeneity problem inherent in web-scraped guides—which contain multimodal content, inconsistent formatting, and human-specific assumptions—by creating an intermediate representation layer that bridges human instruction and agent capability. This closed-loop approach gains significance as organizations increasingly deploy autonomous agents for complex tasks requiring procedural reasoning.

The research emerges from the broader trend of making large language and vision models more capable at long-horizon reasoning. While foundation models demonstrate impressive few-shot capabilities, their ability to leverage external knowledge sources remains limited. MMG2Skill's solution of compiling guides into editable skills and continuously refining them represents a practical engineering advancement that reduces the gap between model capacity and task performance.

The benchmark contribution provides the field with a standardized evaluation framework for guide-to-skill conversion, enabling reproducible progress measurement. The consistent 12.8-25.3 percentage point improvements across six different vision-language model backbones suggest the approach generalizes beyond specific architectures. The trajectory-driven revision mechanism, which learns from agent execution without relying on benchmark scores, offers a path toward self-improving systems that adapt to real-world deployment contexts.

Future developments should focus on scaling this framework to more complex domains and understanding failure modes in skill revision. The early-stopping mechanism preventing late-stage regressions indicates the need for better success detection in open-ended tasks. Integration with more sophisticated skill representations and multi-agent coordination scenarios represents promising research directions.

• →MMG2Skill converts unstructured web guides into executable agent skills through structured compilation and iterative refinement from trajectories.
• →The framework achieves 12.8-25.3 percentage point performance improvements across six vision-language model backbones in GUI control, gameplay, and strategy domains.
• →New MMG2Skill-Bench benchmark provides standardized evaluation for guide-to-skill learning, enabling reproducible progress measurement.
• →Trajectory-driven skill revision without benchmark scores demonstrates potential for self-improving agents in deployment contexts.
• →Early-stopping mechanism saves 25-53% of task attempts by preventing performance regressions when success signals are properly calibrated.