Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolution

Researchers introduce Ace-Skill, a co-evolutionary framework that improves multimodal AI agents by optimizing both data sampling and knowledge organization. The system achieves 35% performance gains on tool-use benchmarks and enables smaller models to inherit capabilities from larger ones without additional training.

Ace-Skill addresses fundamental inefficiencies in self-evolving AI systems by tackling two interconnected problems: data inefficiency and knowledge interference. Traditional agent training wastes computational resources on low-value samples while organizing diverse knowledge in ways that degrade retrieval quality and task alignment. This creates a compounding failure loop where poor rollouts generate noisy knowledge that further undermines agent performance. The framework breaks this cycle through dual optimization—a prioritized sampler using lazy-decay proficiency tracking focuses computational effort on informative and under-mastered samples, while a semantic clustering mechanism organizes knowledge for cleaner retrieval and better task adaptation.

The technical contribution represents meaningful progress in efficient AI training, particularly relevant as computational costs and environmental concerns continue escalating. The results demonstrate substantial practical impact: an open-source 35B model now matches proprietary counterparts, while the knowledge transfers effectively to resource-constrained 4B and 9B models in zero-shot settings. This transfer capability has important implications for democratizing advanced AI capabilities across different hardware constraints.

For the broader AI development ecosystem, Ace-Skill illustrates how thoughtful system design can reduce computational overhead without sacrificing performance. The open-source release on GitHub enables reproducibility and adoption across research and commercial applications. The framework's multimodal focus positions it favorably as real-world applications increasingly demand integrated vision-language-action capabilities. The 35% accuracy improvements and model-agnostic knowledge transfer suggest this approach could become standard practice in agent development pipelines.

• →Ace-Skill reduces training inefficiency by prioritizing informative samples and tracking skill proficiency to focus computational resources more effectively.
• →Semantic clustering of knowledge artifacts eliminates retrieval noise and improves task alignment compared to shared repositories.
• →Open-source 35B model now matches proprietary alternatives across four multimodal benchmarks with 35% relative accuracy improvements.
• →Zero-shot knowledge transfer enables smaller 4B and 9B models to inherit advanced capabilities from larger agents without retraining.
• →Framework addresses the self-reinforcing failure loop where poor rollouts generate noisy knowledge that degrades subsequent performance.