MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery

MLEvolve introduces a self-evolving multi-agent framework powered by large language models that automates machine learning algorithm discovery through enhanced tree search, dynamic memory systems, and hierarchical planning. The system achieves state-of-the-art results on ML engineering benchmarks while operating in half the standard runtime, demonstrating significant advances in automating complex scientific discovery tasks.

MLEvolve addresses a fundamental challenge in machine learning engineering: automating the discovery and optimization of ML algorithms at scale. Traditional approaches to algorithm search suffer from inefficient exploration patterns and inability to leverage prior discoveries, creating redundancy in the optimization process. This research tackles those limitations by introducing three core innovations—Progressive MCGS for cross-branch information sharing, Retrospective Memory for experience reuse, and adaptive coding modes for stability—that fundamentally improve how LLM agents navigate long-horizon optimization tasks.

The framework's performance metrics are particularly noteworthy. Achieving state-of-the-art results on MLE-Bench within 12 hours (half standard runtime) indicates substantial efficiency gains that could accelerate ML development cycles. The demonstrated cross-domain generalization, including outperformance on mathematical algorithm optimization against specialized methods like AlphaEvolve, suggests the approach captures generalizable principles rather than task-specific heuristics.

For the AI and ML engineering ecosystem, MLEvolve represents progress toward more autonomous, self-improving systems that reduce human engineering burden. As LLM-based agents become more capable at sustained, complex reasoning tasks, they unlock new possibilities in scientific discovery automation and optimization. However, the framework's reliance on computational resources during extended search horizons and the need for domain knowledge initialization may limit accessibility for smaller organizations.

The open-source release via GitHub enables broader community validation and integration into existing ML workflows. Future developments likely focus on scaling these multi-agent approaches to larger problem spaces and improving memory efficiency for extended optimization horizons.

• →MLEvolve achieves state-of-the-art ML algorithm discovery while reducing runtime by 50% through novel tree search and memory architecture
• →Progressive MCGS and Retrospective Memory enable agents to share discoveries across search branches and reuse accumulated experience
• →Cross-domain generalization shows the framework outperforms specialized algorithm discovery methods beyond its primary training domain
• →Decoupled strategic planning from code generation improves stability and reliability during extended long-horizon optimization tasks
• →Open-source availability positions MLEvolve for rapid adoption and integration into ML engineering pipelines