Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI

Researchers introduced SOAR, a self-improving language model system that combines evolutionary search with hindsight learning for program synthesis tasks. The method achieved 52% success rate on the challenging ARC-AGI benchmark by iteratively improving through search and refinement cycles.

• →SOAR integrates language models into evolutionary loops that alternate between search and learning phases.
• →The system converts failed search attempts into training data to improve future iterations.
• →Performance gains were demonstrated across different model scales on the ARC-AGI benchmark.
• →The approach achieved 52% success rate on the public test set, showing significant improvement over single-attempt methods.
• →Code has been open-sourced, enabling broader research and development in automated program synthesis.