FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents

Researchers introduce FT-Dojo, an interactive environment for studying autonomous LLM fine-tuning, along with FT-Agent, an AI system that can automatically fine-tune language models without human intervention. The system achieved best performance on 10 out of 13 tasks across five domains, demonstrating the potential for fully automated machine learning workflows while revealing current limitations in AI reasoning capabilities.

• →FT-Dojo provides the first benchmark environment specifically designed for testing autonomous LLM fine-tuning across 13 tasks and 5 domains.
• →FT-Agent successfully automates the complete fine-tuning process including data curation, training pipeline setup, and iterative refinement based on evaluation feedback.
• →Purpose-built fine-tuning agents significantly outperformed general-purpose AI alternatives in experimental testing.
• →The approach scales effectively to smaller 3B parameter models, making autonomous fine-tuning more accessible.
• →Current limitations in causal reasoning highlight the boundaries of what autonomous AI systems can achieve in complex machine learning tasks.