Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?

Researchers introduce Agent^2 RL-Bench, a benchmark testing whether LLM agents can autonomously design and execute reinforcement learning pipelines to improve foundation models. Testing across multiple agent systems reveals significant performance variation, with online RL succeeding primarily on ALFWorld while supervised learning pipelines dominate under fixed computational budgets.

Agent^2 RL-Bench addresses a critical gap in AI research by creating the first systematic evaluation of whether language model agents can engineer their own reinforcement learning post-training workflows. This matters because RL post-training has become central to model alignment and specialization, yet prior benchmarks relied on static evaluation methods that don't capture the interactive optimization process. The research reveals a nuanced landscape: while agents achieved dramatic improvements on some tasks (ALFWorld: 5.97% to 93.28% accuracy), they plateaued on others like DeepSearchQA with marginal gains within measurement noise. This variance indicates that agentic RL isn't universally superior but highly task-dependent. The finding that driver LLM selection dramatically impacts results—causing interactive improvement to swing from near-zero to +78 percentage points within identical scaffolds—underscores how agent capability depends on foundational model choices. More significantly, the benchmark demonstrates that supervised fine-tuning pipelines consistently outperform agent-driven online RL under realistic computational constraints. This challenges assumptions that autonomous RL engineering represents a clear upgrade path. The research suggests the field should temper expectations about fully autonomous model optimization, instead focusing on hybrid approaches where agents enhance rather than replace human-guided workflows. The structured diagnostic capability the benchmark provides enables future analysis of agent decision-making during post-training, potentially identifying where autonomous optimization fails and why.

• →Agent-driven RL post-training shows high variance across tasks, succeeding dramatically on ALFWorld but offering minimal gains on DeepSearchQA.
• →LLM choice significantly impacts agent performance in interactive tasks, with driver selection causing up to 78 percentage point swings in improvement.
• →Supervised fine-tuning pipelines dominate agent-driven online RL under fixed computational budgets, suggesting limitations to fully autonomous optimization.
• →Agent^2 RL-Bench provides automated diagnostics of agent-driven post-training through isolated workspaces and structured run reporting.
• →Results indicate hybrid approaches combining agent assistance with human guidance may be more effective than fully autonomous RL engineering.