ASAP: Agent-System Co-Design for Wall-Clock-Centered Auto HPO Research for ML Experiments

Researchers introduce ASAP, an agent-system co-design that leverages LLMs to coordinate multiple hyperparameter optimization tools while reducing wall-clock execution time through architectural innovations like KV-cache reuse and speculation parallelism. The approach addresses fundamental limitations in current LLM-based HPO methods by treating the language model as an orchestrator rather than a replacement tool, demonstrating consistent performance improvements across diverse ML tasks.

ASAP represents a meaningful evolution in hyperparameter optimization research by reframing how language models contribute to automated machine learning workflows. Rather than positioning LLMs as direct replacements for existing optimization algorithms, the researchers recognize that diverse ML problems benefit from diverse optimization strategies. The core insight—that an LLM can serve as a meta-optimizer selecting among specialized tools—sidesteps the fundamental constraint that any single algorithm inherits inductive biases limiting its generalization across problem families.

The research addresses a critical gap between per-iteration improvements and real-world deployment value. Previous LLM-based HPO work optimized for iteration count while ignoring the computational overhead of LLM inference and tool execution running serially with model training. ASAP's system-level contributions—prefix-stable prompting for KV-cache efficiency, speculation parallelism that hides latency under model evaluation, and adaptive self-tuning—transform theoretical gains into practical wall-clock improvements. This reflects growing maturity in AI systems research, where end-to-end optimization matters as much as algorithmic improvements.

The implications extend beyond academic interest. Practitioners conducting ML experiments face real time and computational costs; tools that genuinely reduce total execution time create tangible value. The agent-system co-design pattern also suggests broader applicability—using LLMs strategically within computational pipelines rather than as monolithic replacements emerges as a viable design principle across domains. As ML experimentation becomes more resource-intensive, optimization techniques that preserve quality while reducing wall-clock time address genuine industry pain points, particularly for organizations running extensive hyperparameter searches on large models.

• →ASAP uses LLMs as meta-optimizers selecting among diverse HPO tools rather than as single-tool replacements, improving adaptability across problem diversity.
• →System-level optimizations including KV-cache reuse and speculation parallelism convert per-iteration gains into measurable wall-clock time savings.
• →The approach demonstrates that agent-system co-design can effectively address both algorithmic and infrastructure challenges in ML automation.
• →Real-world deployment considerations like serialized inference costs drive practical improvements that per-iteration metrics alone would miss.
• →Consistent experimental improvements across diverse HPO tasks suggest the method generalizes beyond narrow problem classes.