Learning While Acting: A Skill-Enhanced Test-Time Co-Evolution Framework for Online Lifelong Learning Agents

Researchers propose LifeSkill, a reinforcement learning framework that enables LLM agents to continuously learn and adapt during test-time interactions rather than relying on static parameters. The system combines skill extraction with real-time parameter updates, achieving 7% performance improvement over existing lifelong learning baselines on benchmark tasks.

LifeSkill addresses a fundamental limitation in current LLM agent architectures: the inability to internalize feedback during deployment. Traditional lifelong learning systems treat inference as a retrieval problem, pulling from static skill libraries or experience databases without updating core model parameters. This approach mirrors how humans learn through practice—we don't merely recall past experiences; we fundamentally restructure our understanding through interaction.

The framework's innovation lies in its two-stage approach. Verifier-Guided Skill Learning uses reward signals from task success rather than relying on human annotations or implicit likelihood measures, ensuring that extracted skills solve actual problems. Online Skill Internalization then transforms trajectories into parameter updates, allowing the agent to refine its reasoning capabilities directly during deployment. This eliminates the context-bloat problem where retrieval-based systems accumulate increasingly massive memory stores.

The technical contribution matters for scalable AI deployment. As LLM agents move from isolated tasks to truly dynamic environments, the ability to learn from operational feedback becomes critical. A chatbot or reasoning agent that improves with each interaction creates compounding value—the model becomes increasingly specialized to its deployment context without requiring retraining cycles.

The 7-point benchmark improvement signals meaningful progress, though generalization beyond LifelongAgentBench remains unclear. The framework's dependency on verifiers suggests scalability challenges in domains lacking clear reward signals. Future development should focus on verifier-free variants and robustness against distribution shifts. This research positions adaptive agents as increasingly viable for production systems where continuous learning offers competitive advantages.

• →LifeSkill enables LLM agents to update parameters during inference, moving beyond static skill libraries toward continuous learning
• →Verifier-guided skill extraction focuses on task utility rather than linguistic plausibility, improving skill quality
• →The framework achieves 7% performance gains over existing lifelong learning baselines on standardized benchmarks
• →Online parameter internalization eliminates context-bloat associated with experience retrieval approaches
• →Deployment scalability depends on access to reliable verifiers for reward signal generation