LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents

Researchers introduce LatentSkill, a framework that converts textual skills into efficient LoRA adapters for LLM agents, storing knowledge in model weights rather than context prompts. The approach reduces token overhead by 64-72% while improving task performance, enabling more scalable and modular AI agent systems.

LatentSkill addresses a fundamental inefficiency in current LLM agent architectures: the need to repeatedly inject skill descriptions into prompts, consuming substantial context windows and limiting scalability. By leveraging a pretrained hypernetwork to convert textual skills into LoRA adapters, the framework moves skill storage from the ephemeral context space into persistent weight space. This architectural shift carries significant implications for production deployments where token costs and latency directly impact operational expenses.

The research builds on years of LoRA optimization work and the growing recognition that LLM agents require more efficient knowledge representation mechanisms than prompt engineering alone can provide. As agent systems grow more complex with multiple composable skills, context window saturation becomes a critical bottleneck. LatentSkill's empirical results—21.4-point improvements on ALFWorld with 64% fewer prefill tokens—demonstrate that weight-space skills aren't merely theoretical improvements but represent genuine practical advantages.

For developers building agent systems, this work suggests a pathway toward more efficient, modular architectures that scale better than current in-context skill approaches. The structured semantic geometry of generated LoRAs and their composability through parameter arithmetic indicate that skill knowledge encodes meaningfully in weight space. This enables fine-grained control and combination capabilities comparable to prompt-based systems while maintaining substantial efficiency gains.

The research opens questions about skill transfer across model families, the computational cost of hypernetwork inference versus token savings, and whether this approach generalizes beyond the tested benchmarks. Future work should explore skill composition at scale and determine optimal strategies for managing large skill libraries.

• →LatentSkill converts textual skills to LoRA adapters via hypernetworks, reducing token overhead by 64-72% while improving task performance.
• →Weight-space skills enable modular composition through parameter arithmetic, allowing precise control via LoRA scaling coefficients.
• →The framework demonstrates structured semantic geometry in generated LoRAs, suggesting skill knowledge encodes meaningfully in model weights.
• →ALFWorld and Search-QA benchmarks show 21.4-point and 3.0-point performance gains respectively with substantially fewer prefill tokens.
• →This approach addresses context window saturation in complex agent systems while reducing plaintext exposure of skill procedures.