Joint Agent Memory and Exploration Learning via Novelty Signals

Researchers introduce JAMEL, a framework that trains AI agents to explore open-ended environments more effectively by jointly developing memory systems and exploration policies through novelty-driven learning. The approach uses natural supervisory signals like code coverage to train compressed memory representations, achieving exploration capabilities that rival closed-source models while reducing computational token consumption.

JAMEL addresses a fundamental limitation in current language model agents: their inability to efficiently explore open-ended environments while maintaining computational efficiency. The framework recognizes that exploration and memory form a symbiotic relationship—agents need memory to distinguish explored from unexplored behaviors, while exploration naturally generates the supervisory signals needed to train effective memory systems. This mutual dependency has previously been treated as separate problems in agent development.

The innovation lies in leveraging deterministic novelty signals, particularly code coverage in GUI domains, to provide annotation-free supervision for memory training. This approach circumvents the expensive labeling requirements that typically constrain agent development. Rather than storing raw interaction histories, JAMEL compresses information into latent memory representations, substantially reducing computational overhead during long-trajectory planning.

For the AI development community, this work demonstrates that open-weight agents can match or approach closed-source model capabilities with thoughtful architectural design. The open-sourcing of both code and models accelerates broader adoption of sophisticated exploration strategies. This research impacts developers building autonomous systems that must navigate complex, partially-known environments—from software automation to robotic control.

The significance extends to resource efficiency in AI development. By reducing token consumption while maintaining exploration depth, JAMEL makes frontier-level agent capabilities more accessible to researchers with limited computational budgets. The framework's generalization to unseen environments suggests its principles could extend beyond GUI domains to other exploration-intensive tasks.

• →JAMEL jointly trains agent memory and exploration policy through novelty-driven learning, solving the supervisory signal problem in latent memory training
• →The framework uses code coverage as a natural, annotation-free supervisory signal for memory module training in GUI environments
• →JAMEL matches closed-source model exploration capabilities while consuming fewer tokens than comparable open-weight baselines
• →Compressed latent memory representations reduce computational overhead compared to storing raw interaction histories
• →Open-sourced implementation accelerates adoption of sophisticated exploration strategies across the AI development community