When Does Memory Help Multi-Trajectory Inference for Tool-Use LLM Agents?

Researchers demonstrate that memory mechanisms in multi-trajectory LLM agents produce inconsistent results depending on the inference strategy used, revealing that previous evaluations conflated memory abstraction properties with inference method effects. The study systematically evaluates four memory methods across three inference strategies on tool-use benchmarks, showing that reflection, fact extraction, and observation injection each perform optimally under different conditions.

This research addresses a fundamental methodological gap in evaluating memory systems for tool-use language model agents. Prior work claimed benefits from various cross-trajectory memory techniques, but lacked controlled comparisons that isolated memory effectiveness from inference strategy confounds. The unified framework proposed here decomposes memory along two critical dimensions—transfer scope and content abstraction—enabling systematic evaluation of how these factors interact with inference methods like best-of-N selection, beam search, and Monte Carlo tree search.

The findings challenge conventional assumptions about memory utility in agentic systems. Trajectory-level reflection, previously touted as universally beneficial, only achieves statistical significance when paired with MCTS, not simpler selection methods. Within-expansion injection shows value exclusively in diversity-constrained beam search scenarios. Atomic fact extraction presents a different value proposition: while maintaining accuracy, it reduces trajectory length by 19-26% on structured tasks, offering efficiency gains without accuracy trade-offs.

For developers building production LLM agents, this research provides critical guidance for architecture decisions. Memory implementation choices cannot be evaluated in isolation; effectiveness depends entirely on which inference strategy the system employs. The verifier-free evaluation setting matches real-world deployment constraints where ground-truth signal remains unavailable. This work effectively raises the bar for agent research by demonstrating that claimed improvements require rigorous factorial validation across inference methods.

• →Memory method effectiveness in LLM agents varies significantly based on the underlying inference strategy used
• →Trajectory-level reflection only achieves statistical significance with MCTS, not simpler best-of-N approaches
• →Atomic fact extraction reduces trajectory length by 19-26% on structured tasks without sacrificing accuracy
• →Within-expansion memory injection benefits only diversity-starved beam search configurations
• →Controlled experimental design across inference strategies is essential for validating memory abstraction properties