More Is Not Always Better: Cross-Component Interference in LLM Agent Scaffolding

Researchers demonstrate that stacking more components into LLM agent systems doesn't improve performance and often degrades it due to cross-component interference. A comprehensive factorial study across 32 configurations shows optimal agent design is task-dependent and model-scale dependent, with the fully-equipped system consistently underperforming smaller, curated subsets by up to 79%.

The research challenges a widespread assumption in AI development: that additive complexity yields better results. The study's experimental rigor—testing all 32 possible combinations of five scaffolding components (planning, tools, memory, self-reflection, retrieval) across two datasets and multiple model scales—provides robust evidence that component interactions create measurable degradation rather than synergy in many cases. On HotpotQA, a minimal single-tool agent outperformed the fully-equipped system by 32%, while on GSM8K, a three-component subset achieved 79% better performance than the all-inclusive version.

This finding reflects broader emerging wisdom in machine learning that parameter efficiency and architectural simplicity can outperform complexity. The identification of 183 submodularity violations (56.3%) indicates greedy component selection strategies are unreliable, forcing developers to reconsider optimization approaches. Notably, the optimal configuration proved scale-sensitive: components that hurt performance at 8B parameters sometimes benefited larger 70B models, though all-in systems still underperformed curated subsets at both scales.

For the AI development community, this research directly impacts production system design. Current industry default practices favor comprehensive agent scaffolding, assuming robustness through redundancy. This study suggests instead that task-specific analysis and interaction-aware subset selection should become standard practice. The discovery of a three-body synergy among Tool Use, Self-Reflection, and Retrieval points toward more nuanced component interplay than previously understood. The replication across Qwen2.5 and robustness to prompt variations strengthen the generalizability of findings, establishing this as foundational guidance for agent architecture decisions rather than a quirk of specific implementations.

• →Maximally-equipped LLM agents consistently underperform smaller task-specific subsets by up to 79% due to cross-component interference.
• →Optimal agent configuration is task-dependent (requiring 1-4 components) and scales differently between 8B and 70B model sizes.
• →Greedy component selection fails in 56% of tested subsets due to non-submodular interactions, requiring exhaustive or interaction-aware analysis.
• →A three-way synergy exists between Tool Use, Self-Reflection, and Retrieval components worth investigating further.
• →Industry defaults should shift from all-inclusive architectures to evidence-based subset selection based on specific task requirements.