How Should Agents Read Demonstrations? Hierarchical Structure Beats Flat Action Logs

A research paper demonstrates that organizing demonstration data hierarchically into labeled subgoals significantly improves LLM agent performance on ambiguous tasks, achieving 90.7% pass rates versus 76.7% for flat action logs. This finding provides concrete design guidance for Programming by Demonstration systems and broader procedural knowledge transfer to AI agents.

This arXiv paper addresses a fundamental challenge in making AI agents more accessible: how to structure procedural demonstrations so agents can learn effectively. Programming by Demonstration removes barriers for non-programmers by allowing users to show rather than explain what they want, but the presentation format of recorded actions directly impacts agent comprehension and execution quality. The researchers' controlled experiment across 85 web automation tasks reveals that hierarchical organization matters primarily when task descriptions are vague or incomplete, improving performance by 14 percentage points in such scenarios while providing no benefit when instructions are precise. This distinction is critical because real-world use cases often involve ambiguous or partially-specified goals where procedural context becomes essential for correct execution. The finding that only subgoal grouping drives improvements—not preconditions, postconditions, or parameter annotations—suggests that agent cognition benefits from organizational structure itself rather than semantic enrichment. This result generalizes beyond PbD systems to any architecture feeding sequential information to LLM agents, implying that prompt engineering and context presentation strategies should prioritize hierarchical segmentation. For the AI development community, this provides empirical validation for design intuitions and offers a practical optimization with measurable impact. The work exemplifies how rigorous experimental methodology can resolve open design questions in emerging agent systems, moving the field toward evidence-based best practices rather than ad-hoc approaches.

• →Hierarchical subgoal grouping improves agent performance on ambiguous tasks by 14 percentage points compared to flat action logs
• →Procedural structure benefits only emerge when task descriptions lack sufficient detail or precision
• →Subgoal naming alone drives performance gains; additional annotations like preconditions add no measurable benefit
• →Findings apply broadly to any system presenting sequential information to LLM agents, not just Programming by Demonstration
• →Design guidance favors hierarchical segmentation over flat step lists for optimal agent instruction comprehension