Region4Web: Rethinking Observation Space Granularity for Web Agents

Region4Web introduces a novel framework that reorganizes how AI web agents perceive and process web pages by shifting from element-level to functional region-level observation granularity. The approach, validated on WebArena benchmark, reduces observation length while improving task success rates across multiple LLM models, demonstrating that hierarchical abstraction of page structure yields more efficient agent performance.

Region4Web addresses a fundamental limitation in current web agent design: the mismatch between observation and action granularity. Traditional web agents operate at element-level throughout their perception pipeline, requiring them to extract page structure and function from individual DOM elements at every interaction step. This research posits that pages have inherent functional organization—regions that serve distinct purposes—which agents should explicitly recognize rather than reconstruct repeatedly.

The technical contribution centers on two components: hierarchical decomposition of the AXTree into semantically meaningful regions, and PageDigest, an inference pipeline that compresses region-level observations into compact per-step summaries. By exposing the page's functional organization explicitly, agents gain clearer context for decision-making. Testing on WebArena demonstrates substantial improvements: observation length decreases significantly while task success rates increase across different LLM backbones, from small to large models.

This work has meaningful implications for AI agent development beyond web automation. The principle of operating at appropriate abstraction levels rather than raw element granularity could apply to other domains requiring perception of complex structured environments. For developers building autonomous systems, the research suggests that preprocessing observations through semantic decomposition yields better performance than forcing agents to discover structure implicitly.

Looking forward, the framework's generalizability to different page types, its scalability to highly dynamic pages, and integration with emerging agent architectures warrant investigation. The work contributes to making web agents more sample-efficient and capable, supporting broader adoption of autonomous web interaction systems.

• →Region4Web shifts web agent perception from element-level to functional region-level granularity, improving both efficiency and accuracy
• →PageDigest compresses page observations into compact per-step digests while maintaining semantic information across interaction steps
• →Functional region-based observation outperforms element-level processing across LLM models of varying sizes on WebArena benchmark
• →The framework demonstrates that exposing implicit page structure yields more effective agent decision-making without increasing observation complexity
• →Results suggest hierarchical semantic abstraction is a generalizable principle for improving perception in structured environment agents