VISTA: An End-to-End Benchmark for Visual Spec-to-Web-App Coding Agents

VISTA is a new benchmark for evaluating how well AI agents can generate functional web applications from visual specifications and text descriptions. The benchmark introduces five different testing conditions with varying levels of design detail and technology stack constraints, using manual annotations and multi-modal evaluation metrics to assess both visual fidelity and functional correctness.

VISTA addresses a critical gap in AI agent evaluation by moving beyond algorithmic code generation toward realistic, UI-centric software development. Traditional benchmarks focus on isolated coding tasks, but modern development increasingly requires agents to transform visual designs and underspecified requirements into functional applications—a far more complex challenge. The benchmark's five-condition framework systematically varies information fidelity and constraints, allowing researchers to isolate which inputs most benefit agent performance.

The evaluation methodology represents a significant advancement in assessment rigor. Rather than relying solely on automated testing tools like Playwright—which struggle with open-ended generation—VISTA combines DOM-grounded verification, behavior-specific tests, and CLIP-based visual similarity metrics. This multi-faceted approach captures the nuanced reality that visual correctness and functional correctness are partially decoupled; an agent might produce a pixel-perfect interface that doesn't work properly, or functional code that looks wrong.

For the AI development community, VISTA establishes a reproducible foundation for measuring progress in agent-based software engineering. The benchmark's findings that agent editing styles vary sharply but remain orthogonal to task quality suggest that how agents approach code generation differs significantly from whether they succeed—an important distinction for understanding agent cognition. This research accelerates the path toward AI agents capable of autonomous, production-grade web development, with implications for development velocity and the role of human engineers in software creation.

• →VISTA introduces five testing conditions varying visual fidelity and technology constraints to comprehensively evaluate web-app generation agents.
• →Manual annotations of interactive components and visual anchors overcome limitations of automated testing tools in open-ended code generation.
• →Visual fidelity and functional correctness are partially decoupled across agents and input conditions, requiring multi-modal evaluation metrics.
• →Agent editing styles vary significantly but are largely orthogonal to task quality, suggesting diverse valid approaches to code generation.
• →The benchmark establishes a reproducible foundation for measuring progress in autonomous, agent-based software engineering research.