GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL

GUI-Libra presents a specialized training methodology for native GUI agents that addresses critical gaps between open-source and closed-source systems through action-aware supervised fine-tuning and improved reinforcement learning with partial verifiability. The work introduces an 81K curated GUI reasoning dataset and demonstrates consistent improvements across web and mobile benchmarks without requiring expensive online data collection.

GUI-Libra tackles a significant challenge in autonomous agent development: the performance gap between proprietary systems and open-source alternatives on complex, long-horizon GUI navigation tasks. The research identifies two core problems limiting current approaches—the disconnect between reasoning and action grounding in standard supervised fine-tuning, and the instability of reinforcement learning when multiple correct actions exist but only one is verified. These technical obstacles have practical implications for deploying reliable autonomous agents across digital interfaces.

The broader context reveals a maturing AI landscape where post-training optimization increasingly determines capability gains rather than model scale alone. Previous approaches borrowed generic reasoning pipelines from language models without accounting for GUI-specific constraints. GUI-Libra's contribution—combining action-aware token reweighting, KL regularization with trust regions, and success-adaptive scaling—represents a paradigm shift toward task-specific training recipes. The release of curated training data and open-source models addresses a critical resource bottleneck that has historically favored well-funded organizations.

For developers and AI companies, this work reduces barriers to building competitive GUI agents with improved data efficiency and training stability. The demonstrated improvements in both offline metrics and online task completion suggest the methodology addresses real deployment challenges. The emphasis on data curation and careful post-training design over raw computational resources makes advanced agent capabilities more accessible to resource-constrained teams. This democratization of agent development could accelerate practical applications in automation, testing, and accessibility tools, potentially influencing how organizations approach RPA and software automation investments.

• →GUI-Libra introduces action-aware supervised fine-tuning that reconciles reasoning with grounding, solving a fundamental tradeoff in current training approaches.
• →KL regularization with trust regions proves critical for stabilizing reinforcement learning under partial verifiability, improving offline-to-online predictability.
• →An 81K curated GUI reasoning dataset addresses data scarcity, enabling stronger agent capabilities without expensive online collection.
• →The methodology achieves consistent improvements across diverse web and mobile benchmarks with data-efficient training rather than scaled online data.
• →Open-source release of code, models, and dataset significantly lowers barriers for developing competitive GUI agents outside well-funded organizations.