#mobile-automation News & Analysis

5 articles tagged with #mobile-automation. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

5 articles

AIBullisharXiv – CS AI · Jun 197/10

🧠

ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis

Researchers present ScaleWoB, a framework that synthesizes high-fidelity interactive environments for training and evaluating GUI agents across mobile, desktop, and automotive platforms. The approach addresses critical limitations of real-world testing by providing verifiable rewards, low resource costs, and accessibility via URL-based backends, with results showing state-of-the-art agents achieve only 27.92% success compared to 92.08% for humans.

AIBearisharXiv – CS AI · Apr 157/10

🧠

Mobile GUI Agents under Real-world Threats: Are We There Yet?

Researchers have identified critical vulnerabilities in mobile GUI agents powered by large language models, revealing that third-party content in real-world apps causes these agents to fail significantly more often than benchmark tests suggest. Testing on 122 dynamic tasks and over 3,000 static scenarios shows misleading rates of 36-42%, raising serious concerns about deploying these agents in commercial settings.

AINeutralarXiv – CS AI · Jun 126/10

🧠

Teach-and-Repeat: Accurately Extracting Operational Knowledge from Mobile Screen Demonstrations to Empower GUI Agents

Researchers introduce Teach VLM, a vision-language model that extracts operational knowledge from mobile screen demonstrations to create interpretable instructions for GUI automation agents. The system uses a novel Teach-and-Repeat paradigm where extracted task procedures guide downstream execution agents, achieving state-of-the-art performance in operation semantics prediction and improving task success rates in Android environments.

AIBullisharXiv – CS AI · May 276/10

🧠

Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation

Researchers introduce HyperTrack, a large-scale dataset of 16,000+ mobile GUI navigation tasks across 650+ Chinese applications, and GUIEvalKit, an open-source benchmarking toolkit for evaluating Vision-Language Models. The study demonstrates that reinforcement-based finetuning substantially outperforms supervised learning for mobile automation tasks, with implications for developing more capable AI agents.

AIBullisharXiv – CS AI · May 116/10

🧠

AgentProg: Empowering Long-Horizon GUI Agents with Program-Guided Context Management

AgentProg introduces a novel program-guided context management system for long-horizon GUI agents that addresses the critical bottleneck of expanding interaction history overhead. By reframing interaction history as structured programs with variables and control flow, the approach preserves semantic information while reducing context requirements, achieving state-of-the-art performance on AndroidWorld benchmarks while maintaining robustness on extended tasks.