Alibaba's Qwen-UI-Agent Beats GPT-5.6 and Claude Opus 4.8 at Controlling Screens — and the Weights Are Free
Alibaba's new GUI agent scores 82.1% on MobileWorld, 12 points ahead of GPT-5.6, and hits 92.2% on real Android handsets. The 2B and 8B open-weight models run on-device, with no commercial API in sight.
Alibaba’s Tongyi-MAI team released Qwen-UI-Agent this week, a GUI agent that reads a screen and operates it directly — no app APIs, no instrumentation — and the benchmark numbers are hard to dismiss. On MobileWorld, the standard mobile-agent evaluation, it scores 82.1%, beating GPT-5.6 by 12 percentage points and Claude Opus 4.8 by 14.6. On tasks run on actual Android handsets rather than emulators, the score climbs to 92.2%. On AndroidDaily, a broader real-phone evaluation, it hits 97.5%.
The desktop story holds up too. On OSWorld-Verified, the computer-use benchmark, Qwen-UI-Agent posts 79.5%, ahead of both GPT-5.5 and Gemini 3.1 Pro — on a test where comparable Western systems have been stalled in the low-to-mid 70s for months.
Two things separate this release from the usual benchmark press cycle. First, the weights are open: MAI-UI-2B and MAI-UI-8B are on Hugging Face, with the code on GitHub under Tongyi-MAI/MAI-UI. Second, the sizes matter. A 2B model that operates a phone UI can plausibly run on the phone itself, which changes the privacy calculus entirely — screen-reading agents from OpenAI and Anthropic ship screenshots of everything you do to a datacenter. An on-device agent doesn’t have to.
The training approach explains the real-device edge. Alibaba trained on data collected from over 100 physical mobile devices running 150 distinct applications, rather than leaning on emulators, and built a custom benchmark of more than 400 tasks. Real screens are messier than emulated ones — notifications interrupt, keyboards pop up, apps lag — and the 10-point gap between Qwen’s emulator and real-device scores running in the favorable direction suggests the messiness was in the training data all along.
Alibaba also shipped guardrails: the model refuses tasks it flags as illegal or high-risk, and halts at sensitive operations — payments, data deletion, privacy authorizations — requesting explicit user confirmation before proceeding. What nobody has answered is what happens when third parties fine-tune the open weights without those constraints. There’s no commercial API or deployment pricing announced, which reads as a deliberate choice: this is an ecosystem play, not a product.
The pattern is now unmistakable. Alibaba’s open-weight releases keep landing above closed frontier models on agentic benchmarks — GUI control today, after a year of Qwen models crowding the top of open-model leaderboards. For developers building automation on screen control, the calculus just shifted: the best-scoring mobile GUI agent available is one you can download, inspect, and run yourself.