multimodal benchmark
AndroidWorld_SR Leaderboard
AndroidWorld_SR is a multimodal benchmark for autonomous agents operating on Android devices across 116 programmatic tasks in 20 real-world Android apps.
Updated Sep 5, 2026
Models8
Model coverage8
MetricScore
EvidenceB
AndroidWorld_SR Ranking
Higher score ranks better on this benchmark.
Sort by
Rank | Model | Score | Percentile | Participants | Evidence | Evaluated |
|---|
| Rank01 | ModelAC | Score71.10% | Percentile100.00% | Participants8 | EvidenceC | Evaluated |
| Rank02 | ModelAC | Score66.40% | Percentile85.71% | Participants8 | EvidenceC | Evaluated |
| Rank03 | ModelAC | Score64.20% | Percentile71.43% | Participants8 | EvidenceC | Evaluated |
| Rank04 | ModelAC | Score63.70% | Percentile57.14% | Participants8 | EvidenceC | Evaluated |
| Rank05 | ModelAC | Score63.70% | Percentile42.86% | Participants8 | EvidenceC | Evaluated |
| Rank06 | ModelAC | Score35.00% | Percentile28.57% | Participants8 | EvidenceC | Evaluated |
| Rank07 | ModelAC | Score25.50% | Percentile14.29% | Participants8 | EvidenceC | Evaluated |
| Rank08 | ModelAC | Score22.00% | Percentile0.00% | Participants8 | EvidenceC | Evaluated |
AndroidWorld_SR Highlights
The leading models and scores on this benchmark.
AndroidWorld_SR Score Distribution
A closer view of the leading scores on this benchmark.
The Top AI Models for AndroidWorld_SR
The first five results on this benchmark, with official price and output speed added where the model identity can be matched.
What is AndroidWorld_SR?
What AndroidWorld_SR measures and how its scores work.
AndroidWorld_SR is the AndroidWorld Success Rate (SR) benchmark, published at ICLR 2025, using screen screenshots, accessibility trees, and natural language instructions to evaluate autonomous agents on Android devices.
It measures agents' success rate in completing tasks such as sending messages, creating calendar events, and navigating mobile interfaces; its metric is Score with unit ratio.
Scores are shown in ratio. This benchmark is not independently verified and has an evidence level of B.
- Family
- AndroidWorld_SR
- Modality
- multimodal
- Primary category
- multimodal
- Score direction
- higher
- LLMBoard eligible
- No
- Evaluation key
- overall
LLM Stats. Benchmark scores retain their original unit. Overall score eligibility is shown separately.
FAQ
Common questions about AndroidWorld_SR.
Which model scores highest on AndroidWorld_SR?
Qwen3.5-35B-A3B is currently ranked first with 71.10%.
What are the top three models on AndroidWorld_SR?
The current leaders are Qwen3.5-35B-A3B (71.10%), Qwen3.5-122B-A10B (66.40%), and Qwen3.5-27B (64.20%).
Which AndroidWorld_SR model has the lowest official input price?
Qwen3.5-35B-A3B has the lowest matched official input price at $0.25 input / $2.0 output per 1M tokens.
Which models are fastest among AndroidWorld_SR results?
No matched runtime record is currently available.
Does the highest score result prove overall model quality?
No. This benchmark measures one defined capability or task. The overall LLMBoard score uses a separate aggregation across eligible benchmark evidence.