MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research

We present MobileGym, a browser-hosted, lightweight, fully controllable environment for everyday mobile use, targeting interaction fidelity without replicating proprietary backends. It enables two capabilities previously out of reach for everyday apps: verifiable outcome signals through deterministic state-based judging over structured JSON state, and scalable online RL through low-cost parallel rollouts. The full environment state is captured, configured, forked, and compared as structured JSON, and a single server can host hundreds of parallel instances, with about 400 MB memory per instance and about 3 s cold start. A layered state model and a declarative task-definition framework keep state programmability and task creation practical at scale, and a single programmatic judging mechanism delivers both deterministic evaluation verdicts and dense RL rewards. The accompanying MobileGym-Bench provides 416 parameterized task templates, including 256 test and 160 train templates, over 28 apps, with deterministic judges and a structured AnswerSheet protocol that avoids free-text matching failures. In a Sim-to-Real case study, GRPO on Qwen3-VL-4B-Instruct gains +12.8 percentage points on the 256-task test set, and on a 59-task real-device signal subset, real-device execution retains 95.1% of the simulation-side training gain. Project page: https://mobilegym.github.io.

📋 한 줄 요약

**[모바일 GUI Agent / 시뮬레이션 플랫폼]** MobileGym이 browser-hosted·deterministic JSON state 기반 검증·인스턴스당 400MB로 수백 병렬 — MobileGym-Bench 416 템플릿·28 앱·GRPO로 Qwen3-VL-4B +12.8pp·real-device 95.1% 학습 효과 retention.

🎯 핵심 기여도

MobileGym 제시 — browser-hosted, lightweight, fully controllable 환경으로 everyday mobile use 타겟, proprietary backend 복제 없이 interaction fidelity 추구.
일상 앱에서 previously out of reach였던 두 capability 활성화 — (1) 구조화 JSON state에 deterministic state-based judging으로 verifiable outcome signal, (2) low-cost parallel rollout으로 scalable online RL.
전체 환경 state를 구조화 JSON으로 capture·configure·fork·compare. 단일 서버에서 수백 parallel instance host, instance당 약 400MB 메모리·약 3s cold start.
Layered state model + declarative task-definition framework로 state programmability·task creation을 scale에서 실용적으로 유지, 단일 programmatic judging이 deterministic 평가 verdict와 dense RL reward 모두 제공. MobileGym-Bench — 416 parameterized task template(256 test + 160 train) 28 앱, deterministic judge·structured AnswerSheet protocol으로 free-text matching 실패 회피.

💡 핵심 아이디어

모바일 GUI agent 연구의 verifiable evaluation과 scalable RL을 양립시키려면 proprietary backend 복제가 아닌 structured JSON state로 환경을 완전 표현해야 하며, deterministic state-based judging이 평가 verdict와 dense reward를 동시에 제공하면서 수백 parallel rollout으로 RL을 scale할 수 있다.

🔬 기술적 접근법

**방법론**: MobileGym — browser-hosted JSON-state 시뮬레이터 + MobileGym-Bench + GRPO 적용.
**핵심 기법**: (1) 환경 state를 구조화 JSON으로 capture·fork·compare, (2) Layered state model + declarative task definition으로 scale에서 task 작성 실용성 유지, (3) Programmatic judging이 deterministic verdict + dense RL reward 동시 제공, (4) 416 parameterized template (256 test + 160 train), 28 앱, structured AnswerSheet, (5) Sim-to-real 검증 — Qwen3-VL-4B-Instruct에 GRPO 적용.

📊 주요 결과

단일 서버 수백 parallel instance, instance당 400MB·3s cold start.
416 task template, 28 앱, deterministic judge.
Sim-to-Real: GRPO로 Qwen3-VL-4B-Instruct가 256-task test set +12.8 percentage points.
59-task real-device signal subset에서 real-device 실행이 시뮬레이션 학습 효과의 95.1% retention.

💭 의의 및 한계

**의의**: 모바일 GUI agent의 verifiable·scalable RL 환경의 표준 후보, sim-to-real gap 95.1% retention으로 실용성 입증, deterministic JSON judging이 free-text 평가 한계 해소, 인프라 효율성(400MB/instance)으로 학계 접근성. **한계**: Proprietary backend 미복제로 실 앱과의 행동 격차 존재, 28 앱 커버리지 한계, browser-hosted 환경의 실 디바이스 UI 격차.

🚀 실용적 활용

모바일 자동화·GUI agent 학습.
LLM-driven 앱 테스팅·UX 검증.
멀티 앱 task automation 연구.