Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
Qiuyue Wang, Mingsheng Li, Jian Guan, Jinhui Ye, Sicheng Xie, Yitao Liu, Junhao Chen, Zhixuan Liang, Jie Zhang, Xintong Hu, Xuhong Huang, Pei Lin, Junyang Lin, Dayiheng Liu, Shuai Bai, Jingren Zhou, Jiazhao Zhang, Haoqi Yuan, Gengze Zhou, Hang Yin, Ye Wang, Yiyang Huang, Zixing Lei, Wujian Peng, Delin Chen, Yingming Zheng, Jingyang Fan, Xianwei Zhuang, Xin Zhou, Haoyang Li, Anzhe Chen, Tong Zhang, Xuejing Liu, Yuchong Sun, Ruizhe Chen, Zhaohai Li, Chenxu Lü, Zhibo Yang, Tao Yu, Xionghui Chen
arXiv:2605.30280 · 2026-05-29 공개 · arXiv · PDF
robot-manipulation vla foundation-model multi-task-learning embodied-intelligence ood-generalization navigation trajectory-prediction
Abstract
Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments. In this work, we study whether heterogeneous embodied decision-making problems can be unified within a single vision-language-action model. We present Qwen-VLA, a unified embodied foundation model that extends Qwen's vision-language modeling stack from perception, understanding, and reasoning to continuous action and trajectory generation through a DiT-based action decoder. Qwen-VLA is trained with a large-scale joint pretraining recipe over diverse data sources, including robotics manipulation trajectories, human egocentric demonstrations, synthetic simulation data, vision-and-language navigation data, trajectory-centric supervision, and auxiliary vision-language data. To support multiple robot platforms, we introduce embodiment-aware prompt conditioning, where robot-specific textual descriptions specify the current embodiment and control convention. We further cast manipulation, navigation, and trajectory prediction into a unified action-and-trajectory prediction framework, enabling transferable visual grounding, spatial reasoning, and continuous action generation across robot morphologies, task families, and environments. Experiments on manipulation, navigation, and trajectory-centric benchmarks show consistent multi-task performance and out-of-distribution generalization under variations in scene layout, background, lighting, object configuration, and robot embodiment. Qwen-VLA-Instruct achieves 97.9% on LIBERO, 73.7% on Simpler-WidowX, 86.1%/87.2% on RoboTwin-Easy/Hard, 69.0% OSR on R2R, 59.6% SR on RxR, 76.9% average OOD success in real-world ALOHA experiments, and 26.6% zero-shot success on DOMINO dynamic manipulation.
한국어 요약
📋 한 줄 요약
**[VLA / Unified Embodied Foundation]** Qwen-VLA가 Qwen VL stack에 DiT 기반 action decoder 결합, embodiment-aware prompt로 조작·내비·궤적을 단일 모델 통합 — LIBERO 97.9%·Simpler-WidowX 73.7%·RoboTwin 86.1/87.2%·R2R OSR 69.0%·실제 ALOHA OOD 76.9%·DOMINO zero-shot 26.6%.
🎯 핵심 기여도
- Embodied intelligence가 보통 조작·내비 등 개별 task별 특화 모델로 연구되어 task·환경·로봇 embodiment에 걸쳐 능력 분절·일반화 한계 가졌음을 진단.
- 이질적 embodied decision-making 문제를 단일 vision-language-action 모델로 통합 가능한지 연구.
- Qwen-VLA — Qwen의 VL 모델링 stack을 perception·이해·추론에서 continuous action·궤적 생성으로 DiT 기반 action decoder를 통해 확장한 unified embodied foundation model.
- 로봇 조작 trajectory·인간 egocentric demonstration·합성 시뮬레이션·VLN·trajectory-centric supervision·auxiliary VL data 등 다양 source에서 large-scale joint pretraining. Embodiment-aware prompt conditioning(로봇별 텍스트 설명으로 embodiment·제어 convention 지정)·조작·내비·궤적 예측을 unified action·trajectory 예측 프레임워크로 변환.
💡 핵심 아이디어
조작·내비·궤적 등 이질적 embodied decision을 단일 VLA로 통합하려면 VL stack에 DiT action decoder를 결합하고, embodiment-aware prompt로 로봇 종류를 텍스트로 명시하며, 다양 source의 joint pretraining으로 transferable visual grounding·spatial reasoning을 확보해야 한다.
🔬 기술적 접근법
- **방법론**: Qwen-VLA — Qwen VL stack + DiT action decoder + embodiment-aware prompt + 통합 action·trajectory 예측.
- **핵심 기법**: (1) Qwen VL을 perception·이해·추론·action·trajectory로 확장, (2) DiT 기반 action decoder로 continuous action·궤적 생성, (3) 로봇 조작·VLN·human egocentric·simulation·trajectory data joint pretraining, (4) Embodiment-aware prompt conditioning으로 로봇·제어 convention 명시, (5) 조작·내비·궤적 예측을 통합 framework로 변환.
📊 주요 결과
- LIBERO 97.9% (조작).
- Simpler-WidowX 73.7%.
- RoboTwin-Easy/Hard 86.1% / 87.2%.
- R2R OSR 69.0%, RxR SR 59.6% (내비).
- 실세계 ALOHA OOD 평균 76.9% success.
- DOMINO dynamic 조작 zero-shot 26.6% success.
- 장면·배경·조명·물체·embodiment 변화 하에서 OOD 일반화 일관 확인.
💭 의의 및 한계
**의의**: Embodied AI의 task·환경·embodiment 분절 문제를 단일 모델로 해소, DiT action decoder의 효과 입증, 다양 source joint pretraining의 transferability 정량화, 실세계 OOD 성능 검증. **한계**: 매우 dynamic·dexterous 조작에서 zero-shot 26.6%는 여전히 도전, embodiment-aware prompt 디자인 부담, 더 다양·소형 embodiment 일반화는 후속.
🚀 실용적 활용
- 일반 로봇 조작·내비·궤적 통합 정책.
- 다중 embodiment 산업 로봇 배포.
- VLA의 표준 backbone.