WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation

Interactive world models are advancing rapidly, yet existing benchmarks cover only part of the required competencies, leaving no unified standard for systematic evaluation. To fill this gap, we introduce WBench, a comprehensive multi-turn benchmark for interactive world model evaluation along five dimensions, namely video quality, setting adherence, interaction adherence, consistency, and physics compliance. WBench contains 289 test cases and 1,058 interaction turns, where each case specifies a world setting and a multi-turn interaction sequence, covering diverse scenes, styles, subjects, and both first- and third-person perspectives, together with four interaction types, including navigation, subject action, event editing, and perspective switching. For navigation, WBench unifies text, 6-DoF pose, and discrete-action control, enabling evaluation of models with different native input interfaces. Evaluation uses 22 automatic sub-metrics that combine specialist vision models with large multimodal models, and all metrics are validated against human judgments. Across 20 state-of-the-art models, we find that no single model performs strongly across all dimensions. We provide detailed diagnostic insights into the characteristic strengths, weaknesses, and open challenges of each model. Code and data are available at https://github.com/meituan-longcat/WBench.

📋 한 줄 요약

**[Interactive World Model 벤치마크]** WBench가 289 test case·1,058 interaction turn으로 5 dimension(video quality·setting adherence·interaction adherence·consistency·physics compliance)에서 20 SOTA world model 평가, 22 자동 sub-metric이 human judgment에 검증됨.

🎯 핵심 기여도

Interactive world model이 빠르게 발전하지만 기존 벤치마크가 요구 능력의 일부만 커버해 systematic 평가의 unified standard가 부재하다는 격차 진단.
WBench 도입 — 5 dimension(video quality·setting adherence·interaction adherence·consistency·physics compliance)의 종합 multi-turn 벤치마크.
289 test case·1,058 interaction turn — 각 case는 world setting과 multi-turn interaction sequence를 명시, diverse scene·style·subject 및 first-/third-person perspective 모두 커버.
4 interaction type — navigation·subject action·event editing·perspective switching. Navigation은 text·6-DoF pose·discrete-action control을 unify해 native input interface가 다른 모델 평가 가능.

💡 핵심 아이디어

Interactive world model의 진정한 능력 평가에는 단일 지표가 아닌 다차원·다턴 시나리오·multiple control modality가 필요하며, specialist vision model과 large multimodal model을 결합한 자동 sub-metric을 human judgment에 검증하면 reproducible·comprehensive 평가가 가능하다.

🔬 기술적 접근법

**방법론**: WBench — 5 dimension multi-turn 종합 벤치마크.
**핵심 기법**: (1) 5 dimension(video quality·setting adherence·interaction adherence·consistency·physics compliance), (2) 4 interaction type(navigation·subject action·event editing·perspective switching), (3) Navigation에 text·6-DoF pose·discrete-action control unify, (4) 22 자동 sub-metric — specialist vision model + large multimodal model 결합, (5) 모든 metric을 human judgment에 validation.

📊 주요 결과

289 test case·1,058 interaction turn의 대규모 평가.
20 SOTA 모델 평가 결과 모든 차원에서 강력한 단일 모델 없음.
각 모델의 characteristic strength·weakness·open challenge에 대한 상세 진단 인사이트 제공.
Code·data 공개: github.com/meituan-longcat/WBench.

💭 의의 및 한계

**의의**: Interactive world model의 통일 평가 표준 정립, 다 modality control input을 unify해 native interface 다양성 흡수, 22 자동 metric의 human judgment 검증으로 신뢰성 확보. **한계**: 289 case·1,058 turn이 모든 시나리오 커버 못함, automatic sub-metric의 human alignment는 평균적 — task별 차이 존재 가능, 새 interaction type 추가 시 메트릭 재설계 부담.

🚀 실용적 활용

Interactive world model 개발의 평가 표준.
World model architecture 비교·진단 도구.
Navigation·event editing 등 specific capability의 ablation.