minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models

Recent video diffusion foundation models have achieved remarkable progress in high-quality video generation, yet turning them into real-time interactive video world models remains challenging. Interactive world models require controllable, causal, and low-latency rollout, which in practice demands a full pipeline spanning data construction, controllable fine-tuning, autoregressive training, few-step distillation, and streaming inference. In this work, we present minWM, a full-stack open-source framework for building real-time interactive video world models. minWM provides an end-to-end pipeline that converts existing bidirectional T2V/TI2V video foundation models into camera-controllable few-step autoregressive world models. Specifically, minWM first fine-tunes a bidirectional video diffusion model with camera control, and then applies the Causal Forcing / Causal Forcing++ pipeline, including AR diffusion training, causal ODE or causal consistency distillation, and asymmetric DMD, to distill it into a few-step autoregressive generator for low-latency rollout. The framework is modular and architecture-extensible: we instantiate it on representative open backbones, including Wan2.1-T2V-1.3B and HY1.5-TI2V-8B, covering both cross-attention-based condition injection and MMDiT-style architectures. minWM also supports adapting existing video world models, such as HY-WorldPlay, to new data distributions, training recipes, and latency targets. Beyond releasing runnable scripts, checkpoints, documentation, and inference code, we provide practical ablations on camera trajectory quality, controllability training steps, and minimal batch-size requirements. We hope minWM serves as a reproducible and extensible recipe for building and adapting real-time interactive video world models. Project Page: [https://github.com/shengshu-ai/minWM](https://github.com/shengshu-ai/minWM)

📋 한 줄 요약

**[Video World Model / Full-Stack 오픈소스]** minWM이 bidirectional T2V/TI2V foundation 모델을 카메라 제어·causal forcing·few-step distillation·streaming inference로 real-time interactive video world model로 변환, Wan2.1-T2V-1.3B·HY1.5-TI2V-8B 등에서 인스턴스화.

🎯 핵심 기여도

최근 비디오 diffusion foundation 모델이 고품질 비디오 생성에서 remarkable 진보 보였지만 real-time interactive video world model로 전환은 여전히 도전적임을 진단.
Interactive world model은 controllable·causal·low-latency rollout 요구 — 데이터 구성·controllable fine-tuning·autoregressive 학습·few-step distillation·streaming inference 전 파이프라인 필요.
minWM — real-time interactive video world model 구축 full-stack 오픈소스 프레임워크. 기존 bidirectional T2V/TI2V 비디오 foundation 모델을 카메라 controllable few-step autoregressive world model로 변환하는 end-to-end 파이프라인.
minWM은 bidirectional video diffusion 모델을 먼저 카메라 제어로 fine-tune, 이후 Causal Forcing / Causal Forcing++ 파이프라인(AR diffusion 학습·causal ODE/causal consistency distillation·asymmetric DMD) 적용해 few-step autoregressive generator로 distill — low-latency rollout. Modular·architecture-extensible — Wan2.1-T2V-1.3B·HY1.5-TI2V-8B 등 대표 오픈 backbone(cross-attention 기반·MMDiT-style architecture)에서 인스턴스화. HY-WorldPlay 같은 기존 비디오 world model을 새 분포·레시피·latency target에 adapt 가능. 실행 가능 script·checkpoint·문서·inference code 외에도 camera trajectory 품질·controllability 학습 step·minimal batch size 등 실용 ablation 제공.

💡 핵심 아이디어

Real-time interactive video world model은 단일 알고리즘이 아닌 데이터·controllable fine-tuning·autoregressive 학습·few-step distillation·streaming inference의 full-stack 파이프라인 문제이며, 기존 bidirectional T2V/TI2V foundation 모델을 카메라 제어·causal forcing·DMD로 single recipe에 변환하는 modular 오픈소스 구현이 분야 진입 장벽을 낮춘다.

🔬 기술적 접근법

**방법론**: minWM — full-stack end-to-end 파이프라인.
**핵심 기법**: (1) Bidirectional video diffusion 모델 카메라 controllable fine-tuning, (2) Causal Forcing / Causal Forcing++로 AR diffusion 학습, (3) Causal ODE·causal consistency distillation, (4) Asymmetric DMD로 few-step autoregressive generator distill, (5) Modular 설계로 cross-attention·MMDiT 등 다양 backbone 지원, streaming inference 구현.

📊 주요 결과

Wan2.1-T2V-1.3B·HY1.5-TI2V-8B 등 대표 오픈 backbone에서 인스턴스화 성공.
Cross-attention 기반·MMDiT-style architecture 모두 커버.
실용 ablation 제공 — camera trajectory 품질·controllability 학습 step·minimal batch size.
HY-WorldPlay 등 기존 모델 adapt 검증.

💭 의의 및 한계

**의의**: Interactive video world model 분야의 reproducible·extensible recipe로 진입 장벽 대폭 감소, 다양 backbone·아키텍처 호환의 modular 설계, 실용 ablation·문서 제공. **한계**: Few-step distillation의 latency·품질 trade-off, 새 backbone 추가 시 fine-tuning 비용, 정량 비교 metric은 abstract에서 비명시.

🚀 실용적 활용

게임·시뮬레이션·VR의 real-time 비디오 world model.
로봇·AV 학습용 interactive 비디오 시뮬레이션.
오픈소스 video diffusion 연구 표준 레시피.