On-Policy Adversarial Flow Distillation for Autoregressive Video Generation

Autoregressive video generators are attractive for streaming, long-horizon, and interactive applications, but distilling strong black-box teachers into causal students remains difficult. The student must learn under its own rollout distribution, whereas practical teachers may expose only prompt-conditioned completed videos and may differ in architecture, capacity, temporal design, and sampling schedule. This interface makes supervised fine-tuning off-policy, score-based distillation inapplicable, and direct adversarial imitation too sparse for denoising-time credit assignment. We propose Adversarial Flow Distillation (AFD), an on-policy framework for heterogeneous black-box video distillation. AFD queries the teacher and rolls out the current student on the same prompts, trains a prompt-paired Bradley-Terry discriminator to estimate clean-sample teacher-student discrepancy, and converts the resulting on-policy advantage into forward-process flow-matching updates on the student's own noised states. Thus, AFD provides dense velocity-field supervision while requiring no teacher scores, latents, denoising trajectories, step alignment, or reverse-chain reinforcement learning. Experiments across two causal AR student families show that AFD consistently improves motion- and physics-sensitive generation while preserving general video quality, and ablations validate the importance of adaptive on-policy feedback and forward-process credit assignment. The method requires only clean teacher videos and student rollouts, providing a practical route for distilling proprietary or heterogeneous video generators into efficient autoregressive students.

📋 한 줄 요약

**[Video Generation / Black-Box Distillation]** AFD가 on-policy로 teacher·student 출력을 prompt-paired Bradley-Terry discriminator로 비교, advantage를 forward-process flow-matching update로 변환해 heterogeneous black-box video teacher를 causal AR student로 distill.

🎯 핵심 기여도

Autoregressive video generator가 streaming·long-horizon·interactive application에 매력적이지만 강력한 black-box teacher를 causal student로 distilling은 어렵다는 문제 — student가 자신의 rollout distribution 하에서 학습해야 하지만 실용 teacher는 prompt-conditioned 완성 비디오만 노출, 아키텍처·capacity·temporal design·sampling schedule 모두 상이.
이 interface가 supervised fine-tuning은 off-policy, score-based distillation은 불가, 직접 adversarial imitation은 denoising-time credit assignment에 너무 sparse하게 만듦을 지적.
AFD(Adversarial Flow Distillation) 제안 — heterogeneous black-box video distillation의 on-policy 프레임워크.
Teacher 질의·student rollout을 동일 prompt에서 수행, prompt-paired Bradley-Terry discriminator로 clean-sample teacher-student discrepancy 추정, 결과 on-policy advantage를 student 자신의 noised state에 forward-process flow-matching update로 변환 — teacher score·latent·denoising trajectory·step alignment·reverse-chain RL 모두 불필요.

💡 핵심 아이디어

Black-box heterogeneous video teacher의 distillation은 score·latent·denoising trajectory 접근 없이 가능해야 하며, teacher·student rollout 쌍을 Bradley-Terry discriminator로 비교해 얻은 on-policy advantage를 forward-process flow-matching update로 변환하면 sparse adversarial imitation의 한계를 우회하면서 dense velocity-field supervision을 얻을 수 있다.

🔬 기술적 접근법

**방법론**: AFD — on-policy adversarial flow distillation.
**핵심 기법**: (1) Teacher·student를 동일 prompt에서 rollout, (2) Prompt-paired Bradley-Terry discriminator로 clean-sample teacher-student discrepancy 추정, (3) On-policy advantage 계산, (4) Student의 noised state에 forward-process flow-matching update로 변환, (5) Teacher score·latent·denoising trajectory·step alignment·reverse-chain RL 없이 dense velocity-field supervision 제공.

📊 주요 결과

두 causal AR student family에서 AFD가 motion·physics-sensitive 생성을 일관 개선.
일반 video 품질 보존.
Ablation으로 adaptive on-policy feedback과 forward-process credit assignment의 중요성 입증.
Clean teacher 비디오와 student rollout만 요구 — proprietary·heterogeneous video generator distillation의 실용적 경로.

💭 의의 및 한계

**의의**: Black-box video distillation의 첫 on-policy 프레임워크, teacher 접근 최소화로 proprietary 모델 distillation 가능, motion·physics 민감 영역의 개선이 streaming AR 비디오에 핵심 가치, forward-process credit assignment 원리 정립. **한계**: 두 student family로 일반화 추가 검증, Bradley-Terry discriminator 품질에 강하게 의존, very long video에서 rollout 비용 우려.

🚀 실용적 활용

Proprietary video generator의 효율적 distillation.
Streaming·interactive AR video 모델 개발.
Heterogeneous teacher-student diffusion·flow 모델 일반 distill 기법.