ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning

Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e.g., cropping) has become a promising route to long-video understanding. However, existing native-RL methods dispatch tool calls sequentially (i.e., one per turn): a single wrong crop propagates errors without peer correction, multi-turn tool calls corrupt context, and inference cost scales linearly with the number of turns. We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn for cleaner context and better fault tolerance. Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling. A cross-model contrast on a weaker-prior LMM supports this claim: format stays stable but RL elicits zero tool calls, indicating that prior strength is the shared driver of both format collapse and tool exploration. We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO), which augments standard RL with two complementary mechanisms: (i) a targeted format reward applied only at the structural-token positions most prone to collapse, and (ii) a per-prompt frame-budget randomization that creates training prompts where calling the tool yields a measurable reward signal over skipping it. Across six long-video understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO lifting training-time format compliance from 0.13 to 0.64. As tool capabilities become increasingly internalized in modern LMMs, RL must cooperate with the resulting priors, and ParaVT offers a general recipe for agentic RL. Code, data, and model weights are publicly available.

📋 한 줄 요약

**[Agentic Video RL / Parallel Tools]** ParaVT가 multi-agent end-to-end RL로 video tool을 단일 turn에 병렬 dispatch — Tool Prior Paradox 진단, PARA-GRPO로 format reward·frame-budget randomization, 6 long-video 벤치마크 평균 +7.9%·format compliance 0.13→0.64.

🎯 핵심 기여도

대형 multimodal 모델(LMM)을 RL로 학습해 video 처리 도구(crop 등)를 natively 호출하는 것이 long-video 이해의 유망 경로지만 기존 native-RL 방법이 tool call을 sequentially dispatch(턴당 하나)해 single wrong crop이 peer correction 없이 오류 전파·multi-turn tool call이 context 오염·inference cost가 턴 수에 linear scale 됨을 지적.
ParaVT 도입 — Parallel Video Tool calling을 위한 첫 multi-agent end-to-end RL framework, 다중 time-window crop을 single turn에 dispatch해 cleaner context·better fault tolerance 달성.
표준 RL을 ParaVT에 적용 시 Tool Prior Paradox 발견 — tool exploration을 가능하게 하는 pretrained tool prior가 동시에 cold-started structural format을 destabilize하고 temperature sampling 하에서 skip-tool reward shortcut을 노출. Weaker-prior LMM의 cross-model contrast로 검증 — format은 안정이지만 RL이 zero tool call 유도, prior strength가 format collapse·tool exploration의 shared driver임 입증.
PARA-GRPO 제안(Parseability-Anchored, Ratio-gAted GRPO) — 두 보완 mechanism: (i) structural-token position 중 collapse에 가장 취약한 곳에만 targeted format reward, (ii) per-prompt frame-budget randomization으로 tool 호출이 skip보다 measurable reward signal을 산출하는 training prompt 생성. 6 long-video 벤치마크에서 Qwen3-VL baseline 대비 평균 +7.9%, PARA-GRPO가 training-time format compliance를 0.13→0.64로 향상.

💡 핵심 아이디어

Long-video 이해에서 tool call을 sequential 대신 parallel로 dispatch하면 fault tolerance·context cleanliness가 개선되지만, pretrained tool prior 강도가 format 안정성과 tool exploration의 동시 driver라는 paradox가 발생 — 이를 풀려면 structural collapse 지점에만 targeted format reward·prompt별 frame budget randomization으로 tool 호출의 reward signal을 명시적으로 만들어야 한다.

🔬 기술적 접근법

**방법론**: ParaVT — multi-agent end-to-end RL for parallel video tool calling + PARA-GRPO.
**핵심 기법**: (1) 다중 time-window crop을 single turn에 dispatch, (2) Cleaner context·fault tolerance 확보, (3) Tool Prior Paradox 메커니즘 분석, (4) PARA-GRPO — targeted format reward(structural-token collapse 지점만), (5) Per-prompt frame-budget randomization으로 tool reward signal 보장.

📊 주요 결과

6 long-video 이해 벤치마크 평균 Qwen3-VL baseline 대비 +7.9%.
PARA-GRPO가 training-time format compliance 0.13→0.64로 향상.
Tool Prior Paradox의 cross-model contrast 검증(weaker-prior LMM에서 RL → zero tool call).
Code·data·model weights 공개.

💭 의의 및 한계

**의의**: Long-video 이해의 효율적 parallel tool calling paradigm 정립, Tool Prior Paradox라는 RL 학습의 새 현상 식별, PARA-GRPO의 두 mechanism이 일반 적용 가능한 패턴, agentic RL의 general recipe로 의의. **한계**: 6 벤치마크 중심으로 다른 video task 일반화 추가 검증, parallel tool 수 확장의 한계는 별도 분석, prior 강도와 paradox의 정확 quantification은 후속.

🚀 실용적 활용

Long-video 이해 모델의 효율적 tool 사용.
Agentic LLM의 parallel tool calling 일반화.
RL 학습의 prior 호환성 설계 원리.