GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation

Open-ended image generation is no longer a simple prompt-to-image problem. High-quality generation often requires an agent to combine a model's internal generative ability with external resources. As requests become more diverse and demanding, we aim to develop a general image-generation agent that can self-evolve through trajectories and use tools more effectively across varied generation challenges. To this end, we propose GenEvolve, a self-evolving framework based on Tool-Orchestrated Visual Experience Distillation. In GenEvolve, each generation attempt is modeled as a tool-orchestrated trajectory, where the agent gathers evidence, selects references, invokes generation skills, and composes them into a prompt-reference program. Unlike existing agentic generation methods that mainly rely on image-level scalar rewards, GenEvolve compares multiple trajectories for the same request and abstracts best-worst differences into structured visual experience, provided only to a privileged teacher branch. Inspired by on-policy self-distillation, Visual Experience Distillation provides dense token-level supervision, helping the student internalize better search, knowledge activation, reference selection, and prompt construction. We further construct GenEvolve-Data and GenEvolve-Bench. Experiments on public benchmarks and GenEvolve-Bench show substantial gains over strong baselines, achieving state-of-the-art performance among current image-generation frameworks. Our website is as follows: https://ephemeral182.github.io/GenEvolve/

📋 한 줄 요약

**[Image Generation Agent / Self-Evolving]** GenEvolve가 generation 시도를 tool-orchestrated trajectory로 모델링·trajectory 간 best-worst 차이를 structured visual experience로 distill해 teacher branch에 제공, dense token-level supervision으로 student 자기 진화.

🎯 핵심 기여도

Open-ended 이미지 생성이 더 이상 단순 prompt-to-image 문제가 아니라 agent가 model의 internal generative ability와 external resource를 결합해야 하는 영역으로 진화했음을 인식.
다양·까다로운 요청에 대응하는 general image-generation agent — trajectory를 통한 self-evolve와 도전마다 효과적 tool 사용 — 가 목표.
GenEvolve 제안 — Tool-Orchestrated Visual Experience Distillation 기반 self-evolving framework.
각 generation 시도를 tool-orchestrated trajectory로 모델링(agent가 evidence 수집·reference 선택·generation skill 호출·prompt-reference program 구성) — 기존 agentic 방법이 image-level scalar reward에만 의존하는 한계를 넘어, 같은 요청의 multiple trajectory를 비교해 best-worst 차이를 structured visual experience로 추상화·privileged teacher branch에만 제공.

💡 핵심 아이디어

이미지 생성 agent의 self-evolution은 image-level scalar reward 대신 같은 request의 multiple trajectory를 best-worst로 비교해 structured visual experience를 distill하고, teacher branch에 dense token-level supervision을 제공해 student가 검색·지식 활성화·reference 선택·prompt 구성을 internalize하도록 만드는 데서 가능하다.

🔬 기술적 접근법

**방법론**: GenEvolve — Tool-Orchestrated Visual Experience Distillation.
**핵심 기법**: (1) Generation 시도를 tool-orchestrated trajectory로 모델링(evidence·reference·skill·prompt-reference program), (2) 같은 request의 multiple trajectory 비교, (3) Best-worst 차이를 structured visual experience로 추상화, (4) Privileged teacher branch에만 experience 제공, (5) On-policy self-distillation에서 영감, dense token-level supervision으로 search·knowledge activation·reference selection·prompt construction internalize.

📊 주요 결과

GenEvolve-Data와 GenEvolve-Bench 구축.
Public benchmark·GenEvolve-Bench에서 강한 baseline 대비 substantial gain.
현재 image-generation framework 중 SOTA.
Web: ephemeral182.github.io/GenEvolve.

💭 의의 및 한계

**의의**: 이미지 생성을 prompt-to-image에서 agentic·tool-use·self-evolution paradigm으로 확장, scalar reward의 한계를 trajectory 비교·structured distillation으로 우회, dense token-level supervision의 generative agent 학습 적용. **한계**: Privileged teacher branch 구성·trajectory 큐레이션 비용, structured experience 추상화의 generality는 task 의존, agentic 추론 overhead.

🚀 실용적 활용

Creative·professional 이미지 생성 agent.
Tool-use 기반 multi-step generation 서비스.
Self-distillation paradigm의 일반 generative 학습 패턴.