CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillation

Customized image editing aims to equip pre-trained diffusion models with specific visual effects using limited paired data, typically via Low-Rank Adaptation (LoRA). As the number of desired effects grows, storing and dynamically loading numerous these effect LoRAs significantly increases deployment overhead. Furthermore, current pipelines typically cascade these effect LoRAs with acceleration modules for fast generation, which triggers severe parameter interference and results in concept bleeding and style degradation. We propose CollectionLoRA, a multi-teacher on-policy distillation framework capable of distilling the concepts of up to 50 different effect LoRAs along with few-step generation capabilities into a single LoRA. This fundamentally resolves the feature interference issue and significantly reduces deployment costs. Specifically, the method introduces (i) a Probabilistic Dual-Stream Routing mechanism that enables the model to randomly switch between data sources during training, effectively enhancing its generalization in unseen scenarios; (ii) an Asymmetric Orthogonal Prompting strategy to achieve concept isolation within the prompt space; (iii) a Coarse-to-Fine Distillation Objective to mitigate the distribution gap between the teacher and student models. Extensive evaluations show that CollectionLoRA distills all customized effects and few-step generation into a single LoRA, reducing deployment overhead while achieving concept fidelity comparable to or better than independently trained teacher models.

📋 한 줄 요약

**[Diffusion Customization / Multi-Teacher Distillation]** CollectionLoRA가 50개 effect LoRA + few-step generation을 single LoRA로 distill — Probabilistic Dual-Stream Routing·Asymmetric Orthogonal Prompting·Coarse-to-Fine Distillation으로 concept bleeding·style 저하 해소·배포 비용 절감.

🎯 핵심 기여도

Customized image editing이 pre-trained diffusion 모델에 specific 시각 effect를 limited 페어 데이터로 부여하는 것을 목표, 보통 LoRA로 수행.
원하는 effect 수가 증가하면 다수 effect LoRA 저장·동적 로드가 배포 overhead 크게 증가시키는 문제 지적.
현 파이프라인이 빠른 생성을 위해 effect LoRA를 acceleration 모듈과 cascade 사용 — severe parameter interference로 concept bleeding·style 저하 발생.
CollectionLoRA 제안 — 최대 50 다른 effect LoRA의 개념과 few-step 생성 능력을 단일 LoRA로 distill하는 multi-teacher on-policy distillation 프레임워크. Feature interference 근본 해결·배포 비용 대폭 감소. (i) Probabilistic Dual-Stream Routing으로 학습 중 데이터 source 무작위 전환해 unseen 시나리오 일반화 강화, (ii) Asymmetric Orthogonal Prompting으로 prompt space 내 concept isolation 달성, (iii) Coarse-to-Fine Distillation Objective로 teacher·student 분포 격차 완화.

💡 핵심 아이디어

50개 customization LoRA + few-step 생성을 단일 LoRA로 합치려면 multi-teacher distillation에 dual-stream routing(일반화)·orthogonal prompting(concept 분리)·coarse-to-fine objective(분포 격차)를 결합해 feature interference를 원천 차단해야 한다.

🔬 기술적 접근법

**방법론**: CollectionLoRA — multi-teacher on-policy distillation.
**핵심 기법**: (1) 50 effect LoRA의 개념을 단일 LoRA로 distill, (2) Probabilistic Dual-Stream Routing으로 학습 중 데이터 source 무작위 전환, (3) Asymmetric Orthogonal Prompting으로 prompt space concept isolation, (4) Coarse-to-Fine Distillation Objective로 teacher·student 분포 격차 완화, (5) Few-step 생성 능력 동시 distill.

📊 주요 결과

50 customized effect와 few-step 생성을 단일 LoRA로 모두 distill.
배포 overhead 대폭 감소.
독립 학습 teacher 모델과 비교해 concept fidelity 동급 또는 우수.
Concept bleeding·style 저하 등 cascade 문제 근본 해결.

💭 의의 및 한계

**의의**: Multi-LoRA 배포 부담의 근본 해결(50→1), few-step 생성과의 동시 distill로 실용 배포 friendly, dual-stream·orthogonal prompting·coarse-to-fine의 결합 레시피 정립. **한계**: 50 이상 effect로의 확장성, on-policy distillation의 학습 비용, teacher LoRA 품질 의존.

🚀 실용적 활용

상용 image editor의 effect 통합.
모바일·엣지 customized 생성 모델.
Few-step 생성·LoRA distillation 일반 레시피.