PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects

Simulation-ready physical 3D assets have emerged as a promising direction owing to their broad applicability in downstream tasks. However, most existing 3D generation methods either neglect physical properties or are limited to a single asset category, e.g., rigid, deformable, or articulated objects. To address these limitations, we introduce PhysX-Omni, a unified framework for simulation-ready physical 3D generation across diverse asset types. Specifically, we develop a novel and efficient geometry representation tailored for Vision-Language Models, which directly encodes high-resolution 3D structures without compression, significantly improving generation performance. In addition, we construct the first general simulation-ready 3D dataset, PhysXVerse, covering diverse indoor and outdoor categories. Furthermore, to comprehensively and flexibly evaluate both generative and understanding capabilities in the wild, we propose PhysX-Bench, which encompasses six key attributes: geometry, absolute scale, material, affordance, kinematics, and function description. Extensive experiments with conventional metrics and PhysX-Bench show that PhysX-Omni performs strongly in both generation and understanding. Moreover, additional studies further validate the potential of PhysX-Omni for applications in simulation-ready scene generation and robotic policy learning. We believe PhysX-Omni can significantly advance a wide range of downstream applications, particularly in embodied AI and physics-based simulation.

📋 한 줄 요약

**[Physical 3D Generation / Unified Simulation-Ready]** PhysX-Omni가 rigid·deformable·articulated 통합 simulation-ready 3D 생성 — VLM 친화 압축 없는 고해상도 표현·PhysXVerse 데이터셋·6 속성 PhysX-Bench로 종합 평가.

🎯 핵심 기여도

Simulation-ready physical 3D asset이 downstream task에서 broad 응용으로 promising 방향이지만, 기존 3D 생성 방법이 physical property를 neglect하거나 single asset category(rigid·deformable·articulated 중 하나)에만 한정됨을 진단.
PhysX-Omni 제안 — 다양 asset type 전반의 simulation-ready physical 3D 생성을 통합하는 framework.
VLM에 맞춘 새 efficient geometry representation 개발 — 압축 없이 고해상도 3D 구조를 직접 encode, generation 성능 substantially 개선.
최초의 general simulation-ready 3D dataset PhysXVerse 구축(다양 indoor·outdoor category), comprehensive·flexible 평가용 PhysX-Bench(geometry·absolute scale·material·affordance·kinematics·function description 6 핵심 속성) 제안.

💡 핵심 아이디어

Simulation-ready 3D 생성은 asset category(rigid·deformable·articulated)별 분리가 아니라 단일 framework로 통합해야 하며, VLM에 맞는 압축 없는 고해상도 geometry 표현과 6 속성 평가·다양 카테고리 데이터셋이 generation·understanding 양면 발전의 핵심 인프라다.

🔬 기술적 접근법

**방법론**: PhysX-Omni — unified framework + 압축 없는 geometry representation + PhysXVerse + PhysX-Bench.
**핵심 기법**: (1) 다양 asset(rigid·deformable·articulated)을 통합하는 generation framework, (2) VLM 친화의 압축 없는 고해상도 3D geometry representation, (3) PhysXVerse — 다양 indoor·outdoor category의 simulation-ready 3D dataset, (4) PhysX-Bench — geometry·absolute scale·material·affordance·kinematics·function description 6 속성 평가, (5) Simulation-ready scene generation·robotic policy learning 응용 검증.

📊 주요 결과

PhysX-Bench와 conventional metric 모두에서 PhysX-Omni가 generation·understanding 양면 강한 성능.
Simulation-ready scene generation·robotic policy learning에서 응용 잠재력 검증.
다양 asset category·indoor/outdoor 일반화.

💭 의의 및 한계

**의의**: 3D 생성의 simulation readiness·physical property를 first-class로 격상, asset category 통합으로 분야 단편화 해소, embodied AI·physics-based simulation 응용의 직접 인프라. **한계**: VLM 친화 표현의 매우 고해상도 scaling 비용, PhysXVerse 카테고리 외 일반화 추가 검증, simulation-ready의 실제 sim engine 호환성 의존.

🚀 실용적 활용

Embodied AI·robotics simulation environment 자동 구축.
Game·VR의 physics-aware asset 생성.
자동 robot policy learning을 위한 합성 데이터.