TriSplat: Simulation-Ready Feed-Forward 3D Scene Reconstruction

Sparse-view 3D reconstruction is increasingly addressed with feed-forward splatting networks that predict explicit primitives directly from images. Yet most existing methods remain centered on Gaussian primitives and expose surfaces only indirectly: extracting a usable mesh for downstream simulation, physics reasoning, or embodied interaction still requires expensive post-hoc steps that break the feed-forward promise. This limitation is especially pronounced in pose-free settings, where scene structure and camera parameters must be estimated jointly from sparse observations. We present TriSplat, a feed-forward reconstruction network that represents scenes with oriented triangle primitives and directly exports simulation-ready mesh scenes from a single forward pass. Given input images, the network predicts local 3D point maps, triangle attributes, camera poses, and optional intrinsics. Rather than regressing triangle orientation as an unconstrained latent variable, our approach constructs geometry normals from the predicted point maps, refines them with an image-conditioned normal head, and converts them into stable local frames for triangle parameterization. A mono-normal bootstrap schedule further stabilizes early training, while opacity and blur scheduling progressively sharpens the learned surface representation for direct mesh extraction. Experiments on RealEstate10K and DL3DV show that this representation produces more geometry-faithful reconstructions than Gaussian feed-forward baselines while maintaining competitive novel-view rendering quality. Because the rendering primitives are themselves surface triangles, the output can be directly ingested by physics engines, collision detectors, and standard rendering pipelines without any conversion, making it a practical simulation-ready solution for feed-forward 3D scene reconstruction.

📋 한 줄 요약

**[Feed-Forward 3D / Triangle Splat]** TriSplat이 oriented triangle primitive로 single forward pass에서 simulation-ready mesh scene을 출력 — predicted point map에서 normal 구성·image-conditioned normal head·mono-normal bootstrap·opacity·blur schedule, RealEstate10K·DL3DV에서 Gaussian baseline보다 geometry-faithful.

🎯 핵심 기여도

Sparse-view 3D reconstruction이 feed-forward splatting network로 빠르게 발전하고 있으나 기존 방법이 대부분 Gaussian primitive 중심으로 surface를 indirectly 노출 — downstream simulation·physics·embodied interaction을 위한 mesh 추출에 expensive post-hoc 단계가 필요해 feed-forward 약속을 깬다는 한계 진단.
Pose-free 환경에서는 scene structure·camera parameter를 sparse observation에서 jointly 추정해야 해 한계가 더 두드러짐.
TriSplat 제시 — oriented triangle primitive로 scene을 표현하고, single forward pass로 simulation-ready mesh scene을 직접 export하는 feed-forward reconstruction network. Network는 local 3D point map·triangle attribute·camera pose·optional intrinsic 예측.
Triangle orientation을 unconstrained latent로 회귀하지 않고 predicted point map에서 geometry normal을 구성, image-conditioned normal head로 정제 후 stable local frame으로 변환해 triangle parameterization. Mono-normal bootstrap schedule이 초기 학습 안정화, opacity·blur scheduling이 surface 표현을 점진 sharpening.

💡 핵심 아이디어

Feed-forward 3D reconstruction의 simulation-ready 출력은 Gaussian primitive 대신 oriented triangle을 사용하고, triangle orientation을 unconstrained latent로 두지 않고 predicted point map에서 normal을 구성·정제하는 geometry-first 설계로 가능하며, mono-normal bootstrap·opacity·blur schedule이 학습 안정성과 surface sharpness를 동시에 확보한다.

🔬 기술적 접근법

**방법론**: TriSplat — oriented triangle primitive feed-forward reconstruction.
**핵심 기법**: (1) 단일 forward pass로 point map·triangle attribute·camera pose·intrinsic 예측, (2) Point map에서 geometry normal 구성 후 image-conditioned normal head로 정제, (3) Normal을 stable local frame으로 변환해 triangle parameterize, (4) Mono-normal bootstrap schedule로 초기 학습 안정, (5) Opacity·blur scheduling으로 surface 표현 progressive sharpening.

📊 주요 결과

RealEstate10K·DL3DV에서 Gaussian feed-forward baseline 대비 more geometry-faithful reconstruction.
경쟁력 있는 novel-view rendering 품질 유지.
Rendering primitive 자체가 surface triangle이므로 physics engine·collision detector·standard rendering pipeline에 직접 ingest 가능 — 변환 없음.
Practical simulation-ready 솔루션 제공.

💭 의의 및 한계

**의의**: Feed-forward 3D의 simulation-ready 출력이라는 약속을 mesh post-hoc 없이 달성, geometry-first 접근의 triangle primitive 채택 정당화, pose-free 환경에서도 동작. **한계**: Triangle primitive의 매우 fine 디테일·thin structure 표현 한계 가능, RealEstate10K·DL3DV 외 야외·동적 scene 일반화 추가 검증, 학습 안정화 schedule의 hyperparameter 부담.

🚀 실용적 활용

시뮬레이션·물리 엔진용 3D scene 생성.
자율주행·로봇 embodied AI 환경 reconstruction.
VR/AR mesh-based asset 자동 생성.