Native3D: End-to-End 3D Scene Generation via Unified Mesh-Texture Modeling and Semantic Alignment

Native3D introduces an end-to-end 3D scene generation framework that eliminates the need for 2D intermediate representations, using a unified mesh-texture modeling approach with semantic alignment to improve geometric and textural fidelity compared to traditional diffusion model-based methods.

Native3D addresses a fundamental bottleneck in 3D generative AI by eliminating the 2D-to-3D domain adaptation pipeline that has plagued previous approaches. Traditional methods rely on pre-trained 2D diffusion models, forcing 3D data into 2D representations before reconstruction—a process that inherently introduces geometric distortion and texture degradation. This research bypasses that entire workflow by developing a native 3D pipeline from the ground up, representing a meaningful architectural shift in how generative models handle volumetric content.

The innovation centers on two technical contributions: a unified mesh-texture joint representation processed through a Transformer encoder, and the 3D REPA Loss that uses contrastive learning to align semantic information across multiple representation levels. This dual approach maintains spatial coherence and visual consistency simultaneously, addressing the trade-off that has limited previous single-representation methods.

For the 3D generation industry, this framework has significant implications. It promises faster generation speeds by eliminating intermediate conversion steps and higher output quality through native 3D optimization. Game developers, VFX studios, and 3D content creators could benefit from improved editing flexibility and fidelity, potentially accelerating adoption of generative 3D tools in production pipelines. The research also demonstrates practical advantages in scene editing capabilities, suggesting the approach scales beyond simple generation to interactive creative workflows.

The broader importance lies in establishing that 3D-native architectures outperform domain-adapted alternatives. This validates a research direction that could reshape how generative models handle volumetric, spatial data across industries, potentially influencing development priorities for major AI labs.

• →Native3D eliminates the 2D intermediate representation step that has caused geometric distortion in previous 3D generation methods
• →The unified mesh-texture modeling approach maintains spatial relationships and visual consistency through Transformer-based scene encoding
• →3D REPA Loss uses contrastive learning to align multi-level semantic representations, enhancing both geometric and textural quality
• →The framework demonstrates superior generation quality and editing flexibility compared to existing domain-adapted approaches
• →Direct 3D-native generation suggests faster pipelines and better scalability for production use in game development and VFX