CLONE: A 3DGS-Based Closed-Loop Differentiable Optimization Framework for Single-Image Normal Estimation

Researchers introduce CLONE, a 3D Gaussian Splatting-based framework that estimates surface normals from single images by creating a closed-loop differentiable optimization pathway. The method unifies discriminative and generative approaches through an image-geometry-image consistency loop, eliminating the need for explicit normal supervision while maintaining geometric accuracy and local detail.

CLONE addresses a fundamental challenge in computer vision: estimating accurate 3D surface normals from 2D images without ground-truth supervision. Traditional discriminative methods require explicit normal annotations, while generative approaches lack stable optimization pathways despite possessing strong learned priors. This research bridges both paradigms through an innovative closed-loop architecture.

The framework leverages 3D Gaussian Splatting to explicitly parameterize scenes and extract continuous surface normals via covariance decomposition, providing direct analytical gradients for geometric optimization. By introducing a differentiable illumination model with learnable light modulation, the system establishes a bidirectional mapping between geometry and image appearance. Reprojection errors then supervise the underlying 3D geometry directly, creating the closed-loop constraint that prevents the multi-solution collapse plaguing previous methods.

To address Gaussian representations' limited local detail expressiveness, the authors implement a deterministic diffusion-inspired refinement network coordinated through cross-domain gating fusion. This maintains end-to-end differentiability while enhancing local geometric fidelity. All components optimize jointly under unified reprojection objectives, establishing stable gradient propagation without requiring paired normal supervision.

This advancement impacts 3D reconstruction, autonomous systems, and computer graphics pipelines. Industries relying on accurate geometry estimation—robotics, AR/VR, autonomous vehicles—benefit from improved single-image reconstruction. The method's unsupervised approach reduces annotation bottlenecks, accelerating training data pipelines. Future applications may extend to multi-view scenarios and real-time processing, reshaping how systems understand 3D environments from limited visual information.

• →CLONE creates a closed-loop differentiable optimization framework eliminating the need for ground-truth normal supervision in single-image estimation
• →3D Gaussian Splatting provides analytical gradient pathways while a differentiable illumination model enables direct geometric supervision through reprojection errors
• →A diffusion-inspired refinement network enhances local geometric details while preserving end-to-end differentiability through cross-domain gating fusion
• →The framework unifies discriminative and generative paradigms, addressing limitations in explicit supervision and optimization stability simultaneously
• →Unsupervised normal estimation reduces annotation requirements and accelerates training for 3D reconstruction in robotics, AR/VR, and autonomous systems