NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance
Abstract
Existing neural implicit surface reconstruction methods have demonstrated notable success in multi-view 3D reconstruction by utilizing explicit geometry priors, such as depth maps or point clouds used for regularization. However, the reconstruction results still fall short in capturing fine details due to the excessive smoothness of the depth map or sparsity of the point cloud.
Within this study, we introduce a novel neural implicit surface reconstruction framework guided by 3DGS to synthesize highly detailed and intricate surfaces. A notable advantage of 3D Gaussian Splatting lies in its ability to produce dense point clouds that exhibit intricate structural details. Despite its effectiveness, a naive implementation of 3D Gaussian Splatting may fail because the generated points represent the centers of these Gaussians, which may not all lie on the desired surface.
By introducing a scale regularizer, we ensure that 3D Gaussians are made extremely narrow through which their centers are pulled close to the surface.
Additionally, our proposed method aims to enhance the point cloud derived from 3DGS by incorporating surface normal priors obtained from neural implicit models, avoiding reliance on a predetermined set of points.
Consequently, surface reconstruction quality improves through guidance by 3D Gaussian splatting with higher accuracy. By optimally combining 3D GS (3D栅格分割) and neural implicit models (NIM), our approach leverages both representations to produce complete surfaces featuring intricate details. Experiments conducted on Tanks and Temples validate the effectiveness of our proposed method.
Figure
Figure 1
A case study demonstrates the collaborative optimization between implicit surface reconstruction and 3DGS.
Figure 2
The NeuSG framework includes three principal components.
-
Optimization of neural implicit surface reconstruction.
-
Geometric constraints from point clouds generated from GS.
-
Refinement of Gaussian Splatting through normal alignment.
Figure 3
An illustration of 3D Gaussian flattening and normal alignment.
Figure 4
Qualitative comparison on Tanks and Temples dataset.
NeuSG demonstrates the capability to achieve fully realized and highly intricate surfaces, whereas baseline approaches typically lead to surfaces that either lack completeness or are contaminated by noise.
Figure 5
The scale regularizations for Gaussian Splatting.
Limitation
Like many neural implicit reconstruction approaches, our method relies on the number of available images, which can limit performance in scenarios with insufficient data. Accurate object reconstruction necessitates dense and diverse viewpoint observations. The reliance on extensive multi-view data can act as a significant limitation when comprehensive coverage is not achievable.
Additionally, our method excels in detailed surface reconstruction tasks but might encounter difficulties in environments where poorly distributed or irregularly spread out image distribution is prevalent, resulting in less precise outcomes.
Conclusion
In this research, we have developed a neural implicit surface reconstruction pipeline that is enhanced through the integration of 3DGS to facilitate the recovery of highly detailed and complete surfaces.
The principal benefit of 3D Gaussian Splatting stems from its ability to generate highly dense point clouds characterized by intricate structural details.
This spatial scaling regularization technique is proposed to convert 3D Gaussians into extremely slender forms, thereby causing the points to be drawn closer to the surface.
Furthermore, our method differs from the standard approach that employs static point sets as priors. Rather than using fixed points, we enhance the 3D Gaussians by incorporating normal priors obtained from surfaces predicted by neural implicit models.
This approach equips 3DGS with highly accurate directional guidance, resulting in improved surface reconstruction. By integrating both the 3DGS and neural implicit model into a unified framework, our proposed method combines the strengths of each technique to generate complete surfaces with exceptional detail.