GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering

Abstract

During the 3DGS optimization process, the scene's geometry is prone to gradual degrading if its structural integrity isn't meticulously maintained. Particularly concerning non-textured areas such as walls, ceilings, and furniture surfaces. This degradation has a significant impact on rendering quality when novel views diverge substantially from those in training datasets.

To address this challenge, we introduce a naming convention for GeoGaussian structures. By examining smoothly connected regions in point clouds, our method systematically establishes thin Gaussians oriented along surface features using an innovative pipeline. The initialization process ensures that these Gaussians align precisely with the geometric characteristics of the surfaces. Through our carefully crafted densification strategy, these features are propagated to subsequent generations while maintaining their structural integrity and functional properties.

By the pipeline, the scene's geometry and texture are preserved via constrained optimization processes with geometric constraints.

Thanks to the proposed architecture, the generative capability of 3D Gaussians has been notably enhanced, particularly in structured regions. The proposed pipeline demonstrates state-of-the-art performance in novel view synthesis and geometric reconstruction through qualitative and quantitative evaluations on public datasets.

Figure

Figure 1

An analysis of novel viewpoint rendering techniques compared to the 3D Gaussian model across the Replica Datasets.

GeoGaussian is capable of clearly delineating an exceptionally sharp contrast between two low-textured surfaces. However, 3DGS exhibits poor resolution characteristics due to inaccuracies inherent in its underlying 3D Gaussian model, which manifests as observed blurring effects in this particular area.

Figure 2

Geometry-aware strategies of GeoGaussian.

Within smoothly connected regions, the parameterisation of thin Gaussians exhibits clear geometric significance within the mean vector and covariance matrix.

注

smoothness constraint with photometric constraint

Figure 3

Comparisons of novel view rendering on public datasets.

When encountering challenging viewpoints that exhibit greater disparities in translation and orientation movements than those in the training set, the 3DGS and LightGS face difficulties achieving photorealistic rendering.

(d) shows the two training views around the novel one.

Figure 4

Statistics of the number of Gaussians in sequences of Replica.

Figure 5

Performance metrics during training and evaluation indicate significant rendering efficiency. Each curve within (b) was generated during the evaluation phase, reflecting the system's performance characteristics.

Figure 6

由GeoGaussian和3DGS生成的高斯模型基于复制序列进行了分析

Figure 7

Viewpoints are utilized during the training phase and evaluation phase, where the red camera is specifically employed for assessment.

Figure 8

系统性地分析训练数据集与评估数据集的渲染性能对比研究

Figure 9

Comparisons of novel view rendering on the ICL-NUIM datasets.

In these scenarios, 3DGS and LightGS achieve photorealistic rendering.

Figure 10

Scene of the lag-cabinet sequence and render quality.

Figure 11

Reconstructionerror visualization.

Figure 12

Rendering performance in the first 10,000 iterations.

Conclusion

In this paper, we present a new method named GeoGaussian, which highlights the significance of maintaining precise geometric features in Gaussian models to improve their depiction in three-dimensional space.

Our method first imparts a more precise geometric interpretation to Gaussian parameters. The third scalar in the scale vector serves to regulate thickness, while the third axis in rotation matrices identifies and defines the primary orientation axis for thin ellipsoids, with its primary orientation derived from initial point cloud data.

Additionally, we propose a carefully designed densification approach to effectively organize the newly generated ellipsoids.

Within the optimization module, we promote ellipsoids in the neighborhood to be situated within a co-planar region, with enhanced precision to improve representation quality.

Experimental results on public datasets show the effectiveness of our method as far as geometry precision and The ability to render novel views in a photo-realistic manner compared to SOTA approaches.

In the future, we anticipate developing diverse strategies to enhance the geometry of Gaussian models. These measures will integrate depth, normals, and camera poses into the 3DGS optimization process, minimizing their dependence on point cloud normal vectors.