RaDe-GS: Rasterizing Depth in Gaussian Splatting

Abstract

3DGS demonstrates remarkable effectiveness within the domain of novel view synthesis, capable of achieving high-quality and real-time rendering. However, there is still a significant unexplored capability concerning the reconstruction of detailed 3D shapes.

现有方法通常面临形状精度受限的问题, 这是由于高斯小波的离散且非结构化特性所导致的, 从而使得形状提取变得困难.

While recent techniques, such as 2DGS, have sought to enhance shape reconstruction, they commonly reformulate Gaussian primitives with the goal of improving rendering quality while maintaining computational efficiency.

为了解决这些问题, 我们的研究提出了一个基于栅格化的方法：用于生成深度图和法线图的通用三维高斯球体。

Through our method, it is notable that the 3D shape reconstruction fidelity is notably enhanced, while simultaneously maintaining the computational efficiency foundational to Gaussian Splatting.

The system demonstrates a Chamfer distance error on par with NeuraLangelo on the DTU dataset and retains similar computational efficiency compared to traditional 3D GS approaches.

Our method represents a revolutionary breakthrough within Gaussian Splatting and can seamlessly integrate into existing Gaussian Splatting-based systems.

Figure

Figure 1

We present a rasterized method to compute the depth and surface normal maps of general Gaussian splats.

Our method is capable of performing high-fidelity 3D shape reconstruction while maintaining superior performance in both training and rendering.

When compared to enforcing Gaussian splats to maintain a planar structure akin to that of 2DGS, the approach leads to blurry novel view rendering and introduces noise into the 3D shape. Moreover, 2DGS natively manages noise post mesh extraction and offers a computationally efficient solution.

Figure 2

Quality results on Mip-NeRF 360 dataset.

Figure 3

Description of the local affine transformations in Gaussian Splatting. It approximates the perspective transformations by a local parallel transformation for each Gaussian splat.

Figure 4

Intersecting a radiant beam with a three-dimensional Gaussian distribution within the camera coordinate system and ray coordinate system.

The Gaussian center

is transformed to

.

A point

is transformed to

.

The green curve represents the collection of intersection points resulting from the Gaussian's interaction with diverging light rays.

Figure 5

A comparative analysis of our method with existing Gaussian-based approaches within the DTU dataset.

Figure 6

Analysis of our approach against existing GS-based techniques in the domain of novel view synthesis.

This method has achieved high-quality images. In contrast, other methods create unsharp results around The benches and show signs of artifacts on The distant wall, as illustrated in Figure 1.

Figure 7

Comparative analysis of our method and previous Gaussian-based methods for the Synthetic NeRF dataset.

Figure 8

Surface reconstruction results on the Tanks & Templs dataset.

Figure 9

Surface reconstruction results on the DTU dataset.

Figure 10

An extra analysis of novel viewpoints' synthesis between our approach and previous methods. The testing scenes for our experiments are sourced from the Tanks & Temples and Mip-NeRF360 datasets.

Limitation And Conclusion

Introducing RaDe-GS, we highlight its innovative approach by incorporating a rasterization-based method to calculate the depth and surface normal maps. This advancement enhances 3DGS's 3D shape reconstruction performance while maintaining its training efficiency and rendering performance.

Our method is based on commonly used Gaussian primitives, and it can be seamlessly integrated into existing 3DGS-based methods.

Our experiments on diverse public datasets demonstrate that RaDe-GS outperforms existing implicit and explicit approaches.

---

Our current TSDF fusion is based on coarse-resolution voxel grids for large-scale scenes, which prevents the accurate extraction of Gaussian surfaces. The self-adaptive TSDF-based methods perform hierarchically extracting meshes while significantly reducing memory usage and maintaining high geometric accuracy.

1. 面对包含有反射表面的对象，
   我们的办法无法精确重建这些反射表面，
   这些表面受到使用于3DGS的基本颜色函数的限制。
2. 这个问题可以通过结合高阶的颜色表示方法得以缓解，
   而这个方法已经被GaussianShader所采用。