Revising Densification in Gaussian Splatting

Abstract

In this paper, we address the limitations of Adaptive Density Control (ADC) in 3DGS, a scene representation method achieving high-quality, photo-realistic results for novel view synthesis. ADC has been introduced for automatic 3D point primitive management, controlling densification and pruning , however, with certain limitations in the densification logic.

Our main contribution is a more principled, pixel-error driven formulation for density control in 3DGS, leveraging an auxiliary, per-pixel error function as the criterion for densification.

We further introduce a mechanism to control the total number of primitives generated per scene and correct a bias in the current opacity handling strategy of ADC during cloning operations.

Our approach leads to consistent quality improvements across a variety of benchmark scenes, without sacrificing the method’s efficiency.

Figure

Figure 1

Densification is a critical component of 3DGS, and a common failure point.

3DGS fail to add primitives to high-texture areas, like the grass in the bottom part of the pictures, producing large and blurry artifacts.

Our approach solves this issue by comprehensivelyrevising densification in 3DGS.

Figure 2

Consider rendering a single splatted Gaussian in its center pixel with opacity before and after cloning.

Before we clone , the rendered color depends with weight on what comes next.

After we clone , since we preserve the opacity, the rendered color depends with weight on what comes next.

Since , we have a bias towards weighting more Gaussian primitives that get cloned. The proposed correction changes the opacity post clone to so that the bias is removed.

Figure 3

Evolution of thenumber of Gaussians in 3DGS, and in our method with upper limit set to the number reached by 3DGS (on the garden scene from the Mip-NeRF 360 dataset).

While 3DGS’ ADC process stops after 15k iterations, ours remains active for27k.

This is not immediately visible from the plot, since pruned primitives are immediately replaced by newly spawned ones , keeping the overall number stable once the maximum is reached.

Figure 4

Qualitative results on the Mip-NeRF 360, Tanks and Temples and Deep Blending validation sets. Note that 3DGS and Ours use the same number of primitives.

Figure 5

Qualitative results with highlights from Tanks and Temples and MipNeRF360 datasets. We compare ground-truth, 3DGS and our proposed method.

Figure 6

Qualitative result on MipNeRF360-Flowers scene.

(a) 3DGS with standard densification strategy , which yields 4.2M primitives.

(b) 3DGS with our proposed growing strategy and thresholding bypassed to push the number of primitives to 10M.

Figure 7

Qualitative result on a validation image from flowers with as thedensification guiding error.

Limitations

While our method appears to be quite effective at solving under-fitting issues, these can still be present in especially difficult scenes (e.g. treehill in the MipNeRF 360 dataset, both scenes from the Deep Blending dataset).

Focusing on the problematic areas that our ADC approach handles successfully, we observe that, while perceptually more “correct” , the reconstruction there can still be quite inaccurate when closely compared to the ground truth.

We suspect both these issues might be related to 3DGS’ intrinsiclimits in handling

i) strong view-dependent effects;

ii) appearance variations across images;

iii) errors induced by the linear approximation in the Splatting operation.

An interesting future direction could be to combine our approach with works that address these issues, e.g. Spec-Gaussian for**(i)** and GS++ for**(iii)**.

Conclusion

In this paper, we addressed the limitations of the Adaptive Density Control (ADC) mechanism in 3DGS, a scene representation method for high-quality, photorealistic rendering.

Our main contribution is a more principled, pixel-error driven formulation for density control in 3DGS.

We propose how to leverage a novel decision criterion for densification based on** per-pixel errors** and introduce a mechanism to control thetotal number of primitives generated per scene.

We also correct a bias in the current opacity handling in ADC during cloning.

Our approach leads to consistent and systematic improvements over previous methods, particularly in perceptual metrics like LPIPS.