DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes
Abstract
We introduce DrivingGaussian as an efficient and effective framework for dynamic surroundings in autonomous driving scenarios.
For complex scenes with moving objects ,
we initially construct the static background of the whole scene using stepwise incremental 3D Gaussians.
After that, we implemented a composite dynamic Gaussian graph to manage multiple moving objects. For each object, we individually reconstructed its structure and restored its exact positions as well as its occlusion relationships within the scene.
Additionally, we incorporate a LiDAR prior in the context of Gaussian Splatting to achieve more detailed scene reconstruction, ensuring global consistency.
DrivingGaussian surpasses current approaches in the context of dynamic driving scene reconstruction, while enabling the production of photorealistic surround-view images that maintain both high fidelity and multi-camera coherence.
Figure
Figure 1
DrivingGaussian deliversstate-of-the-art photorealistic rendering performance in surrounding dynamic autonomous driving environments.
Naive approaches are prone to generating unpleasant artifacts and blurring effects in the large-scale background, while they also face difficulties in reconstructing dynamic objects and capturing detailed scene geometry.
Driving Gaussian by introducing Enhanced Gaussian Mapping, the method effectively represents stationary backgrounds and various moving entities within complex driving environments.
DrivingGaussian permits the high-quality generation of surrounding views in multi-camera setups and assists in achieving long-term, dynamic scene reconstruction.
Figure 2
Overall pipeline of our method.
Left: DrivingGaussian acquires sequence data across multiple sensors, such as images captured by multiple cameras and LiDAR.
Middle: 为了表示大型动态驾驶场景, 我们提出复合高斯投射, 其由两个组件构成
The first segment assembles the large-scale static background incrementally, while the second generates multiple dynamic objects using a Gaussian graph and integrates them into the scene in a dynamic manner.
DrivingGaussian exhibits superior performance across diverse task domains and application contexts.
Figure 3
A Composite-based Splatting Method Utilizing Incrementally Static 3D Gaussian Structures and Dynamic Gaussian Graph Structures.
Composite Gaussian Splatting is employed to disassemble the entire scene into static backgrounds and moving objects, with each segment independently reconstructed before merging into a comprehensive rendering framework.
Figure 4
Qualitative comparison on dynamic reconstruction.
We present comprehensive analysis results in nuScenes by comparing with our leading competitors, EmerNeRF and 3DGS, on real-time reconstruction tasks.
DrivingGaussian facilitates the achievement of high-quality dynamic object reconstruction with a high rate, while ensuring temporal coherence.
Figure 5
Analysis of visualization techniques involves evaluating the performance of various initialization approaches on the KITTI-360 dataset.
Analysis of visualization techniques involves evaluating the performance of various initialization approaches on the KITTI-360 dataset.
In contrast to the initialization using SfM points, the application of LiDAR prior permits Gaussian estimation of more accurate geometric structures within the scene.
Figure 6
Example of corner case simulation.
Extreme case simulation based on the DrivingGaussian model: When a pedestrian is crossing the street, they fall unexpectedly, and a vehicle approaches from ahead.
Figure 7
Visualization of surrounding multi-camera views in nuScenes dataset.
The surrounding views exhibit minor overlaps among multiple cameras, however, and exhibit major intervals over time.
Figure 8
Analysis of bin distribution within the context of incremental static 3D Gaussians.
The small overlap between two adjacent bins is employed to match the static background characteristics of the two bins.
Figure 9
Visualization of Global Rendering vis GS.
DrivingGaussian enables the reconstruction of multiple dynamic objects alongside precise positioning information and occlusion relationships.
Figure 10
Qualitative comparison on the nuScenes dataset.
We present a comprehensive evaluation of the qualitative comparison results against our key competitors NSG, EmerNeRF, and 3DGS in terms of driving scene generation for the nuScenes dataset.
Figure 11
Qualitative comparison on the KITTI-360 dataset.
We present the comparative analysis outcomes when evaluating against our primary competitors, DNMP and 3DGS, in reconstructing driving scenes from the KITTI-360 dataset.
Figure 12
Qualitative analysis of various initialization techniques for 3D Gaussians.
LiDAR-based prior information for 3D Gaussians assists in the acquisition of more accurate geometric configurations and highly precise surface features.
Figure 13
Rendering with or w/o the Incremental Static 3D Gaussians (IS3G).
IS3G is capable of ensuring excellent geometry and rigorous topology integrity for stationary environments within extensive driving scenarios.
Figure 14
Rendering with or w/o the Composite Dynamic Gaussian Graph (CDGG).
The CDGG framework achieves the reconstruction of dynamic objects within driving scenarios at any desired speed, including vehicles, bicycles, and pedestrians.
Figure 15
Failure Cases.
Distortions\ are\ prevalent\ in\ small\ objects\ and\ reflective\ surfaces\ such as\ roadside\ pebbles\ and\ glass. The approach does not require excessive Gaussian allocation when modeling large-scale static backgrounds, thus circumventing high computational costs.
Conclusion
We present DrivingGaussian as a cutting-edge framework for modeling large-scale dynamic autonomous driving scenarios built upon the concept of Composite Gaussian Splatting.
DrivingGaussian逐步建模了静态背景，并使用递增的静态三维高斯函数捕获多个移动物体。进一步利用LiDAR先验来实现精确的几何结构和多视图一致性。
DrivingGaussian demonstrates world-leading effectiveness within the domains of autonomous vehicle testing, showcasing its capability of generating highly detailed 360-degree surrounding views alongside real-time dynamic scene reconstruction.