Novel View Synthesis (NVS) from unconstrained photo collections is challenging in computer graphics. Recently, 3D Gaussian Splatting (3DGS) has shown promise for photorealistic and real-time NVS of static scenes. Building on 3DGS, we propose an efficient point-based differentiable rendering framework for scene reconstruction from photo collections. Our key innovation is a residual-based spherical harmonic coefficients transfer module that adapts 3DGS to varying lighting conditions and photometric post-processing. This lightweight module can be pre-computed and ensures efficient gradient propagation from rendered images to 3D Gaussian attributes. Additionally, we observe that the appearance encoder and the transient mask predictor, the two most critical parts of NVS from unconstrained photo collections, can be mutually beneficial. We introduce a plug-and-play lightweight spatial attention module to simultaneously predict transient occluders and latent appearance representation for each image. After training and preprocessing, our method aligns with the standard 3DGS format and rendering pipeline, facilitating seamlessly integration into various 3DGS applications. Extensive experiments on diverse datasets show our approach outperforms existing approaches on the rendering quality of novel view and appearance synthesis with high converge and rendering speed.
从不受限制的照片集合进行新视角合成（NVS）在计算机图形学中具有挑战性。最近，3D高斯涂抹（3DGS）在静态场景的光真实和实时新视角合成中显示出了前景。基于3DGS，我们提出了一个高效的基于点的可微分渲染框架，用于从照片集合重建场景。我们的关键创新是一个基于残差的球面谐波系数转移模块，该模块适应不同的照明条件和光度后处理，以调整3DGS。这个轻量级模块可以预计算，并确保从渲染图像到3D高斯属性的有效梯度传播。此外，我们观察到外观编码器和瞬态遮挡物预测器这两个不受限照片集合NVS中最关键的部分可以相互促进。我们引入了一个即插即用的轻量级空间注意力模块，用于同时预测每幅图像的瞬态遮挡物和潜在的外观表示。经过训练和预处理后，我们的方法与标准的3DGS格式和渲染管道对齐，便于无缝集成到各种3DGS应用中。在多样化数据集上进行的广泛实验表明，我们的方法在新视角和外观合成的渲染质量、收敛速度和渲染速度方面均优于现有方法。