High-quality scene reconstruction and novel view synthesis based on Gaussian Splatting (3DGS) typically require steady, high-quality photographs, often impractical to capture with handheld cameras. We present a method that adapts to camera motion and allows high-quality scene reconstruction with handheld video data suffering from motion blur and rolling shutter distortion. Our approach is based on detailed modelling of the physical image formation process and utilizes velocities estimated using visual-inertial odometry (VIO). Camera poses are considered non-static during the exposure time of a single image frame and camera poses are further optimized in the reconstruction process. We formulate a differentiable rendering pipeline that leverages screen space approximation to efficiently incorporate rolling-shutter and motion blur effects into the 3DGS framework. Our results with both synthetic and real data demonstrate superior performance in mitigating camera motion over existing methods, thereby advancing 3DGS in naturalistic settings.
高质量场景重建和新视角合成基于高斯喷溅（3DGS）通常需要稳定、高质量的照片，这往往难以通过手持相机捕捉实现。我们提出了一种方法，该方法能够适应相机运动，并允许使用受运动模糊和卷帘快门畸变影响的手持视频数据进行高质量场景重建。我们的方法基于对物理图像形成过程的详细建模，并利用视觉-惯性测程（VIO）估计出的速度。考虑到单个图像帧的曝光时间内相机姿态是非静态的，并且在重建过程中进一步优化相机姿态。我们构建了一个可微渲染管线，该管线利用屏幕空间近似高效地将卷帘快门和运动模糊效果纳入到3DGS框架中。我们使用合成数据和真实数据的结果展示了在减轻相机运动方面相较于现有方法的优越性能，从而推进了3DGS在自然场景设置中的应用。