Dynamic reconstruction of deformable tissues in endoscopic video is a key technology for robot-assisted surgery. Recent reconstruction methods based on neural radiance fields (NeRFs) have achieved remarkable results in the reconstruction of surgical scenes. However, based on implicit representation, NeRFs struggle to capture the intricate details of objects in the scene and cannot achieve real-time rendering. In addition, restricted single view perception and occluded instruments also propose special challenges in surgical scene reconstruction. To address these issues, we develop SurgicalGaussian, a deformable 3D Gaussian Splatting method to model dynamic surgical scenes. Our approach models the spatio-temporal features of soft tissues at each time stamp via a forward-mapping deformation MLP and regularization to constrain local 3D Gaussians to comply with consistent movement. With the depth initialization strategy and tool mask-guided training, our method can remove surgical instruments and reconstruct high-fidelity surgical scenes. Through experiments on various surgical videos, our network outperforms existing method on many aspects, including rendering quality, rendering speed and GPU usage. The project page can be found at this https URL.
内窥镜视频中可变形组织的动态重建是机器人辅助手术的一项关键技术。最近基于神经辐射场(NeRF)的重建方法在手术场景重建方面取得了显著成果。然而，由于基于隐式表示，NeRF 难以捕捉场景中物体的精细细节，且无法实现实时渲染。此外，受限的单视图感知和被遮挡的器械也为手术场景重建提出了特殊挑战。为解决这些问题，我们开发了 SurgicalGaussian，一种可变形的 3D 高斯散射方法来建模动态手术场景。我们的方法通过前向映射变形 MLP 和正则化来建模每个时间戳的软组织时空特征，约束局部 3D 高斯符合一致运动。通过深度初始化策略和工具掩码引导训练，我们的方法可以去除手术器械并重建高保真度的手术场景。通过在各种手术视频上的实验，我们的网络在渲染质量、渲染速度和 GPU 使用等多个方面都优于现有方法。