Most existing human rendering methods require every part of the human to be fully visible throughout the input video. However, this assumption does not hold in real-life settings where obstructions are common, resulting in only partial visibility of the human. Considering this, we present OccFusion, an approach that utilizes efficient 3D Gaussian splatting supervised by pretrained 2D diffusion models for efficient and high-fidelity human rendering. We propose a pipeline consisting of three stages. In the Initialization stage, complete human masks are generated from partial visibility masks. In the Optimization stage, 3D human Gaussians are optimized with additional supervision by Score-Distillation Sampling (SDS) to create a complete geometry of the human. Finally, in the Refinement stage, in-context inpainting is designed to further improve rendering quality on the less observed human body parts. We evaluate OccFusion on ZJU-MoCap and challenging OcMotion sequences and find that it achieves state-of-the-art performance in the rendering of occluded humans.
现有的大多数人类渲染方法要求视频中人体的每个部分都必须完全可见。然而，在现实生活中，遮挡是常见的，导致人体只能部分可见。基于此考虑，我们提出了OccFusion方法，它利用高效的三维高斯光滑技术，由预训练的二维扩散模型进行监督，实现高效且高保真度的人体渲染。我们提出了一个包含三个阶段的流程。在初始化阶段，从部分可见掩模生成完整的人体掩模。在优化阶段，通过得分蒸馏抽样（SDS）额外监督，优化三维人体高斯模型，以创建完整的人体几何形状。最后，在细化阶段，设计了上下文修复技术，进一步改善对少见人体部位的渲染质量。我们在ZJU-MoCap和具有挑战性的OcMotion序列上评估了OccFusion，并发现它在渲染遮挡人体方面达到了最先进的性能水平。