Gaze estimation encounters generalization challenges when dealing with out-of-distribution data. To address this problem, recent methods use neural radiance fields (NeRF) to generate augmented data. However, existing methods based on NeRF are computationally expensive and lack facial details. 3D Gaussian Splatting (3DGS) has become the prevailing representation of neural fields. While 3DGS has been extensively examined in head avatars, it faces challenges with accurate gaze control and generalization across different subjects. In this work, we propose GazeGaussian, a high-fidelity gaze redirection method that uses a two-stream 3DGS model to represent the face and eye regions separately. By leveraging the unstructured nature of 3DGS, we develop a novel eye representation for rigid eye rotation based on the target gaze direction. To enhance synthesis generalization across various subjects, we integrate an expression-conditional module to guide the neural renderer. Comprehensive experiments show that GazeGaussian outperforms existing methods in rendering speed, gaze redirection accuracy, and facial synthesis across multiple datasets. We also demonstrate that existing gaze estimation methods can leverage GazeGaussian to improve their generalization performance.
凝视估计在处理分布外数据时面临泛化挑战。为解决这一问题，近期方法尝试使用 NeRF（神经辐射场）生成增强数据。然而，基于 NeRF 的现有方法计算代价高昂且缺乏面部细节。随着 3D Gaussian Splatting (3DGS) 成为神经场的主流表示，其在头部头像建模中已有广泛应用，但在准确的凝视控制和跨主体的泛化方面仍存在挑战。 为此，我们提出 GazeGaussian，一种高保真凝视重定向方法，使用双流 3DGS 模型分别表示面部和眼睛区域。通过利用 3DGS 的非结构化特性，我们设计了一种基于目标凝视方向的刚性眼球旋转新颖表示方法。为增强在不同主体间的合成泛化能力，我们引入了一个 表情条件模块，以引导神经渲染器。 全面实验表明，GazeGaussian 在渲染速度、凝视重定向精度以及面部合成质量上均优于现有方法，并在多个数据集上表现出卓越的性能。此外，我们进一步证明，现有的凝视估计方法可以利用 GazeGaussian 提升其泛化能力，从而改进对分布外数据的适应性。