Recent advances in novel view synthesis (NVS), particularly neural radiance fields (NeRF) and Gaussian splatting (3DGS), have demonstrated impressive results in photorealistic scene rendering. These techniques hold great potential for applications in virtual tourism and teleportation, where immersive realism is crucial. However, the high-performance demands of virtual reality (VR) systems present challenges in directly utilizing even such fast-to-render scene representations like 3DGS due to latency and computational constraints. In this paper, we propose foveated rendering as a promising solution to these obstacles. We analyze state-of-the-art NVS methods with respect to their rendering performance and compatibility with the human visual system. Our approach introduces a novel foveated rendering approach for Virtual Reality, that leverages the sharp, detailed output of neural point rendering for the foveal region, fused with a smooth rendering of 3DGS for the peripheral vision. Our evaluation confirms that perceived sharpness and detail-richness are increased by our approach compared to a standard VR-ready 3DGS configuration. Our system meets the necessary performance requirements for real-time VR interactions, ultimately enhancing the user's immersive experience.
最新的视图合成 (NVS) 技术，尤其是神经辐射场 (NeRF) 和高斯喷涂 (3D Gaussian Splatting, 3DGS)，在真实感场景渲染方面展示了令人印象深刻的效果。这些技术在虚拟旅游和远程呈现等需要高度沉浸感的应用中具有巨大潜力。然而，虚拟现实 (VR) 系统对高性能的需求使得直接使用即便是快速渲染的场景表示（如 3DGS）也面临延迟和计算限制方面的挑战。 本文提出了注视点渲染作为应对这些挑战的有效解决方案。我们分析了最新 NVS 方法在渲染性能及其与人类视觉系统兼容性方面的表现。我们的方法为 VR 引入了一种创新的注视点渲染技术，利用神经点渲染的清晰细节输出处理注视区域，并结合 3DGS 平滑渲染处理外周视觉区域。评估结果表明，与标准 VR 适配的 3DGS 配置相比，我们的方法显著提升了感知清晰度和细节丰富度。 我们的系统满足实时 VR 交互所需的性能要求，从而进一步增强了用户的沉浸式体验。