We present a novel appearance model that simultaneously realizes explicit high-quality 3D surface mesh recovery and photorealistic novel view synthesis from sparse view samples. Our key idea is to model the underlying scene geometry Mesh as an Atlas of Charts which we render with 2D Gaussian surfels (MAtCha Gaussians). MAtCha distills high-frequency scene surface details from an off-the-shelf monocular depth estimator and refines it through Gaussian surfel rendering. The Gaussian surfels are attached to the charts on the fly, satisfying photorealism of neural volumetric rendering and crisp geometry of a mesh model, i.e., two seemingly contradicting goals in a single model. At the core of MAtCha lies a novel neural deformation model and a structure loss that preserve the fine surface details distilled from learned monocular depths while addressing their fundamental scale ambiguities. Results of extensive experimental validation demonstrate MAtCha's state-of-the-art quality of surface reconstruction and photorealism on-par with top contenders but with dramatic reduction in the number of input views and computational time. We believe MAtCha will serve as a foundational tool for any visual application in vision, graphics, and robotics that require explicit geometry in addition to photorealism.
我们提出了一种新颖的外观模型，可以同时实现高质量的3D表面网格重建和基于稀疏视图样本的真实感新视角合成。我们的核心思想是将底层场景几何网格（Mesh）建模为一组二维图表构成的图集（Atlas of Charts），并通过二维高斯面元（Gaussian surfels）进行渲染，称为 MAtCha Gaussians。MAtCha 利用现成的单目深度估计器提取场景表面的高频细节，并通过高斯面元渲染进一步优化这些细节。 高斯面元动态附加到图表上，从而在一个模型中同时实现神经体积渲染的真实感和网格模型的清晰几何结构，即解决了两个看似矛盾的目标。MAtCha 的核心是一种新颖的神经变形模型和结构损失函数，这些创新既保留了从学习的单目深度中提取的细致表面细节，又解决了深度估计固有的尺度歧义问题。 广泛的实验验证表明，MAtCha 在表面重建质量和真实感方面达到了当前最先进的水平，与顶尖方法相当，同时显著减少了所需的输入视图数量和计算时间。我们相信，MAtCha 将成为视觉、图形和机器人领域中任何需要几何显式表示和真实感的视觉应用的基础工具。