Reconstructing and simulating elastic objects from visual observations is crucial for applications in computer vision and robotics. Existing methods, such as 3D Gaussians, provide modeling for 3D appearance and geometry but lack the ability to simulate physical properties or optimize parameters for heterogeneous objects. We propose Spring-Gaus, a novel framework that integrates 3D Gaussians with physics-based simulation for reconstructing and simulating elastic objects from multi-view videos. Our method utilizes a 3D Spring-Mass model, enabling the optimization of physical parameters at the individual point level while decoupling the learning of physics and appearance. This approach achieves great sample efficiency, enhances generalization, and reduces sensitivity to the distribution of simulation particles. We evaluate Spring-Gaus on both synthetic and real-world datasets, demonstrating accurate reconstruction and simulation of elastic objects. This includes future prediction and simulation under varying initial states and environmental parameters.
从视觉观察重建和模拟弹性对象对于计算机视觉和机器人学的应用至关重要。现有方法，如3D高斯，为3D外观和几何提供了建模，但缺乏模拟物理属性或为异质对象优化参数的能力。我们提出了一种名为Spring-Gaus的新型框架，将3D高斯与基于物理的模拟整合起来，用于从多视角视频重建和模拟弹性对象。我们的方法利用了一个3D弹簧质量模型，使得在个别点水平上优化物理参数成为可能，同时解耦了物理和外观的学习。这种方法实现了极高的样本效率，增强了泛化能力，并减少了对模拟粒子分布的敏感性。我们在合成和现实世界数据集上评估了Spring-Gaus，展示了准确重建和模拟弹性对象的能力。这包括在不同初始状态和环境参数下的未来预测和模拟。