As consumer Virtual Reality (VR) and Mixed Reality (MR) technologies gain momentum, there's a growing focus on the development of engagements with 3D virtual content. Unfortunately, traditional techniques for content creation, editing, and interaction within these virtual spaces are fraught with difficulties. They tend to be not only engineering-intensive but also require extensive expertise, which adds to the frustration and inefficiency in virtual object manipulation. Our proposed VR-GS system represents a leap forward in human-centered 3D content interaction, offering a seamless and intuitive user experience. By developing a physical dynamics-aware interactive Gaussian Splatting in a Virtual Reality setting, and constructing a highly efficient two-level embedding strategy alongside deformable body simulations, VR-GS ensures real-time execution with highly realistic dynamic responses. The components of our Virtual Reality system are designed for high efficiency and effectiveness, starting from detailed scene reconstruction and object segmentation, advancing through multi-view image in-painting, and extending to interactive physics-based editing. The system also incorporates real-time deformation embedding and dynamic shadow casting, ensuring a comprehensive and engaging virtual experience.
随着消费级虚拟现实（VR）和混合现实（MR）技术的日益普及，对3D虚拟内容的开发与互动越来越受到关注。不幸的是，传统的内容创建、编辑和虚拟空间内的互动技术充满困难。这些技术不仅需要大量工程技术，还需要广泛的专业知识，这增加了虚拟对象操纵的挫败感和低效率。我们提出的VR-GS系统代表了人本中心的3D内容互动的一大飞跃，提供了无缝且直观的用户体验。通过在虚拟现实环境中开发一个意识到物理动力学的交互式高斯溅射，并构建一个高效的双层嵌入策略以及可变形体仿真，VR-GS确保了实时执行和高度逼真的动态响应。我们的虚拟现实系统组件旨在实现高效和有效，从详细的场景重建和对象分割开始，通过多视图图像修复，到交互式基于物理的编辑。该系统还包括实时形变嵌入和动态阴影投射，确保全面而引人入胜的虚拟体验。