Abstract
3D Gaussian Splatting has achieved remarkable success in reconstructing both static and dynamic 3D scenes. However, in a scene represented by 3D Gaussian primitives, interactions between objects suffer from inaccurate 3D segmentation, imprecise deformation among different materials, and severe rendering artifacts. To address these challenges, we introduce PIG: Physically-Based Multi-Material Interaction with 3D Gaussians, a novel approach that combines 3D object segmentation with the simulation of interacting objects in high precision. Firstly, our method facilitates fast and accurate mapping from 2D pixels to 3D Gaussians, enabling precise 3D object-level segmentation. Secondly, we assign unique physical properties to correspondingly segmented objects within the scene for multi-material coupled interactions. Finally, we have successfully embedded constraint scales into deformation gradients, specifically clamping the scaling and rotation properties of the Gaussian primitives to eliminate artifacts and achieve geometric fidelity and visual consistency. Experimental results demonstrate that our method not only outperforms the state-of-the-art (SOTA) in terms of visual quality, but also opens up new directions and pipelines for the field of physically realistic scene generation.
Abstract (translated)
3D高斯点阵（Gaussian Splatting）在重建静态和动态的三维场景方面取得了显著的成功。然而，在由3D高斯原语表示的场景中，物体之间的交互受到不准确的三维分割、不同材料间变形的不精确性以及严重的渲染伪影的影响。为了应对这些挑战，我们引入了PIG：基于物理的多材料相互作用与3D高斯方法（Physically-Based Multi-Material Interaction with 3D Gaussians），这是一种结合了快速且精准地从2D像素映射到3D高斯点阵，并在物体级进行精确三维分割的新方法，同时模拟场景内交互对象之间的复杂物理现象。具体而言： 首先，我们的方法支持将二维像素快速、准确地映射为3D高斯，从而实现物体级别的精细三维分割。 其次，在场景中我们为不同材料的对应分割对象分配特定的物理属性，以实现多材料耦合互动。 最后，我们将约束尺度成功嵌入到变形梯度中，特别针对高斯原语的缩放和旋转特性进行了限制，这有助于消除伪影，并达到几何精确性和视觉一致性的目标。 实验结果表明，我们的方法不仅在视觉质量上超过了最先进的技术（SOTA），而且还为物理现实场景生成领域开辟了新的方向和技术途径。
URL
https://arxiv.org/abs/2506.07657
