Abstract
Vision Foundation Models (VFMs) have achieved remarkable success when applied to various downstream 2D tasks. Despite their effectiveness, they often exhibit a critical lack of 3D awareness. To this end, we introduce Splat and Distill, a framework that instills robust 3D awareness into 2D VFMs by augmenting the teacher model with a fast, feed-forward 3D reconstruction pipeline. Given 2D features produced by a teacher model, our method first lifts these features into an explicit 3D Gaussian representation, in a feedforward manner. These 3D features are then ``splatted" onto novel viewpoints, producing a set of novel 2D feature maps used to supervise the student model, ``distilling" geometrically grounded knowledge. By replacing slow per-scene optimization of prior work with our feed-forward lifting approach, our framework avoids feature-averaging artifacts, creating a dynamic learning process where the teacher's consistency improves alongside that of the student. We conduct a comprehensive evaluation on a suite of downstream tasks, including monocular depth estimation, surface normal estimation, multi-view correspondence, and semantic segmentation. Our method significantly outperforms prior works, not only achieving substantial gains in 3D awareness but also enhancing the underlying semantic richness of 2D features. Project page is available at this https URL
Abstract (translated)
视觉基础模型（VFMs）在各种下游二维任务中取得了显著的成功。尽管它们表现卓越，但这些模型通常缺乏对三维空间的理解。为此，我们提出了一种名为“Splat and Distill”的框架，该框架通过向教师模型添加快速的前馈式3D重建管道来增强2D VFMs的鲁棒性三维意识。给定由教师模型生成的二维特征，我们的方法首先以一种前馈的方式将这些特征提升到显式的3D高斯表示中。然后，将这些3D特征“投射”到新的视点上，产生一组用于监督学生模型的新2D特征图，从而“提炼”出几何基础的知识。通过用我们快速的前馈式提升方法替代先前工作的每个场景优化过程，我们的框架避免了特征平均化带来的伪影，并创建了一个动态学习过程，在这个过程中教师和学生的连贯性同步提高。 我们在一系列下游任务上进行了全面评估，包括单目深度估计、表面法线估计、多视图对应以及语义分割。与先前的工作相比，我们方法在三维感知方面取得了显著的性能提升，同时也增强了二维特征的基础语义丰富度。项目的网页可以在这个URL找到：[此链接应为原文中的具体项目页面链接，请根据实际情况插入正确的网址]
URL
https://arxiv.org/abs/2602.06032
