Abstract
Understanding semantics and dynamics has been crucial for embodied agents in various tasks. Both tasks have much more data redundancy than the static scene understanding task. We formulate the view selection problem as an active learning problem, where the goal is to prioritize frames that provide the greatest information gain for model training. To this end, we propose an active learning algorithm with Fisher Information that quantifies the informativeness of candidate views with respect to both semantic Gaussian parameters and deformation networks. This formulation allows our method to jointly handle semantic reasoning and dynamic scene modeling, providing a principled alternative to heuristic or random strategies. We evaluate our method on large-scale static images and dynamic video datasets by selecting informative frames from multi-camera setups. Experimental results demonstrate that our approach consistently improves rendering quality and semantic segmentation performance, outperforming baseline methods based on random selection and uncertainty-based heuristics.
Abstract (translated)
理解语义和动态对于各种任务中的具身代理（embodied agents）至关重要。与静态场景理解任务相比，这些任务的数据冗余度更高。我们将视角选择问题形式化为一种主动学习问题，在这种情况下，目标是优先考虑那些能为模型训练提供最大信息增益的帧。为此，我们提出了一种基于Fisher Information的主动学习算法，该算法量化了候选视图在语义高斯参数和变形网络方面的相关信息性。这一方法使我们的技术能够同时处理语义推理和动态场景建模，并且为启发式或随机策略提供了一个有原则性的替代方案。 我们在大规模静态图像和动态视频数据集上通过从多摄像头设置中选择信息丰富的帧来评估我们提出的方法。实验结果表明，与基于随机选择和不确定性启发式的基线方法相比，我们的方法在渲染质量和语义分割性能方面始终表现出色。
URL
https://arxiv.org/abs/2512.22771
