Abstract
3D Visual Question Answering (3D VQA) is crucial for enabling models to perceive the physical world and perform spatial reasoning. In 3D VQA, the free-form nature of answers often leads to improper annotations that can confuse or mislead models when training on the entire dataset. While other text generation tasks can mitigate this issue by learning on large-scale datasets, the scarcity of 3D scene data enlarges the negative effect of misleading annotations. Although active learning strategies can select valuable instances for training, they fail to identify and resolve misleading labels, which the oracle inevitably provides in practice. To address this issue, we propose a multi-turn interactive active learning strategy. This strategy selects data based on models' semantic uncertainty to form a solid knowledge foundation more effectively and actively requests reannotation from an oracle to resolve potentially misleading labels. For uncertainty assessment, we utilize a variance-based metric that takes semantic relationships between terms into consideration, thus avoiding the uniform inter-class similarity assumption of previous assessment metrics. Extensive experiments exhibit better model performance and a substantial reduction in training costs, with a halving of training costs for achieving relatively high accuracy. The code is available at this https URL.
Abstract (translated)
3D 视觉问答（3D VQA）对于使模型感知物理世界并进行空间推理至关重要。在 3D VQA 中，答案的自由形式性质通常会导致不正确的标注，在整个数据集上训练时可能会混淆或误导模型。虽然其他文本生成任务可以通过大规模数据集的学习来减轻这一问题的影响，但三维场景数据的稀缺性加剧了误导性标签带来的负面影响。尽管主动学习策略可以挑选有价值的实例用于训练，它们却无法识别和解决由实际操作中提供的潜在误导性标签。为了解决这个问题，我们提出了一种多轮互动式主动学习策略。该策略基于模型语义不确定性来选择数据，从而更有效地形成坚实的知识基础，并积极请求来自专家（oracle）的重新标注以解决潜在误导性的标签问题。在不确定性评估方面，我们使用一种考虑术语间语义关系的方差基元度量方法，从而避免了先前评估指标中的一致类内相似性假设。广泛的实验展示了模型性能的提升和训练成本的大幅降低，在实现相对较高的准确性时，训练成本可以减少一半。代码可在该网址获取：[请在此处插入具体的URL链接]。
URL
https://arxiv.org/abs/2507.04630
