Abstract
Systematic failures of computer vision models on subsets with coherent visual patterns, known as error slices, pose a critical challenge for robust model evaluation. Existing slice discovery methods are primarily developed for image classification, limiting their applicability to multi-instance tasks such as detection, segmentation, and pose estimation. In real-world scenarios, error slices often arise from corner cases involving complex visual relationships, where existing instance-level approaches lacking fine-grained reasoning struggle to yield meaningful insights. Moreover, current benchmarks are typically tailored to specific algorithms or biased toward image classification, with artificial ground truth that fails to reflect real model failures. To address these limitations, we propose SliceLens, a hypothesis-driven framework that leverages LLMs and VLMs to generate and verify diverse failure hypotheses through grounded visual reasoning, enabling reliable identification of fine-grained and interpretable error slices. We further introduce FeSD (Fine-grained Slice Discovery), the first benchmark specifically designed for evaluating fine-grained error slice discovery across instance-level vision tasks, featuring expert-annotated and carefully refined ground-truth slices with precise grounding to local error regions. Extensive experiments on both existing benchmarks and FeSD demonstrate that SliceLens achieves state-of-the-art performance, improving Precision@10 by 0.42 (0.73 vs. 0.31) on FeSD, and identifies interpretable slices that facilitate actionable model improvements, as validated through model repair experiments.
Abstract (translated)
计算机视觉模型在具有连贯视觉模式的子集上出现系统性错误，这些错误被称为误差切片（error slices），对稳健的模型评估构成了重大挑战。现有的切片发现方法主要针对图像分类任务开发，这限制了它们在检测、分割和姿态估计等多实例任务中的应用范围。在实际场景中，误差切片通常源于涉及复杂视觉关系的边缘情况，在这种情况下，现有仅基于单个实例的方法由于缺乏细粒度推理而难以提供有价值的见解。此外，现有的基准测试通常针对特定算法或偏向于图像分类，并且其人工真值无法准确反映模型的实际错误。 为了克服这些局限性，我们提出了SliceLens这一假设驱动的框架，该框架利用大型语言模型（LLMs）和视觉语言模型（VLMs），通过基于场景的视觉推理生成并验证多样化的失败假设，从而可靠地识别出细粒度且具有解释性的误差切片。此外，我们引入了FeSD (Fine-grained Slice Discovery)，这是首个专门用于评估实例级计算机视觉任务中细粒度错误切片发现性能的基准测试平台，它包括专家注释和精心调整的真实错误区域。 在现有基准测试和FeSD上的广泛实验表明，SliceLens达到了最先进的性能，在FeSD上将Precision@10从0.31提高到了0.73，并且能够识别出解释性很强的误差切片，这些切片通过模型修复实验证明有助于推动有效的模型改进。
URL
https://arxiv.org/abs/2512.24592
