Abstract
Pavement condition assessment is essential for road safety and maintenance. Existing research has made significant progress. However, most studies focus on conventional computer vision tasks such as classification, detection, and segmentation. In real-world applications, pavement inspection requires more than visual recognition. It also requires quantitative analysis, explanation, and interactive decision support. Current datasets are limited. They focus on unimodal perception. They lack support for multi-turn interaction and fact-grounded reasoning. They also do not connect perception with vision-language analysis. To address these limitations, we introduce PaveBench, a large-scale benchmark for pavement distress perception and interactive vision-language analysis on real-world highway inspection images. PaveBench supports four core tasks: classification, object detection, semantic segmentation, and vision-language question answering. It provides unified task definitions and evaluation protocols. On the visual side, PaveBench provides large-scale annotations and includes a curated hard-distractor subset for robustness evaluation. It contains a large collection of real-world pavement images. On the multimodal side, we introduce PaveVQA, a real-image question answering (QA) dataset that supports single-turn, multi-turn, and expert-corrected interactions. It covers recognition, localization, quantitative estimation, and maintenance reasoning. We evaluate several state-of-the-art methods and provide a detailed analysis. We also present a simple and effective agent-augmented visual question answering framework that integrates domain-specific models as tools alongside vision-language models. The dataset is available at: this https URL.
Abstract (translated)
路面状况评估对于道路安全与养护至关重要。现有研究已取得显著进展，但多数工作聚焦于分类、检测与分割等传统计算机视觉任务。在实际应用中，路面检测不仅需要视觉识别，还需定量分析、解释说明与交互式决策支持。当前数据集存在局限：侧重单模态感知，缺乏多轮交互与事实验证推理支持，且未实现感知与视觉-语言分析的结合。为应对这些挑战，我们提出PaveBench——首个面向真实高速公路巡检图像的大规模路面病害感知与交互式视觉-语言分析基准。该基准支持分类、目标检测、语义分割与视觉-语言问答四大核心任务，并提供统一的任务定义与评估协议。在视觉层面，PaveBench提供大规模标注数据，并包含精心筛选的困难干扰子集以评估模型鲁棒性；在 multimodal 层面，我们引入PaveVQA——首个支持单轮、多轮及专家修正交互的真实图像问答数据集，涵盖识别、定位、定量估算与养护推理等维度。我们评估了多项先进方法并展开详细分析，同时提出一种简单有效的智能体增强型视觉问答框架，可将领域专用模型作为工具与视觉-语言模型协同使用。数据集已公开：this https URL。
URL
https://arxiv.org/abs/2604.02804
