Abstract
Automated structural defect annotation is essential for ensuring infrastructure safety while minimizing the high costs and inefficiencies of manual labeling. A novel agentic annotation framework, Agent-based Defect Pattern Tagger (ADPT), is introduced that integrates Large Vision-Language Models (LVLMs) with a semantic pattern matching module and an iterative self-questioning refinement mechanism. By leveraging optimized domain-specific prompting and a recursive verification process, ADPT transforms raw visual data into high-quality, semantically labeled defect datasets without any manual supervision. Experimental results demonstrate that ADPT achieves up to 98% accuracy in distinguishing defective from non-defective images, and 85%-98% annotation accuracy across four defect categories under class-balanced settings, with 80%-92% accuracy on class-imbalanced datasets. The framework offers a scalable and cost-effective solution for high-fidelity dataset construction, providing strong support for downstream tasks such as transfer learning and domain adaptation in structural damage assessment.
Abstract (translated)
自动化结构缺陷标注对于确保基础设施安全同时减少人工标注的高昂成本和低效性至关重要。本文介绍了一种新颖的代理注释框架——基于智能体的缺陷模式标签器（ADPT），该框架结合了大型视觉语言模型（LVLM）与语义模式匹配模块以及迭代自我质疑精炼机制。通过利用优化后的领域特定提示和递归验证过程，ADPT能够将原始视觉数据转化为高质量、具有语义标注的缺陷数据集，并且完全无需人工监督。 实验结果显示，在区分有缺陷图像和无缺陷图像方面，ADPT可达到高达98%的准确率；在平衡类别的四种类别中，其注释准确率可达85%-98%；而在不平衡类别数据集中，该模型的准确率也达到了80%-92%。此框架提供了一种可扩展且成本效益高的解决方案，用于构建高保真度的数据集，并为下游任务（如迁移学习和领域适应）中的结构损伤评估提供了强有力的支持。
URL
https://arxiv.org/abs/2510.00603
