Abstract
Ecological monitoring is increasingly automated by vision models, yet opaque predictions limit trust and field adoption. We present an inpainting-guided, perturbation-based explanation technique that produces photorealistic, mask-localized edits that preserve scene context. Unlike masking or blurring, these edits stay in-distribution and reveal which fine-grained morphological cues drive predictions in tasks such as species recognition and trait attribution. We demonstrate the approach on a YOLOv9 detector fine-tuned for harbor seal detection in Glacier Bay drone imagery, using Segment-Anything-Model-refined masks to support two interventions: (i) object removal/replacement (e.g., replacing seals with plausible ice/water or boats) and (ii) background replacement with original animals composited onto new scenes. Explanations are assessed by re-scoring perturbed images (flip rate, confidence drop) and by expert review for ecological plausibility and interpretability. The resulting explanations localize diagnostic structures, avoid deletion artifacts common to traditional perturbations, and yield domain-relevant insights that support expert validation and more trustworthy deployment of AI in ecology.
Abstract (translated)
生态监测越来越依赖于视觉模型的自动化，然而不透明的预测限制了信任和现场应用。我们提出了一种基于修复引导、扰动分析的解释技术，该技术可以生成保留场景上下文的真实感光编辑，并且这些编辑集中在特定的遮罩区域。与简单的蒙版或模糊处理不同，这种编辑保持在数据分布之内，并揭示了驱动预测的具体细粒度形态特征，在物种识别和属性归因等任务中尤为关键。 我们在冰川湾无人机图像上对YOLOv9检测器进行微调，用于海港 seal 的检测，并利用Segment-Anything-Model优化的掩码支持两种干预措施：（i）对象移除/替换（例如，将seal替换成合理的冰、水或船只），以及（ii）背景替换，即在新的场景中以原始动物为素材进行合成。我们通过重新评估扰动后的图像得分（翻转率和置信度下降）和专家评审的生态合理性及可解释性来评估这种方法的效果。 结果表明，这些解释能够定位诊断结构，避免传统扰动常见的删除伪影，并提供与领域相关的见解以支持专家验证和更值得信赖的人工智能在生态学中的应用部署。
URL
https://arxiv.org/abs/2510.03317
