Abstract
Since human and environmental factors interfere, captured polyp images usually suffer from issues such as dim lighting, blur, and overexposure, which pose challenges for downstream polyp segmentation tasks. To address the challenges of noise-induced degradation in polyp images, we present AgentPolyp, a novel framework integrating CLIP-based semantic guidance and dynamic image enhancement with a lightweight neural network for segmentation. The agent first evaluates image quality using CLIP-driven semantic analysis (e.g., identifying ``low-contrast polyps with vascular textures") and adapts reinforcement learning strategies to dynamically apply multi-modal enhancement operations (e.g., denoising, contrast adjustment). A quality assessment feedback loop optimizes pixel-level enhancement and segmentation focus in a collaborative manner, ensuring robust preprocessing before neural network segmentation. This modular architecture supports plug-and-play extensions for various enhancement algorithms and segmentation networks, meeting deployment requirements for endoscopic devices.
Abstract (translated)
由于人类和环境因素的干扰，捕获到的息肉图像通常会遇到诸如光线昏暗、模糊和过度曝光等问题，这些问题给下游的息肉分割任务带来了挑战。为了应对由噪声引起的息肉图像退化问题，我们提出了AgentPolyp，这是一种新型框架，它结合了基于CLIP（Contrastive Language-Image Pre-training）的语义指导以及使用轻量级神经网络进行分割的动态图像增强技术。该框架首先利用CLIP驱动的语义分析评估图像质量（例如，识别“低对比度且具有血管纹理的息肉”），并根据强化学习策略自适应地应用多模态增强操作（如去噪和调整对比度）。一个质量评估反馈循环以协作的方式优化像素级别的增强处理以及分割聚焦点，在神经网络分割前确保进行稳健的预处理。这种模块化架构支持各种增强算法和分割网络的即插即用扩展，满足内窥镜设备部署的需求。
URL
https://arxiv.org/abs/2504.10978
