Abstract
Thanks to the recent achievements in task-driven image quality enhancement (IQE) models like ESTR, the image enhancement model and the visual recognition model can mutually enhance each other's quantitation while producing high-quality processed images that are perceivable by our human vision systems. However, existing task-driven IQE models tend to overlook an underlying fact -- different levels of vision tasks have varying and sometimes conflicting requirements of image features. To address this problem, this paper proposes a generalized gradient promotion (GradProm) training strategy for task-driven IQE of medical images. Specifically, we partition a task-driven IQE system into two sub-models, i.e., a mainstream model for image enhancement and an auxiliary model for visual recognition. During training, GradProm updates only parameters of the image enhancement model using gradients of the visual recognition model and the image enhancement model, but only when gradients of these two sub-models are aligned in the same direction, which is measured by their cosine similarity. In case gradients of these two sub-models are not in the same direction, GradProm only uses the gradient of the image enhancement model to update its parameters. Theoretically, we have proved that the optimization direction of the image enhancement model will not be biased by the auxiliary visual recognition model under the implementation of GradProm. Empirically, extensive experimental results on four public yet challenging medical image datasets demonstrated the superior performance of GradProm over existing state-of-the-art methods.
Abstract (translated)
感谢最近在任务驱动型图像质量增强（IQE）模型如ESTR方面取得的成就，图像增强模型和视觉识别模型可以在生成高质量且可被人类视觉系统感知的处理图片的同时相互提高各自的性能。然而，现有的任务驱动型IQE模型往往忽视了一个事实——不同级别的视觉任务对图像特征的需求各不相同，有时甚至是冲突的。为了解决这个问题，本文提出了一种针对医学图像的任务驱动型IQE的泛化梯度推广（GradProm）训练策略。 具体来说，我们将一个任务驱动型IQE系统划分为两个子模型：一个是用于图像增强的主要流模型，另一个是用于视觉识别的辅助模型。在训练过程中，GradProm仅使用视觉识别模型和图像增强模型的梯度来更新图像增强模型的参数，并且只有当这两个子模型的梯度方向一致时才会进行更新（一致性通过余弦相似性测量）。如果两个子模型的梯度方向不一致，则GradProm只利用图像增强模型的梯度来更新其参数。理论上，我们已经证明，在实现GradProm的情况下，图像增强模型的优化方向不会被辅助视觉识别模型偏移。 在实验方面，我们在四个公开且具有挑战性的医学图像数据集上进行了广泛的实验，并展示了GradProm相比现有最先进的方法具有优越的表现能力。
URL
https://arxiv.org/abs/2501.01114
