Abstract
Weakly supervised semantic segmentation (WSSS) in medical imaging struggles with effectively using sparse annotations. One promising direction for WSSS leverages gaze annotations, captured via eye trackers that record regions of interest during diagnostic procedures. However, existing gaze-based methods, such as GazeMedSeg, do not fully exploit the rich information embedded in gaze data. In this paper, we propose GradTrack, a framework that utilizes physicians' gaze track, including fixation points, durations, and temporal order, to enhance WSSS performance. GradTrack comprises two key components: Gaze Track Map Generation and Track Attention, which collaboratively enable progressive feature refinement through multi-level gaze supervision during the decoding process. Experiments on the Kvasir-SEG and NCI-ISBI datasets demonstrate that GradTrack consistently outperforms existing gaze-based methods, achieving Dice score improvements of 3.21\% and 2.61\%, respectively. Moreover, GradTrack significantly narrows the performance gap with fully supervised models such as nnUNet.
Abstract (translated)
在医学成像中的弱监督语义分割（WSSS）面临有效利用稀疏标注的挑战。一个有前景的研究方向是利用通过眼动追踪器记录下的诊断过程中感兴趣区域的眼动数据来增强WSSS效果。然而，现有的基于眼动的方法（如GazeMedSeg）未能充分挖掘眼动数据中的丰富信息。在本文中，我们提出了GradTrack框架，该框架利用了医生的注视轨迹，包括固定点、持续时间和时间顺序等信息，以提升弱监督语义分割的表现。 GradTrack包含两个关键组件：注视轨迹图生成和轨迹注意力机制，这两个部分合作通过多级眼动监督来逐步改进特征，在解码过程中实现渐进式的特征精炼。在Kvasir-SEG和NCI-ISBI数据集上的实验表明，GradTrack相比现有的基于眼动的方法表现出显著的优势，分别取得了3.21% 和 2.61% 的Dice分数提升。此外，GradTrack大幅度缩小了与全监督模型（如nnUNet）的性能差距。
URL
https://arxiv.org/abs/2505.22230
