Abstract
Early and accurate interpretation of screening mammograms is essential for effective breast cancer detection, yet it remains a complex challenge due to subtle imaging findings and diagnostic ambiguity. Many existing AI approaches fall short by focusing on single view inputs or single-task outputs, limiting their clinical utility. To address these limitations, we propose a novel multi-view, multitask hybrid deep learning framework that processes all four standard mammography views and jointly predicts diagnostic labels and BI-RADS scores for each breast. Our architecture integrates a hybrid CNN VSSM backbone, combining convolutional encoders for rich local feature extraction with Visual State Space Models (VSSMs) to capture global contextual dependencies. To improve robustness and interpretability, we incorporate a gated attention-based fusion module that dynamically weights information across views, effectively handling cases with missing data. We conduct extensive experiments across diagnostic tasks of varying complexity, benchmarking our proposed hybrid models against baseline CNN architectures and VSSM models in both single task and multi task learning settings. Across all tasks, the hybrid models consistently outperform the baselines. In the binary BI-RADS 1 vs. 5 classification task, the shared hybrid model achieves an AUC of 0.9967 and an F1 score of 0.9830. For the more challenging ternary classification, it attains an F1 score of 0.7790, while in the five-class BI-RADS task, the best F1 score reaches 0.4904. These results highlight the effectiveness of the proposed hybrid framework and underscore both the potential and limitations of multitask learning for improving diagnostic performance and enabling clinically meaningful mammography analysis.
Abstract (translated)
早期且准确地解读乳腺筛查摄影对于有效检测乳腺癌至关重要，但由于影像学发现细微及诊断模糊等原因，这一过程仍然是一项复杂挑战。许多现有的人工智能方法由于专注于单一视图输入或单任务输出而效果不佳，从而限制了其临床应用价值。为解决这些局限性，我们提出了一种新颖的多视图、多任务混合深度学习框架，该框架处理所有四种标准乳腺摄影视图，并且能够共同预测每个乳房的诊断标签和BI-RADS评分。 我们的架构结合了一个混合CNN VSSM骨干网，它融合了卷积编码器以提取丰富的局部特征以及视觉状态空间模型（VSSMs）来捕捉全局上下文依赖关系。为了提高鲁棒性和可解释性，我们引入了一种基于门控注意力的融合模块，该模块可以动态加权各视图之间的信息，并有效地处理缺失数据的情况。 我们在不同复杂度的诊断任务上进行了广泛的实验，在单一任务和多任务学习设置下将我们的混合模型与基准CNN架构及VSSM模型进行对比。在所有任务中，混合模型始终优于基线模型。 具体来说，在二元BI-RADS 1 vs. 5分类任务中，共享混合模型达到了AUC为0.9967和F1得分为0.9830的性能。对于更具挑战性的三类分类问题，它获得了F1得分为0.7790的成绩，在五级BI-RADS任务中，最佳F1得分达到0.4904。 这些结果强调了所提出的混合框架的有效性，并突显了多任务学习在改善诊断性能和实现临床意义重大的乳腺摄影分析方面的潜力与局限。
URL
https://arxiv.org/abs/2507.16955
