Abstract
Unsupervised anomaly detection (UAD) presents a complementary alternative to supervised learning for brain tumor segmentation in magnetic resonance imaging (MRI), particularly when annotated datasets are limited, costly, or inconsistent. In this work, we propose a novel Multimodal Vision Transformer Autoencoder (MViT-AE) trained exclusively on healthy brain MRIs to detect and localize tumors via reconstruction-based error maps. This unsupervised paradigm enables segmentation without reliance on manual labels, addressing a key scalability bottleneck in neuroimaging workflows. Our method is evaluated in the BraTS-GoAT 2025 Lighthouse dataset, which includes various types of tumors such as gliomas, meningiomas, and pediatric brain tumors. To enhance performance, we introduce a multimodal early-late fusion strategy that leverages complementary information across multiple MRI sequences, and a post-processing pipeline that integrates the Segment Anything Model (SAM) to refine predicted tumor contours. Despite the known challenges of UAD, particularly in detecting small or non-enhancing lesions, our method achieves clinically meaningful tumor localization, with lesion-wise Dice Similarity Coefficient of 0.437 (Whole Tumor), 0.316 (Tumor Core), and 0.350 (Enhancing Tumor) on the test set, and an anomaly Detection Rate of 89.4% on the validation set. These findings highlight the potential of transformer-based unsupervised models to serve as scalable, label-efficient tools for neuro-oncological imaging.
Abstract (translated)
无监督异常检测（UAD）为磁共振成像（MRI）中的脑肿瘤分割提供了一种补充的替代方案，尤其是在标注数据集有限、成本高昂或不一致的情况下。在这项工作中，我们提出了一种新颖的多模态视觉变压器自编码器（MViT-AE），该模型仅在健康的大脑MRI上进行训练，通过基于重建的误差图来检测和定位肿瘤。这种无监督方法使分割可以无需依赖人工标注标签，解决了神经影像工作流程中的关键可扩展性瓶颈问题。我们的方法在BraTS-GoAT 2025灯塔数据集中进行了评估，该数据集包括多种类型的肿瘤，如胶质瘤、脑膜瘤和儿童脑肿瘤。 为了提高性能，我们引入了一种多模态早期-晚期融合策略，利用了多个MRI序列之间的互补信息，并且采用了一个后处理管道，将Segment Anything Model（SAM）整合进来以精炼预测的肿瘤轮廓。尽管UAD在检测小病变或非增强性病灶方面存在已知挑战，我们的方法仍然实现了具有临床意义的肿瘤定位，在测试集上全瘤体、肿瘤核心和增强型肿瘤的Dice相似系数分别为0.437、0.316和0.350，并且在验证集上的异常检测率为89.4%。这些发现突显了基于变压器的无监督模型作为神经肿瘤学成像中的高效可扩展工具的巨大潜力。
URL
https://arxiv.org/abs/2510.15684
