Transfer_Learning
-
Intuitions of Machine Learning Researchers about Transfer Learning for Medical Image Classification
2025-10-01 13:44:46Yucheng Lu, Hubert Dariusz Zaj\k{a}c, Veronika Cheplygina, Amelia Jim\'enez-S\'anchezAbstract
Transfer learning is crucial for medical imaging, yet the selection of source datasets - which can impact the generalizability of algorithms, and thus patient outcomes - often relies on researchers' intuition rather than systematic principles. This study investigates these decisions through a task-based survey with machine learning practitioners. Unlike prior work that benchmarks models and experimental setups, we take a human-centered HCI perspective on how practitioners select source datasets. Our findings indicate that choices are task-dependent and influenced by community practices, dataset properties, and computational (data embedding), or perceived visual or semantic similarity. However, similarity ratings and expected performance are not always aligned, challenging a traditional "more similar is better" view. Participants often used ambiguous terminology, which suggests a need for clearer definitions and HCI tools to make them explicit and usable. By clarifying these heuristics, this work provides practical insights for more systematic source selection in transfer learning.
Abstract (translated)
迁移学习在医学影像领域至关重要，但选择源数据集——这会影响算法的泛化能力，进而影响患者结果——往往依赖于研究人员的直觉而非系统的准则。本研究通过面向机器学习从业者的任务调研来探讨这些决策。与以往侧重模型基准和实验设置的研究不同，我们从以人为本的人机交互（HCI）视角出发，考察从业者选择源数据集的方式。我们的发现表明，选择取决于具体任务，并受到社区实践、数据集属性以及计算嵌入或感知的视觉或语义相似性的影响。然而，相似性的评分与预期性能并不总是保持一致，这挑战了传统的“越相似越好”的观点。 参与者经常使用含糊不清的技术术语，这一点提示我们需要更清晰的定义和人机交互工具，以使这些概念明确且易于理解。通过阐明这些启发式方法，本研究为迁移学习中更具系统性的源数据集选择提供了实际见解。
URL
https://arxiv.org/abs/2510.00902
PDF
-
LVLMs as inspectors: an agentic framework for category-level structural defect annotation
2025-10-01 07:31:42Sheng Jiang, Yuanmin Ning, Bingxi Huang, Peiyin Chen, Zhaohui ChenAbstract
Automated structural defect annotation is essential for ensuring infrastructure safety while minimizing the high costs and inefficiencies of manual labeling. A novel agentic annotation framework, Agent-based Defect Pattern Tagger (ADPT), is introduced that integrates Large Vision-Language Models (LVLMs) with a semantic pattern matching module and an iterative self-questioning refinement mechanism. By leveraging optimized domain-specific prompting and a recursive verification process, ADPT transforms raw visual data into high-quality, semantically labeled defect datasets without any manual supervision. Experimental results demonstrate that ADPT achieves up to 98% accuracy in distinguishing defective from non-defective images, and 85%-98% annotation accuracy across four defect categories under class-balanced settings, with 80%-92% accuracy on class-imbalanced datasets. The framework offers a scalable and cost-effective solution for high-fidelity dataset construction, providing strong support for downstream tasks such as transfer learning and domain adaptation in structural damage assessment.
Abstract (translated)
自动化结构缺陷标注对于确保基础设施安全同时减少人工标注的高昂成本和低效性至关重要。本文介绍了一种新颖的代理注释框架——基于智能体的缺陷模式标签器（ADPT），该框架结合了大型视觉语言模型（LVLM）与语义模式匹配模块以及迭代自我质疑精炼机制。通过利用优化后的领域特定提示和递归验证过程，ADPT能够将原始视觉数据转化为高质量、具有语义标注的缺陷数据集，并且完全无需人工监督。 实验结果显示，在区分有缺陷图像和无缺陷图像方面，ADPT可达到高达98%的准确率；在平衡类别的四种类别中，其注释准确率可达85%-98%；而在不平衡类别数据集中，该模型的准确率也达到了80%-92%。此框架提供了一种可扩展且成本效益高的解决方案，用于构建高保真度的数据集，并为下游任务（如迁移学习和领域适应）中的结构损伤评估提供了强有力的支持。
URL
https://arxiv.org/abs/2510.00603
PDF
-
Transformer Classification of Breast Lesions: The BreastDCEDL_AMBL Benchmark Dataset and 0.92 AUC Baseline
2025-09-30 15:58:02Naomi Fridman (Ariel University), Anat Goldstein (Ariel University)Abstract
The error is caused by special characters that arXiv's system doesn't recognize. Here's the cleaned version with all problematic characters replaced: Breast magnetic resonance imaging is a critical tool for cancer detection and treatment planning, but its clinical utility is hindered by poor specificity, leading to high false-positive rates and unnecessary biopsies. This study introduces a transformer-based framework for automated classification of breast lesions in dynamic contrast-enhanced MRI, addressing the challenge of distinguishing benign from malignant findings. We implemented a SegFormer architecture that achieved an AUC of 0.92 for lesion-level classification, with 100% sensitivity and 67% specificity at the patient level - potentially eliminating one-third of unnecessary biopsies without missing malignancies. The model quantifies malignant pixel distribution via semantic segmentation, producing interpretable spatial predictions that support clinical decision-making. To establish reproducible benchmarks, we curated BreastDCEDL_AMBL by transforming The Cancer Imaging Archive's AMBL collection into a standardized deep learning dataset with 88 patients and 133 annotated lesions (89 benign, 44 malignant). This resource addresses a key infrastructure gap, as existing public datasets lack benign lesion annotations, limiting benign-malignant classification research. Training incorporated an expanded cohort of over 1,200 patients through integration with BreastDCEDL datasets, validating transfer learning approaches despite primary tumor-only annotations. Public release of the dataset, models, and evaluation protocols provides the first standardized benchmark for DCE-MRI lesion classification, enabling methodological advancement toward clinical deployment.
Abstract (translated)
错误是由arXiv系统无法识别的特殊字符引起的。以下是替换所有问题字符后的清理版本： 乳房磁共振成像是癌症检测和治疗计划中的关键工具，但其临床效用受到较低特异性的限制，导致假阳性率高并引发不必要的活检。本研究介绍了一种基于变压器框架的方法，用于动态对比增强MRI中乳腺病变的自动分类，解决了区分良性与恶性发现这一挑战。我们实施了SegFormer架构，在病灶级别的分类上达到了0.92的AUC值，并且在患者级别上实现了100%的灵敏度和67%的特异性——有可能减少三分之一不必要的活检而不遗漏恶性肿瘤。该模型通过语义分割量化恶性的像素分布，生成支持临床决策制定的可解释的空间预测结果。 为了建立可重复的标准基准，我们整理了BreastDCEDL_AMBL数据集，将癌症影像档案馆的AMBL集合转换为包含88名患者和133个标记病灶（其中89个良性，44个恶性）的标准深度学习数据集。这一资源弥补了一个关键基础设施缺口，因为现有的公共数据集中缺少良性病变标注信息，限制了良恶性分类的研究。 训练阶段整合了超过1,200名患者的扩增队列，并通过与BreastDCEDL数据集的集成验证了迁移学习方法的有效性，尽管主要肿瘤注释仅限于单一肿瘤。公开发布该数据集、模型以及评估协议为DCE-MRI病灶分类提供了首个标准基准，促进了向临床应用的方法改进。
URL
https://arxiv.org/abs/2509.26440
PDF
-
LTA-L2S: Lexical Tone-Aware Lip-to-Speech Synthesis for Mandarin with Cross-Lingual Transfer Learning
2025-09-30 02:13:55Kang Yang, Yifan Liang, Fangkun Liu, Zhenping Xie, Chengshi ZhengAbstract
Lip-to-speech (L2S) synthesis for Mandarin is a significant challenge, hindered by complex viseme-to-phoneme mappings and the critical role of lexical tones in intelligibility. To address this issue, we propose Lexical Tone-Aware Lip-to-Speech (LTA-L2S). To tackle viseme-to-phoneme complexity, our model adapts an English pre-trained audio-visual self-supervised learning (SSL) model via a cross-lingual transfer learning strategy. This strategy not only transfers universal knowledge learned from extensive English data to the Mandarin domain but also circumvents the prohibitive cost of training such a model from scratch. To specifically model lexical tones and enhance intelligibility, we further employ a flow-matching model to generate the F0 contour. This generation process is guided by ASR-fine-tuned SSL speech units, which contain crucial suprasegmental information. The overall speech quality is then elevated through a two-stage training paradigm, where a flow-matching postnet refines the coarse spectrogram from the first stage. Extensive experiments demonstrate that LTA-L2S significantly outperforms existing methods in both speech intelligibility and tonal accuracy.
Abstract (translated)
唇部到语音（Lip-to-Speech，简称L2S）合成对于普通话来说是一个重大的挑战，这主要是由于复杂的唇形单元至音素映射以及声调在可懂度中的关键作用。为了解决这一问题，我们提出了一个考虑到词汇音调的唇部到语音合成方法（Lexical Tone-Aware Lip-to-Speech, LTA-L2S）。为了应对唇形单元至音素复杂性的挑战，我们的模型通过跨语言迁移学习策略来调整一个预先训练好的英语音频-视觉自监督学习(SSL)模型。这一策略不仅将从大量英语数据中习得的通用知识转移到普通话领域，而且还避免了从头开始训练此类模型所需的高昂成本。 为了具体建模词汇音调并增强可懂度，我们进一步使用了一个流匹配（flow-matching）模型来生成基频（F0）轮廓。这一生成过程受经过语音识别细化后的SSL语音单元指导，这些单位包含至关重要的超分段信息。通过一个两阶段的训练范式，整体语音质量得到了提升，在此过程中，流匹配后处理网络在第一阶段生成粗略光谱图之后对其进行精细调整。 广泛的实验表明，LTA-L2S方法在语音可懂度和声调准确性方面显著优于现有技术。
URL
https://arxiv.org/abs/2509.25670
PDF
-
MetaChest: Generalized few-shot learning of patologies from chest X-rays
2025-09-29 23:27:44Berenice Montalvo-Lezama, Gibran Fuentes-PinedaAbstract
The limited availability of annotated data presents a major challenge for applying deep learning methods to medical image analysis. Few-shot learning methods aim to recognize new classes from only a small number of labeled examples. These methods are typically studied under the standard few-shot learning setting, where all classes in a task are new. However, medical applications such as pathology classification from chest X-rays often require learning new classes while simultaneously leveraging knowledge of previously known ones, a scenario more closely aligned with generalized few-shot classification. Despite its practical relevance, few-shot learning has been scarcely studied in this context. In this work, we present MetaChest, a large-scale dataset of 479,215 chest X-rays collected from four public databases. MetaChest includes a meta-set partition specifically designed for standard few-shot classification, as well as an algorithm for generating multi-label episodes. We conduct extensive experiments evaluating both a standard transfer learning approach and an extension of ProtoNet across a wide range of few-shot multi-label classification tasks. Our results demonstrate that increasing the number of classes per episode and the number of training examples per class improves classification performance. Notably, the transfer learning approach consistently outperforms the ProtoNet extension, despite not being tailored for few-shot learning. We also show that higher-resolution images improve accuracy at the cost of additional computation, while efficient model architectures achieve comparable performance to larger models with significantly reduced resource requirements.
Abstract (translated)
标注数据的有限可用性为将深度学习方法应用于医学图像分析带来了重大挑战。少数样本学习（few-shot learning）方法旨在仅从少量有标签的例子中识别新的类别。这些方法通常在标准少数样本学习设置下进行研究，其中任务中的所有类都是全新的。然而，在诸如胸部X光片病理分类的医疗应用中，往往需要同时学习新类和利用已知类的知识，这种情况更符合泛化少数样本分类（generalized few-shot classification）的情景。尽管其实用性很强，但在这个背景下很少有对少数样本学习的研究。 在本工作中，我们介绍了MetaChest，这是一个大规模的数据集，包含来自四个公开数据库的479,215张胸部X光片图像。MetaChest包括一个专门为标准少数样本分类设计的元数据集分割，并且包含用于生成多标签序列的算法。我们进行了广泛的实验，评估了一种传统的迁移学习方法和ProtoNet（一种原型网络）的扩展在一系列广泛的少数样本多标签分类任务中的表现。我们的结果表明，在每个序列中增加类的数量以及每种类别训练示例的数量可以提高分类性能。 值得注意的是，尽管不是为少数样本学习量身定制，但转移学习的方法始终优于ProtoNet的扩展方法。我们还展示了高分辨率图像在牺牲更多计算资源的情况下提高了准确性，而高效的模型架构则实现了与更大模型相当的表现力，同时显著减少了资源需求。
URL
https://arxiv.org/abs/2509.25590
PDF
-
Learning Distinguishable Representations in Deep Q-Networks for Linear Transfer
2025-09-29 15:44:35Sooraj Sathish, Keshav Goyal, Raghuram Bharadwaj DiddigiAbstract
Deep Reinforcement Learning (RL) has demonstrated success in solving complex sequential decision-making problems by integrating neural networks with the RL framework. However, training deep RL models poses several challenges, such as the need for extensive hyperparameter tuning and high computational costs. Transfer learning has emerged as a promising strategy to address these challenges by enabling the reuse of knowledge from previously learned tasks for new, related tasks. This avoids the need for retraining models entirely from scratch. A commonly used approach for transfer learning in RL is to leverage the internal representations learned by the neural network during training. Specifically, the activations from the last hidden layer can be viewed as refined state representations that encapsulate the essential features of the input. In this work, we investigate whether these representations can be used as input for training simpler models, such as linear function approximators, on new tasks. We observe that the representations learned by standard deep RL models can be highly correlated, which limits their effectiveness when used with linear function approximation. To mitigate this problem, we propose a novel deep Q-learning approach that introduces a regularization term to reduce positive correlations between feature representation of states. By leveraging these reduced correlated features, we enable more effective use of linear function approximation in transfer learning. Through experiments and ablation studies on standard RL benchmarks and MinAtar games, we demonstrate the efficacy of our approach in improving transfer learning performance and thereby reducing computational overhead.
Abstract (translated)
深度强化学习（RL）通过将神经网络与RL框架相结合，成功解决了复杂的序列决策问题。然而，训练深度RL模型存在一些挑战，例如需要进行广泛的超参数调整和高昂的计算成本。迁移学习作为一种有前景的方法已经出现，它能够使模型在执行新任务时复用之前任务中学到的知识，从而避免完全从头开始重新训练模型的需求。 在RL中的一个常用的迁移学习方法是利用神经网络在训练过程中学到的内部表示。具体来说，最后一层隐藏层的激活可以被视为细化的状态表示，这些状态表示封装了输入的关键特征。在这项工作中，我们探讨了是否可以通过使用这些表示作为新任务中较简单模型（如线性函数近似器）的输入来进行学习。 我们的研究发现，标准深度RL模型学到的表示可能具有高度的相关性，这在与线性函数近似的结合时会限制其有效性。为了解决这个问题，我们提出了一种新的深度Q学习方法，该方法引入了一个正则化项来减少状态特征表示之间的正相关关系。通过利用这些较低相关的特征，我们能够更有效地使用线性函数近似来进行迁移学习。 我们在标准RL基准和MinAtar游戏中进行了实验和消融研究，结果表明我们的方法在提高迁移学习性能方面是有效的，并且因此降低了计算开销。
URL
https://arxiv.org/abs/2509.24947
PDF
-
LogAction: Consistent Cross-system Anomaly Detection through Logs via Active Domain
2025-09-29 07:09:19Chiming Duan, Minghua He, Pei Xiao, Tong Jia, Xin Zhang, Zhewei Zhong, Xiang Luo, Yan Niu, Lingzhe Zhang, Yifan Wu, Siyu Yu, Weijie Hong, Ying Li, Gang HuangAbstract
Log-based anomaly detection is a essential task for ensuring the reliability and performance of software systems. However, the performance of existing anomaly detection methods heavily relies on labeling, while labeling a large volume of logs is highly challenging. To address this issue, many approaches based on transfer learning and active learning have been proposed. Nevertheless, their effectiveness is hindered by issues such as the gap between source and target system data distributions and cold-start problems. In this paper, we propose LogAction, a novel log-based anomaly detection model based on active domain adaptation. LogAction integrates transfer learning and active learning techniques. On one hand, it uses labeled data from a mature system to train a base model, mitigating the cold-start issue in active learning. On the other hand, LogAction utilize free energy-based sampling and uncertainty-based sampling to select logs located at the distribution boundaries for manual labeling, thus addresses the data distribution gap in transfer learning with minimal human labeling efforts. Experimental results on six different combinations of datasets demonstrate that LogAction achieves an average 93.01% F1 score with only 2% of manual labels, outperforming some state-of-the-art methods by 26.28%. Website: this https URL
Abstract (translated)
基于日志的异常检测是确保软件系统可靠性和性能的一项重要任务。然而，现有的异常检测方法的效果很大程度上依赖于标注，而对大量日志进行标注是一个极具挑战性的任务。为了解决这一问题，许多基于迁移学习和主动学习的方法被提出。不过，这些方法的有效性受到源系统与目标系统数据分布差异以及冷启动问题等因素的限制。 在本文中，我们提出了LogAction，这是一种新型的日志异常检测模型，采用了主动领域自适应技术。LogAction结合了迁移学习和主动学习的技术：一方面，它利用成熟系统的标注数据训练基础模型，减轻了主动学习中的冷启动问题；另一方面，LogAction通过基于自由能的采样和不确定性导向的采样方法选择边界分布上的日志进行人工标注，从而在最小的人工干预下解决了迁移学习中的数据分布差异问题。 实验结果表明，在六种不同的数据集组合上，LogAction仅使用2%的手动标签就能达到平均93.01%的F1分数，显著优于一些最新的方法（性能提升高达26.28%）。 官方网站：[此链接](https://this-url.com) （请将“this https URL”替换为实际网站地址）。
URL
https://arxiv.org/abs/2510.03288
PDF
-
An Efficient Transfer Learning Method Based on Adapter with Local Attributes for Speech Emotion Recognition
2025-09-28 10:36:59Haoyu Song, Ian McLoughlin, Qing Gu, Nan Jiang, Yan SongAbstract
Existing speech emotion recognition (SER) methods commonly suffer from the lack of high-quality large-scale corpus, partly due to the complex, psychological nature of emotion which makes accurate labeling difficult and time consuming. Recently, transfer learning based methods that exploit the encoders pretrained on large-scale speech corpus (e.g., Wav2Vec2.0 and HuBERT) have shown strong potential for downstream SER tasks. However, task-specific fine-tuning remains necessary for various conversational scenarios of different topics, speakers and languages to achieve satisfactory performance. It generally requires costly encoder retraining for individual SER tasks. To address this issue, we propose to train an adapter with local attributes for efficient transfer learning. Specifically, a weighted average pooling-Transformer (WAP-Transformer) is proposed as a lightweight backbone to enrich the frame-level representation. An adapter with teacher-student branches is exploited for task-agnostic transfer learning, where the student branch is jointly optimized via mask prediction and self-distillation objectives, and the teacher branch is obtained online from the student via exponential moving average (EMA). Meanwhile, local attributes are learned from the teacher branch via unsupervised clustering, which aims to act as a universal model that provides additional semantic-rich supervisions. A statistical attentive pooling (SAP) module is proposed to obtain utterance representation for fine-tuning. To evaluate the effectiveness of the proposed adapter with local attributes, extensive experiments have been conducted on IEMOCAP. Superior performance has been reported, compared to the previous state-of-the-art methods in similar settings.
Abstract (translated)
现有的语音情感识别（SER）方法通常因为高质量的大规模语料库的缺乏而表现不佳，这在一定程度上是因为情感具有复杂和心理性质的特点，使得准确标记情感变得困难且耗时。最近，基于迁移学习的方法通过利用大规模语音语料库（如Wav2Vec2.0和HuBERT）预先训练的编码器，在下游SER任务中表现出强大的潜力。然而，为了在不同主题、说话者和语言的各种对话场景中实现满意的性能，针对特定任务的微调仍然是必要的，这通常需要对每个SER任务进行昂贵的编码器重新训练。 为了解决这个问题，我们提出了通过具有局部属性的适配器来进行高效的迁移学习的方法。具体来说，我们提出了一种轻量级的骨干网络——加权平均池化-变压器（WAP-Transformer），以丰富帧级别的表示。一个带有教师-学生分支的适配器被用于无任务特定性的迁移学习中，其中学生分支通过掩码预测和自我蒸馏目标联合优化，而教师分支则在线从学生处通过指数移动平均（EMA）获得。同时，局部属性通过无监督聚类从教师分支中学得，并旨在作为一个通用模型提供额外的语义丰富的监督信息。一个统计注意力池化（SAP）模块被提出用于获取进行微调的语音表示。 为了评估具有局部属性的适配器的有效性，在IEMOCAP数据集上进行了广泛的实验，结果表明该方法在类似设置下比之前的最先进的SER方法表现出色。
URL
https://arxiv.org/abs/2509.23795
PDF
-
Deep Learning Approaches with Explainable AI for Differentiating Alzheimer Disease and Mild Cognitive Impairment
2025-09-27 16:17:14Fahad Mostafa, Kannon Hossain, Hafiz KhanAbstract
Early and accurate diagnosis of Alzheimer Disease is critical for effective clinical intervention, particularly in distinguishing it from Mild Cognitive Impairment, a prodromal stage marked by subtle structural changes. In this study, we propose a hybrid deep learning ensemble framework for Alzheimer Disease classification using structural magnetic resonance imaging. Gray and white matter slices are used as inputs to three pretrained convolutional neural networks such as ResNet50, NASNet, and MobileNet, each fine tuned through an end to end process. To further enhance performance, we incorporate a stacked ensemble learning strategy with a meta learner and weighted averaging to optimally combine the base models. Evaluated on the Alzheimer Disease Neuroimaging Initiative dataset, the proposed method achieves state of the art accuracy of 99.21% for Alzheimer Disease vs. Mild Cognitive Impairment and 91.0% for Mild Cognitive Impairment vs. Normal Controls, outperforming conventional transfer learning and baseline ensemble methods. To improve interpretability in image based diagnostics, we integrate Explainable AI techniques by Gradient weighted Class Activation, which generates heatmaps and attribution maps that highlight critical regions in gray and white matter slices, revealing structural biomarkers that influence model decisions. These results highlight the frameworks potential for robust and scalable clinical decision support in neurodegenerative disease diagnostics.
Abstract (translated)
阿尔茨海默病的早期和准确诊断对于有效的临床干预至关重要，特别是在将其与轻度认知障碍（MCI）区分开来时。MCI是一个前驱阶段，以细微的结构变化为标志。在本研究中，我们提出了一种使用结构磁共振成像进行阿尔茨海默病分类的混合深度学习集成框架。灰质和白质切片被用作三个预训练卷积神经网络（如ResNet50、NASNet和MobileNet）的输入，每个模型通过端到端的过程进行了微调以适应特定任务。 为了进一步提高性能，我们采用了一种堆叠式集成学习策略，并结合了一个元学习器以及加权平均法来优化地组合基础模型。在阿尔茨海默病神经成像倡议（ADNI）数据集上评估后，所提出的方法实现了99.21%的最高准确率用于区分阿尔茨海默病与轻度认知障碍，以及91.0%的准确性用于区分轻度认知障碍与正常对照组，超越了传统的迁移学习和基准集成方法。 为了提高基于图像诊断的解释性，我们通过梯度加权类激活（Grad-CAM）技术整合了解释人工智能的方法。这种方法生成热图和归因图，突出显示灰质和白质切片中的关键区域，并揭示影响模型决策的结构生物标志物。 这些结果强调了该框架在神经退行性疾病诊断中提供强大且可扩展临床决策支持方面的潜力。
URL
https://arxiv.org/abs/2510.00048
PDF
-
Brain Tumor Classification from MRI Scans via Transfer Learning and Enhanced Feature Representation
2025-09-26 21:41:30Ahta-Shamul Hoque Emran, Hafija Akter, Abdullah Al Shiam, Abu Saleh Musa Miah, Anichur Rahman, Fahmid Al Farid, Hezerul Abdul KarimAbstract
Brain tumors are abnormal cell growths in the central nervous system (CNS), and their timely detection is critical for improving patient outcomes. This paper proposes an automatic and efficient deep-learning framework for brain tumor detection from magnetic resonance imaging (MRI) scans. The framework employs a pre-trained ResNet50 model for feature extraction, followed by Global Average Pooling (GAP) and linear projection to obtain compact, high-level image representations. These features are then processed by a novel Dense-Dropout sequence, a core contribution of this work, which enhances non-linear feature learning, reduces overfitting, and improves robustness through diverse feature transformations. Another major contribution is the creation of the Mymensingh Medical College Brain Tumor (MMCBT) dataset, designed to address the lack of reliable brain tumor MRI resources. The dataset comprises MRI scans from 209 subjects (ages 9 to 65), including 3671 tumor and 13273 non-tumor images, all clinically verified under expert supervision. To overcome class imbalance, the tumor class was augmented, resulting in a balanced dataset well-suited for deep learning research.
Abstract (translated)
脑肿瘤是中枢神经系统（CNS）中的异常细胞生长，其及时检测对于改善患者预后至关重要。本文提出了一种自动且高效的深度学习框架，用于从磁共振成像（MRI）扫描中检测脑肿瘤。该框架采用预训练的ResNet50模型进行特征提取，并通过全局平均池化（GAP）和线性投影来获得紧凑、高层次的图像表示。这些特征随后由一种新颖的Dense-Dropout序列处理，这是本文的核心贡献之一，它增强了非线性特征学习能力，减少了过拟合，并通过多样化的特征转换提高了模型的鲁棒性。 另一个重大贡献是创建了梅蒙辛格医学院脑肿瘤（MMCBT）数据集，旨在解决可靠脑肿瘤MRI资源缺乏的问题。该数据集包括209名受试者（年龄范围为9至65岁）的MRI扫描图像，其中包括3671张肿瘤影像和13273张非肿瘤影像，所有影像均在专家监督下临床验证过。为了克服类别不平衡问题，对肿瘤类进行了增强处理，从而生成了一个适合深度学习研究的平衡数据集。
URL
https://arxiv.org/abs/2509.22956
PDF
-
Convolutional Set Transformer
2025-09-26 20:13:00Federico Chinello, Giacomo BoracchiAbstract
We introduce the Convolutional Set Transformer (CST), a novel neural architecture designed to process image sets of arbitrary cardinality that are visually heterogeneous yet share high-level semantics - such as a common category, scene, or concept. Existing set-input networks, e.g., Deep Sets and Set Transformer, are limited to vector inputs and cannot directly handle 3D image tensors. As a result, they must be cascaded with a feature extractor, typically a CNN, which encodes images into embeddings before the set-input network can model inter-image relationships. In contrast, CST operates directly on 3D image tensors, performing feature extraction and contextual modeling simultaneously, thereby enabling synergies between the two processes. This design yields superior performance in tasks such as Set Classification and Set Anomaly Detection and further provides native compatibility with CNN explainability methods such as Grad-CAM, unlike competing approaches that remain opaque. Finally, we show that CSTs can be pre-trained on large-scale datasets and subsequently adapted to new domains and tasks through standard Transfer Learning schemes. To support further research, we release CST-15, a CST backbone pre-trained on ImageNet (this https URL).
Abstract (translated)
我们介绍了卷积集变压器（Convolutional Set Transformer，CST），这是一种新颖的神经网络架构，旨在处理任意大小、视觉上异质但具备高级语义关联（如共同类别、场景或概念）的图像集合。现有的集输入网络，例如Deep Sets和Set Transformer，仅限于向量输入，并且无法直接处理3D图像张量。因此，它们必须与特征提取器级联使用，通常是CNN，在集输入网络能够建模图间关系之前，将图像编码为嵌入表示。相比之下，CST可以直接操作3D图像张量，在执行特征提取的同时进行上下文建模，从而实现这两种过程的协同作用。这种设计在集合分类和集合异常检测等任务中表现出了更优的效果，并且与仅限于向量输入的方法不同，它原生兼容CNN可解释性方法（如Grad-CAM）。最后，我们展示了CST可以通过大规模数据集预训练，并通过标准的迁移学习方案适应新的领域和任务。为了支持进一步的研究，我们将发布CST-15，这是一个在ImageNet上进行过预训练的CST骨干网络（详见此链接: this https URL）。
URL
https://arxiv.org/abs/2509.22889
PDF
-
Multimodal Slice Interaction Network Enhanced by Transfer Learning for Precise Segmentation of Internal Gross Tumor Volume in Lung Cancer PET/CT Imaging
2025-09-26 18:48:08Yi Luo, Yike Guo, Hamed Hooshangnejad, Rui Zhang, Xue Feng, Quan Chen, Wil Ngwa, Kai DingAbstract
Lung cancer remains the leading cause of cancerrelated deaths globally. Accurate delineation of internal gross tumor volume (IGTV) in PET/CT imaging is pivotal for optimal radiation therapy in mobile tumors such as lung cancer to account for tumor motion, yet is hindered by the limited availability of annotated IGTV datasets and attenuated PET signal intensity at tumor boundaries. In this study, we present a transfer learningbased methodology utilizing a multimodal interactive perception network with MAMBA, pre-trained on extensive gross tumor volume (GTV) datasets and subsequently fine-tuned on a private IGTV cohort. This cohort constitutes the PET/CT subset of the Lung-cancer Unified Cross-modal Imaging Dataset (LUCID). To further address the challenge of weak PET intensities in IGTV peripheral slices, we introduce a slice interaction module (SIM) within a 2.5D segmentation framework to effectively model inter-slice relationships. Our proposed module integrates channel and spatial attention branches with depthwise convolutions, enabling more robust learning of slice-to-slice dependencies and thereby improving overall segmentation performance. A comprehensive experimental evaluation demonstrates that our approach achieves a Dice of 0.609 on the private IGTV dataset, substantially surpassing the conventional baseline score of 0.385. This work highlights the potential of transfer learning, coupled with advanced multimodal techniques and a SIM to enhance the reliability and clinical relevance of IGTV segmentation for lung cancer radiation therapy planning.
Abstract (translated)
肺癌仍然是全球癌症相关死亡的主要原因。在正电子发射断层扫描/计算机断层扫描（PET/CT）成像中精确划分内部大体肿瘤体积（IGTV），对于移动性肿瘤如肺癌的放射治疗来说至关重要，以考虑肿瘤运动的影响。然而，这一过程受限于标注好的IGTV数据集的可用性以及肿瘤边界处正电子发射信号强度的衰减。 在本研究中，我们提出了一种基于迁移学习的方法，使用多模态交互感知网络（MAMBA），该网络在广泛的原发肿瘤体积（GTV）数据集上进行了预训练，并随后在一个私有的IGTV队列上进行微调。此队列构成了肺癌统一跨模式影像数据集（LUCID）中的PET/CT子集。为了进一步解决弱PET信号强度在IGTV外围切片中带来的挑战，我们引入了一个基于2.5D分割框架的切片交互模块（SIM），以有效地建模不同切片之间的关系。我们的提议模块结合了通道和空间注意力分支以及深度卷积，从而增强了对切片间依赖性的学习能力，并提高了整体分割性能。 全面的实验评估表明，我们在私有IGTV数据集上达到了0.609的Dice系数，显著超越了传统基线分数0.385。这项工作突显了迁移学习与高级多模态技术以及SIM相结合的潜力，以增强肺癌放射治疗计划中IGTV分割的可靠性和临床相关性。
URL
https://arxiv.org/abs/2509.22841
PDF
-
Low-Resource English-Tigrinya MT: Leveraging Multilingual Models, Custom Tokenizers, and Clean Evaluation Benchmarks
2025-09-24 15:02:57Hailay Kidu Teklehaymanot, Gebrearegawi Gidey, Wolfgang NejdlAbstract
Despite advances in Neural Machine Translation (NMT), low-resource languages like Tigrinya remain underserved due to persistent challenges, including limited corpora, inadequate tokenization strategies, and the lack of standardized evaluation benchmarks. This paper investigates transfer learning techniques using multilingual pretrained models to enhance translation quality for morphologically rich, low-resource languages. We propose a refined approach that integrates language-specific tokenization, informed embedding initialization, and domain-adaptive fine-tuning. To enable rigorous assessment, we construct a high-quality, human-aligned English-Tigrinya evaluation dataset covering diverse domains. Experimental results demonstrate that transfer learning with a custom tokenizer substantially outperforms zero-shot baselines, with gains validated by BLEU, chrF, and qualitative human evaluation. Bonferroni correction is applied to ensure statistical significance across configurations. Error analysis reveals key limitations and informs targeted refinements. This study underscores the importance of linguistically aware modeling and reproducible benchmarks in bridging the performance gap for underrepresented languages. Resources are available at this https URL and this https URL
Abstract (translated)
尽管神经机器翻译（NMT）取得了进展，但由于持续存在的挑战，如有限的语料库、不充分的分词策略和缺乏标准化的评估基准，像提格尼尼亚语这样的低资源语言仍然得不到充分服务。本文探讨了使用多语言预训练模型进行迁移学习技术，以提高形态丰富的低资源语言翻译质量的方法。我们提出了一种经过改进的方法，该方法结合了特定于语言的分词、信息嵌入初始化和领域适应性微调。 为了实现严格的评估，我们构建了一个高质量的人工对齐英提格尼尼亚语评价数据集，涵盖了不同的领域。实验结果表明，在自定义分词器的帮助下进行迁移学习显著优于零样本基准模型，并通过BLEU、chrF和定性的人工评估进行了验证。应用邦弗伦尼校正以确保在不同配置下的统计意义。错误分析揭示了关键的局限性并为针对性改进提供了信息。 这项研究强调了语言意识建模和可重复基准测试在弥合代表性不足的语言性能差距方面的的重要性。相关资源可以在[这里](https://this.url)和[这里](https://that.url)获得。请注意，提供的URL仅为占位符，请访问原文获取实际链接。
URL
https://arxiv.org/abs/2509.20209
PDF
-
CorIL: Towards Enriching Indian Language to Indian Language Parallel Corpora and Machine Translation Systems
2025-09-24 09:48:26Soham Bhattacharjee, Mukund K Roy, Yathish Poojary, Bhargav Dave, Mihir Raj, Vandan Mujadia, Baban Gain, Pruthwik Mishra, Arafat Ahsan, Parameswari Krishnamurthy, Ashwath Rao, Gurpreet Singh Josan, Preeti Dubey, Aadil Amin Kak, Anna Rao Kulkarni, Narendra VG, Sunita Arora, Rakesh Balbantray, Prasenjit Majumdar, Karunesh K Arora, Asif Ekbal, Dipti Mishra SharmaAbstract
India's linguistic landscape is one of the most diverse in the world, comprising over 120 major languages and approximately 1,600 additional languages, with 22 officially recognized as scheduled languages in the Indian Constitution. Despite recent progress in multilingual neural machine translation (NMT), high-quality parallel corpora for Indian languages remain scarce, especially across varied domains. In this paper, we introduce a large-scale, high-quality annotated parallel corpus covering 11 of these languages : English, Telugu, Hindi, Punjabi, Odia, Kashmiri, Sindhi, Dogri, Kannada, Urdu, and Gujarati comprising a total of 772,000 bi-text sentence pairs. The dataset is carefully curated and systematically categorized into three key domains: Government, Health, and General, to enable domain-aware machine translation research and facilitate effective domain adaptation. To demonstrate the utility of CorIL and establish strong benchmarks for future research, we fine-tune and evaluate several state-of-the-art NMT models, including IndicTrans2, NLLB, and BhashaVerse. Our analysis reveals important performance trends and highlights the corpus's value in probing model capabilities. For instance, the results show distinct performance patterns based on language script, with massively multilingual models showing an advantage on Perso-Arabic scripts (Urdu, Sindhi) while other models excel on Indic scripts. This paper provides a detailed domain-wise performance analysis, offering insights into domain sensitivity and cross-script transfer learning. By publicly releasing CorIL, we aim to significantly improve the availability of high-quality training data for Indian languages and provide a valuable resource for the machine translation research community.
Abstract (translated)
印度的语言景观是世界上最多样化的一种，包含了超过120种主要语言和大约1600种其他语言。根据《印度宪法》，其中有22种语言被列为计划语言（scheduled languages）。尽管近期在多语神经机器翻译(NMT)方面取得了一些进展，但在各种领域中高质量的平行语料库仍然十分稀缺。在这篇论文中，我们介绍了一个大规模且高质量标注的平行语料库，涵盖了11种印度的语言：英语、泰卢固语、印地语、旁遮普语、奥迪亚语、克什米尔语、信德语、多格里语、卡纳达语、乌尔都语和古吉拉特语。这个语料库总共包含772,000对双语文本句子。 该数据集经过精心策划，并被系统地分类为三大关键领域：政府、健康以及通用，以支持具有域意识的机器翻译研究并促进有效的领域适应。为了展示CorIL（可能是指上述提到的大规模平行语料库）的实用性，并为未来的研究建立强大的基准，我们对几种最先进的NMT模型进行了微调和评估，包括IndicTrans2、NLLB和BhashaVerse。我们的分析揭示了重要的性能趋势，并突出了该语料库在探索模型能力方面的价值。例如，结果表明，基于书写系统的不同语言模式具有明显的性能差异：大规模多语种模型在使用波斯-阿拉伯书写的语言（如乌尔都语、信德语）上表现出优势，而其他模型则在印度书写系统上的表现更为出色。 本文还提供了详细的领域内性能分析，为理解领域的敏感性和跨书写系统的迁移学习提供见解。通过公开发布CorIL，我们旨在显著提高印度语言高质量训练数据的可用性，并为此领域的机器翻译研究社区提供宝贵的资源。
URL
https://arxiv.org/abs/2509.19941
PDF
-
SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs
2025-09-24 09:10:27Ruyue Liu, Rong Yin, Xiangzhen Bo, Xiaoshuai Hao, Yong Liu, Jinwen Zhong, Can Ma, Weiping WangAbstract
Large scale pretrained models have revolutionized Natural Language Processing (NLP) and Computer Vision (CV), showcasing remarkable cross domain generalization abilities. However, in graph learning, models are typically trained on individual graph datasets, limiting their capacity to transfer knowledge across different graphs and tasks. This approach also heavily relies on large volumes of annotated data, which presents a significant challenge in resource-constrained settings. Unlike NLP and CV, graph structured data presents unique challenges due to its inherent heterogeneity, including domain specific feature spaces and structural diversity across various applications. To address these challenges, we propose a novel structure aware self supervised learning method for Text Attributed Graphs (SSTAG). By leveraging text as a unified representation medium for graph learning, SSTAG bridges the gap between the semantic reasoning of Large Language Models (LLMs) and the structural modeling capabilities of Graph Neural Networks (GNNs). Our approach introduces a dual knowledge distillation framework that co-distills both LLMs and GNNs into structure-aware multilayer perceptrons (MLPs), enhancing the scalability of large-scale TAGs. Additionally, we introduce an in-memory mechanism that stores typical graph representations, aligning them with memory anchors in an in-memory repository to integrate invariant knowledge, thereby improving the model's generalization ability. Extensive experiments demonstrate that SSTAG outperforms state-of-the-art models on cross-domain transfer learning tasks, achieves exceptional scalability, and reduces inference costs while maintaining competitive performance.
Abstract (translated)
大规模预训练模型已经在自然语言处理（NLP）和计算机视觉（CV）领域引发了革命，展示了卓越的跨域泛化能力。然而，在图学习中，模型通常在单个图数据集上进行训练，这限制了它们在不同图和任务间迁移知识的能力。这种方法还严重依赖于大量标注数据，在资源受限的情况下带来了重大挑战。与NLP和CV相比，图结构数据由于其固有的异质性（包括特定领域的特征空间及跨各种应用的结构性多样性）而提出了独特的挑战。 为了解决这些难题，我们提出了一种新颖的针对文本属性图（Text Attributed Graphs, TAGs）的感知结构自监督学习方法。通过利用文本作为图学习中的统一表示媒介，SSTAG弥合了大型语言模型（LLMs）的语义推理能力和图神经网络（GNNs）的结构建模能力之间的差距。我们的方法引入了一个双重知识蒸馏框架，将LLMs和GNNs的知识共同蒸馏到感知结构的多层感知机（MLPs）中，从而增强了大规模TAG的可扩展性。此外，我们还提出了一种内存机制，存储典型图表示，并将其与内存仓库中的记忆锚点对齐，以整合不变知识，从而提高模型的泛化能力。 广泛的实验表明，SSTAG在跨域迁移学习任务上超越了当前最先进的方法，在保持竞争性能的同时实现了卓越的可扩展性和推理成本降低。
URL
https://arxiv.org/abs/2510.01248
PDF
-
Towards Self-Supervised Foundation Models for Critical Care Time Series
2025-09-24 08:34:46Katja Naasunnguaq Jagd, Rachael DeVries, Ole WintherAbstract
Domain-specific foundation models for healthcare have expanded rapidly in recent years, yet foundation models for critical care time series remain relatively underexplored due to the limited size and availability of datasets. In this work, we introduce an early-stage pre-trained foundation model for critical care time-series based on the Bi-Axial Transformer (BAT), trained on pooled electronic health record datasets. We demonstrate effective transfer learning by fine-tuning the model on a dataset distinct from the training sources for mortality prediction, where it outperforms supervised baselines, particularly for small datasets ($<5,000$). These contributions highlight the potential of self-supervised foundation models for critical care times series to support generalizable and robust clinical applications in resource-limited settings.
Abstract (translated)
近年来，医疗领域的特定领域基础模型发展迅速，然而由于数据集规模和可用性的限制，针对重症监护时间序列的基础模型仍然相对较少被探索。在这项工作中，我们引入了一个基于双轴变换器（Bi-Axial Transformer, BAT）的早期预训练基础模型，并在汇总的电子健康记录数据集上进行了训练。通过将该模型微调应用于与训练源不同的数据集来进行死亡率预测，我们展示了有效的迁移学习能力，其性能优于监督基线模型，尤其是在小型数据集（少于5000个样本）的情况下。这些贡献突显了自我监督基础模型在重症监护时间序列中的潜力，能够支持资源有限环境下的通用性和稳健性的临床应用。
URL
https://arxiv.org/abs/2509.19885
PDF
-
SMILES-Inspired Transfer Learning for Quantum Operators in Generative Quantum Eigensolver
2025-09-24 02:54:09Zhi Yin, Xiaoran Li, Shengyu Zhang, Xin Li, Xiaojin ZhangAbstract
Given the inherent limitations of traditional Variational Quantum Eigensolver(VQE) algorithms, the integration of deep generative models into hybrid quantum-classical frameworks, specifically the Generative Quantum Eigensolver(GQE), represents a promising innovative approach. However, taking the Unitary Coupled Cluster with Singles and Doubles(UCCSD) ansatz which is widely used in quantum chemistry as an example, different molecular systems require constructions of distinct quantum operators. Considering the similarity of different molecules, the construction of quantum operators utilizing the similarity can reduce the computational cost significantly. Inspired by the SMILES representation method in computational chemistry, we developed a text-based representation approach for UCCSD quantum operators by leveraging the inherent representational similarities between different molecular systems. This framework explores text pattern similarities in quantum operators and employs text similarity metrics to establish a transfer learning framework. Our approach with a naive baseline setting demonstrates knowledge transfer between different molecular systems for ground-state energy calculations within the GQE paradigm. This discovery offers significant benefits for hybrid quantum-classical computation of molecular ground-state energies, substantially reducing computational resource requirements.
Abstract (translated)
鉴于传统变分量子特征值求解器（VQE）算法的固有限制，将深度生成模型集成到混合量子-经典框架中，特别是生成量子特征值求解器（GQE），代表了一种有前景的创新方法。然而，以在量子化学领域广泛应用的单双激发酉耦合簇近似（UCCSD）为例，不同的分子系统需要构建不同的量子算子。考虑到不同分子之间的相似性，利用这种相似性来构造量子算子可以显著降低计算成本。 受计算机化学中SMILES表示方法的启发，我们开发了一种基于文本表示的方法，用于表示UCCSD量子算子，该方法利用了不同分子系统之间固有的表征相似性。这一框架探索了量子算子中的文本模式相似性，并采用文本相似度度量来建立迁移学习框架。在我们的方法中，通过一个简单的基线设置展示了GQE范式下不同分子系统之间的知识转移，在地态能量计算方面的应用。 这种方法为混合量子-经典计算的分子地态能量提供了显著的好处，大大减少了所需的计算资源需求。
URL
https://arxiv.org/abs/2509.19715
PDF
-
Parameter-Efficient Multi-Task Learning via Progressive Task-Specific Adaptation
2025-09-23 21:51:04Neeraj Gangwar, Anshuka Rangi, Rishabh Deshmukh, Holakou Rahmanian, Yesh Dattatreya, Nickvash KaniAbstract
Parameter-efficient fine-tuning methods have emerged as a promising solution for adapting pre-trained models to various downstream tasks. While these methods perform well in single-task learning, extending them to multi-task learning exacerbates common challenges, such as task interference and negative transfer, due to the limited number of trainable parameters. To address these issues, we introduce progressive task-specific multi-task adaptation, a novel parameter-efficient approach for multi-task learning. This approach introduces adapter modules in a pre-trained model such that these modules are shared across all tasks in the initial layers and become progressively more task-specific in the later layers. The motivation is to reduce the conflicts among tasks by allowing transfer learning across all tasks in the initial layers and enabling task-specific learning toward the prediction heads. Additionally, we propose a gradient-based approach for computing task similarity and use this measure to allocate similar tasks to the shared adapter modules. Our task similarity method introduces minimal overhead in the pipeline. We evaluate our approach by adapting the Swin Transformer for dense prediction tasks. Experiments on the PASCAL and NYUD-v2 datasets demonstrate that our approach outperforms a fully fine-tuned multi-task model while requiring only one-fifth of the trainable parameters. This approach achieves better relative improvement to single-task fine-tuning while reducing the number of trainable parameters and surpasses the current state-of-the-art methods for parameter-efficient multi-task learning.
Abstract (translated)
参数高效的微调方法已成为将预训练模型适应各种下游任务的有希望的解决方案。尽管这些方法在单一任务学习中表现出色，但将其扩展到多任务学习时，由于可训练参数数量有限，会加剧诸如任务干扰和负向迁移等常见挑战。为了解决这些问题，我们引入了一种新颖的参数高效多任务适应方法——渐进式特定任务多任务自适应。该方法在预训练模型中引入适配器模块，并且这些模块在初始层被所有任务共享，在后续层逐渐变得更具任务特异性。其动机在于通过允许跨所有任务的迁移学习来减少任务间的冲突，同时使预测头朝向任务特定的学习。此外，我们提出了一种基于梯度的方法来计算任务相似性，并使用这种方法将相似的任务分配给共享适配器模块。我们的任务相似性方法在管道中引入了最小开销。 我们在密集预测任务上通过微调Swin Transformer来评估此方法。实验结果表明，在PASCAL和NYUD-v2数据集上的性能优于完全微调的多任务模型，而所需训练参数仅为后者的五分之一。该方法相对于单一任务微调取得了更好的相对改进，并且在减少可训练参数数量的同时超越了当前最先进的参数高效多任务学习方法。
URL
https://arxiv.org/abs/2509.19602
PDF
-
A Realistic Evaluation of Cross-Frequency Transfer Learning and Foundation Forecasting Models
2025-09-23 18:19:50Kin G. Olivares, Malcolm Wolff, Tatiana Konstantinova, Shankar Ramasubramanian, Andrew Gordon Wilson, Andres Potapczynski, Willa Potosnak, Mengfei Cao, Boris Oreshkin, Dmitry EfimovAbstract
Cross-frequency transfer learning (CFTL) has emerged as a popular framework for curating large-scale time series datasets to pre-train foundation forecasting models (FFMs). Although CFTL has shown promise, current benchmarking practices fall short of accurately assessing its performance. This shortcoming stems from many factors: an over-reliance on small-scale evaluation datasets; inadequate treatment of sample size when computing summary statistics; reporting of suboptimal statistical models; and failing to account for non-negligible risks of overlap between pre-training and test datasets. To address these limitations, we introduce a unified reimplementation of widely-adopted neural forecasting networks, adapting them for the CFTL setup; we pre-train only on proprietary and synthetic data, being careful to prevent test leakage; and we evaluate on 15 large, diverse public forecast competition datasets. Our empirical analysis reveals that statistical models' accuracy is frequently underreported. Notably, we confirm that statistical models and their ensembles consistently outperform existing FFMs by more than 8.2% in sCRPS, and by more than 20% MASE, across datasets. However, we also find that synthetic dataset pre-training does improve the accuracy of a FFM by 7% percent.
Abstract (translated)
跨频率迁移学习（CFTL）已成为一种流行框架，用于对大规模时间序列数据集进行预处理以训练基础预测模型（FFMs）。尽管CFTL显示出潜力，但当前的基准测试方法未能准确评估其性能。这一不足源于多个因素：过度依赖小规模评估数据集；在计算汇总统计数据时忽视样本量的影响；报告次优统计模型的表现；以及未能考虑到预训练和测试数据集之间重叠的风险。为了解决这些问题，我们引入了一种广泛采用的神经预测网络的统一重新实现方法，并针对CFTL设置进行了调整；仅使用专有及合成数据进行预训练，同时小心避免测试泄漏；并在15个大型且多样的公共预测竞赛数据集中进行评估。我们的实证分析揭示了统计模型的准确性经常被低估。值得注意的是，我们确认统计模型及其集成始终优于现有FFMs，在sCRPS指标上超过8.2%，在MASE（平均绝对规模误差）指标上超过20%。然而，我们也发现使用合成数据集进行预训练可以将FFM的准确率提高7%。
URL
https://arxiv.org/abs/2509.19465
PDF
-
No Labels Needed: Zero-Shot Image Classification with Collaborative Self-Learning
2025-09-23 12:54:52Matheus Vin\'icius Todescato, Joel Lu\'is CarboneraAbstract
While deep learning, including Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), has significantly advanced classification performance, its typical reliance on extensive annotated datasets presents a major obstacle in many practical scenarios where such data is scarce. Vision-language models (VLMs) and transfer learning with pre-trained visual models appear as promising techniques to deal with this problem. This paper proposes a novel zero-shot image classification framework that combines a VLM and a pre-trained visual model within a self-learning cycle. Requiring only the set of class names and no labeled training data, our method utilizes a confidence-based pseudo-labeling strategy to train a lightweight classifier directly on the test data, enabling dynamic adaptation. The VLM identifies high-confidence samples, and the pre-trained visual model enhances their visual representations. These enhanced features then iteratively train the classifier, allowing the system to capture complementary semantic and visual cues without supervision. Notably, our approach avoids VLM fine-tuning and the use of large language models, relying on the visual-only model to reduce the dependence on semantic representation. Experimental evaluations on ten diverse datasets demonstrate that our approach outperforms the baseline zero-shot method.
Abstract (translated)
虽然深度学习（包括卷积神经网络(CNNs)和视觉变换器(ViTs)）显著提升了分类性能，但其通常依赖于大规模的标注数据集，在许多实际场景中存在重大障碍，因为在这些场景中很难获得足够的标注数据。视觉-语言模型(VLMs)和基于预训练视觉模型的迁移学习被视为解决这一问题的有前景的技术。本文提出了一种新的零样本图像分类框架，该框架结合了VLM和一个预训练的视觉模型，并在一个自我学习循环内工作。我们的方法仅需类别名称集合而不需要标注的训练数据，并采用基于置信度的伪标签策略，在测试数据上直接训练轻量级分类器，从而实现动态适应性。VLM识别高置信度样本，而预训练的视觉模型则增强这些样本的视觉表示。这些改进后的特征将迭代地用于训练分类器，使系统能够无监督地捕捉互补的语义和视觉线索。值得注意的是，我们的方法避免了对VLM进行微调以及使用大型语言模型，并依赖于仅基于视觉的模型来减少对语义表达的依赖。 在十个不同数据集上的实验评估表明，与基线零样本方法相比，我们的方法表现更优。
URL
https://arxiv.org/abs/2509.18938
PDF
