Transfer_Learning
-
Conditional Graph Neural Network for Predicting Soft Tissue Deformation and Forces
2025-07-07 13:33:39Madina Kojanazarova, Florentin Bieder, Robin Sandk\"uhler, Philippe C. CattinAbstract
Soft tissue simulation in virtual environments is becoming increasingly important for medical applications. However, the high deformability of soft tissue poses significant challenges. Existing methods rely on segmentation, meshing and estimation of stiffness properties of tissues. In addition, the integration of haptic feedback requires precise force estimation to enable a more immersive experience. We introduce a novel data-driven model, a conditional graph neural network (cGNN) to tackle this complexity. Our model takes surface points and the location of applied forces, and is specifically designed to predict the deformation of the points and the forces exerted on them. We trained our model on experimentally collected surface tracking data of a soft tissue phantom and used transfer learning to overcome the data scarcity by initially training it with mass-spring simulations and fine-tuning it with the experimental data. This approach improves the generalisation capability of the model and enables accurate predictions of tissue deformations and corresponding interaction forces. The results demonstrate that the model can predict deformations with a distance error of 0.35$\pm$0.03 mm for deformations up to 30 mm and the force with an absolute error of 0.37$\pm$0.05 N for forces up to 7.5 N. Our data-driven approach presents a promising solution to the intricate challenge of simulating soft tissues within virtual environments. Beyond its applicability in medical simulations, this approach holds the potential to benefit various fields where realistic soft tissue simulations are required.
Abstract (translated)
在虚拟环境中进行软组织模拟对于医学应用变得越来越重要。然而，由于软组织的高度可变形性，这对现有技术提出了重大挑战。目前的方法依赖于分割、网格划分和组织硬度属性的估算。此外，为了实现更加沉浸式的体验，触觉反馈的集成需要精确的力量估计。我们引入了一种基于数据驱动的新模型——条件图神经网络（cGNN），以解决这一复杂性问题。我们的模型接受表面点及其受力位置作为输入，并专门设计用于预测这些点的变形及作用在它们上的力量。 我们在软组织仿体实验收集到的表面跟踪数据上训练了该模型，并通过转移学习克服数据稀缺的问题，即先使用质量-弹簧仿真进行初步训练，然后用实验数据进行微调。这种方法提升了模型的泛化能力，使其能够准确预测组织变形及其相互作用力。实验证明，在30毫米范围内的变形距离误差为0.35±0.03毫米，在7.5牛顿范围内的力量绝对误差为0.37±0.05牛顿。 我们基于数据驱动的方法为在虚拟环境中模拟软组织提供了一个有前景的解决方案。除了医学仿真领域的应用，这种方法还具有潜在的应用价值，适用于任何需要现实软组织仿真的领域。
URL
https://arxiv.org/abs/2507.05315
PDF
-
HGNet: High-Order Spatial Awareness Hypergraph and Multi-Scale Context Attention Network for Colorectal Polyp Detection
2025-07-07 11:09:05Xiaofang Liu, Lingling Sun, Xuqing Zhang, Yuannong Ye, Bin zhaoAbstract
Colorectal cancer (CRC) is closely linked to the malignant transformation of colorectal polyps, making early detection essential. However, current models struggle with detecting small lesions, accurately localizing boundaries, and providing interpretable decisions. To address these issues, we propose HGNet, which integrates High-Order Spatial Awareness Hypergraph and Multi-Scale Context Attention. Key innovations include: (1) an Efficient Multi-Scale Context Attention (EMCA) module to enhance lesion feature representation and boundary modeling; (2) the deployment of a spatial hypergraph convolution module before the detection head to capture higher-order spatial relationships between nodes; (3) the application of transfer learning to address the scarcity of medical image data; and (4) Eigen Class Activation Map (Eigen-CAM) for decision visualization. Experimental results show that HGNet achieves 94% accuracy, 90.6% recall, and 90% mAP@0.5, significantly improving small lesion differentiation and clinical interpretability. The source code will be made publicly available upon publication of this paper.
Abstract (translated)
结直肠癌（CRC）与结直肠息肉的恶性转化密切相关，因此早期检测至关重要。然而，目前的模型在检测小病变、准确确定边界以及提供可解释性决策方面存在困难。为了解决这些问题，我们提出了HGNet，该网络结合了高阶空间感知超图和多尺度上下文注意力机制。 关键创新包括： 1. **高效多尺度上下文注意（EMCA）模块**：增强了病灶特征表示和边界建模。 2. **在检测头部之前部署空间超图卷积模块**：以捕获节点之间的高阶空间关系。 3. **应用迁移学习**：以应对医疗图像数据的稀缺性问题。 4. **特征类激活图（Eigen-CAM）**：用于决策可视化。 实验结果显示，HGNet达到了94%的准确率、90.6%的召回率和90%mAP@0.5，在小病变区分和临床可解释性方面有显著提升。该论文发表后，源代码将公开发布。
URL
https://arxiv.org/abs/2507.04880
PDF
-
Towards Human-in-the-Loop Onset Detection: A Transfer Learning Approach for Maracatu
Abstract
We explore transfer learning strategies for musical onset detection in the Afro-Brazilian Maracatu tradition, which features complex rhythmic patterns that challenge conventional models. We adapt two Temporal Convolutional Network architectures: one pre-trained for onset detection (intra-task) and another for beat tracking (inter-task). Using only 5-second annotated snippets per instrument, we fine-tune these models through layer-wise retraining strategies for five traditional percussion instruments. Our results demonstrate significant improvements over baseline performance, with F1 scores reaching up to 0.998 in the intra-task setting and improvements of over 50 percentage points in best-case scenarios. The cross-task adaptation proves particularly effective for time-keeping instruments, where onsets naturally align with beat positions. The optimal fine-tuning configuration varies by instrument, highlighting the importance of instrument-specific adaptation strategies. This approach addresses the challenges of underrepresented musical traditions, offering an efficient human-in-the-loop methodology that minimizes annotation effort while maximizing performance. Our findings contribute to more inclusive music information retrieval tools applicable beyond Western musical contexts.
Abstract (translated)
我们在非洲巴西马腊卡图传统中的音乐起音检测中探索了迁移学习策略，这种传统包含复杂的节奏模式，给常规模型带来了挑战。我们对两个时间卷积网络架构进行了调整：一个预先训练用于起音检测（任务内），另一个则用于节拍跟踪（跨任务）。仅使用每个乐器5秒的标注片段，通过逐层再训练策略对这两种模型进行微调，以适应五种传统打击乐器。我们的实验结果表明，在任务内的设置下，F1分数达到了0.998的显著改善，并且在最佳情况下比基线性能提高了超过50个百分点。跨任务调整对于节奏控制型乐器尤为有效，因为起音自然与节拍位置对齐。最有效的微调配置因乐器而异，这强调了针对特定乐器制定适应策略的重要性。这种做法解决了不被充分代表的音乐传统所面临的挑战，并提供了一种高效的人机协作方法，在最小化标注工作量的同时最大化性能。我们的发现有助于开发适用于非西方音乐背景下的更包容性音乐信息检索工具。
URL
https://arxiv.org/abs/2507.04858
PDF
-
Model Compression using Progressive Channel Pruning
2025-07-07 09:12:03Jinyang Guo, Weichen Zhang, Wanli Ouyang, Dong XuAbstract
In this work, we propose a simple but effective channel pruning framework called Progressive Channel Pruning (PCP) to accelerate Convolutional Neural Networks (CNNs). In contrast to the existing channel pruning methods that prune channels only once per layer in a layer-by-layer fashion, our new progressive framework iteratively prunes a small number of channels from several selected layers, which consists of a three-step attempting-selecting-pruning pipeline in each iteration. In the attempting step, we attempt to prune a pre-defined number of channels from one layer by using any existing channel pruning methods and estimate the accuracy drop for this layer based on the labelled samples in the validation set. In the selecting step, based on the estimated accuracy drops for all layers, we propose a greedy strategy to automatically select a set of layers that will lead to less overall accuracy drop after pruning these layers. In the pruning step, we prune a small number of channels from these selected layers. We further extend our PCP framework to prune channels for the deep transfer learning methods like Domain Adversarial Neural Network (DANN), in which we effectively reduce the data distribution mismatch in the channel pruning process by using both labelled samples from the source domain and pseudo-labelled samples from the target domain. Our comprehensive experiments on two benchmark datasets demonstrate that our PCP framework outperforms the existing channel pruning approaches under both supervised learning and transfer learning settings.
Abstract (translated)
在这项工作中，我们提出了一种简单但有效的通道剪枝框架，称为渐进式通道剪枝（PCP），用于加速卷积神经网络（CNN）。与现有的逐层一次性修剪一层中所有通道的通道剪枝方法不同，我们的新渐进式框架通过迭代地从选定的几层中剪除少量通道来工作。每个迭代包括尝试、选择和剪枝三个步骤的管道。 在尝试阶段，我们使用任何现有通道剪枝方法对一个预定义数量的通道进行逐层修剪，并基于验证集中的标记样本估计该层的精度损失。 在选择阶段，根据所有层上的估算精度下降值，我们提出了一种贪婪策略以自动选择一组经过这些操作后整体精度下降最小的层级。 在剪枝阶段，从选定的各层中实际剪除少量通道。 此外，我们将PCP框架扩展到了深层迁移学习方法（例如域对抗神经网络DANN）中的通道剪枝。在这个过程中，我们通过使用源领域和目标领域的标记样本以及伪标签样本来有效减少数据分布不匹配问题，在通道修剪流程中提升了性能表现。 我们在两个基准数据集上的全面实验表明，我们的PCP框架在监督学习和迁移学习设置下均优于现有的通道剪枝方法。
URL
https://arxiv.org/abs/2507.04792
PDF
-
MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model
2025-07-06 03:38:05K. Sahit Reddy, N. Ragavenderan, Vasanth K., Ganesh N. Naik, Vishalakshi Prabhu, Nagaraja G. SAbstract
Recent advances in natural language processing (NLP) have been driven bypretrained language models like BERT, RoBERTa, T5, and GPT. Thesemodels excel at understanding complex texts, but biomedical literature, withits domain-specific terminology, poses challenges that models likeWord2Vec and bidirectional long short-term memory (Bi-LSTM) can't fullyaddress. GPT and T5, despite capturing context, fall short in tasks needingbidirectional understanding, unlike BERT. Addressing this, we proposedMedicalBERT, a pretrained BERT model trained on a large biomedicaldataset and equipped with domain-specific vocabulary that enhances thecomprehension of biomedical terminology. MedicalBERT model is furtheroptimized and fine-tuned to address diverse tasks, including named entityrecognition, relation extraction, question answering, sentence similarity, anddocument classification. Performance metrics such as the F1-score,accuracy, and Pearson correlation are employed to showcase the efficiencyof our model in comparison to other BERT-based models such as BioBERT,SciBERT, and ClinicalBERT. MedicalBERT outperforms these models onmost of the benchmarks, and surpasses the general-purpose BERT model by5.67% on average across all the tasks evaluated respectively. This work alsounderscores the potential of leveraging pretrained BERT models for medicalNLP tasks, demonstrating the effectiveness of transfer learning techniques incapturing domain-specific information. (PDF) MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model. Available from: this https URL [accessed Jul 06 2025].
Abstract (translated)
近期，自然语言处理（NLP）领域的进展主要归功于像BERT、RoBERTa、T5和GPT这样的预训练语言模型。这些模型在理解复杂文本方面表现出色，但生物医学文献因其领域特定的术语而提出了挑战，这是Word2Vec和双向长短时记忆网络（Bi-LSTM）等传统方法难以完全解决的问题。虽然GPT和T5能够捕捉上下文信息，但在需要双向理解的任务中不如BERT表现优秀。为此，我们提出了一种称为MedicalBERT的新模型，这是一种基于大量生物医学数据集训练的预训练BERT模型，并配备了领域特定词汇表以增强对生物医学术语的理解能力。通过进一步优化和微调，MedicalBERT能够在命名实体识别、关系抽取、问答、句子相似度和文档分类等多样化任务中发挥出色性能。 为了展示我们的模型相较于其他基于BERT的模型（如BioBERT、SciBERT和ClinicalBERT）在效率上的优势，我们采用了诸如F1分数、准确率以及Pearson相关系数等多种性能指标进行评估。实验结果表明，在大多数基准测试上，MedicalBERT均超越了这些模型，并且平均而言比通用型的BERT模型高出5.67%的表现（按任务分别计算）。这项工作不仅强调了利用预训练的BERT模型来处理医疗NLP任务的巨大潜力，还证明了迁移学习技术在捕捉领域特定信息方面的有效性。 论文《MedicalBERT：使用基于预训练BERT的模型增强生物医学自然语言处理》可以从以下链接下载：[此链接] [最后访问日期: 2025年7月6日]。
URL
https://arxiv.org/abs/2507.08013
PDF
-
Mixed-Sample SGD: an End-to-end Analysis of Supervised Transfer Learning
2025-07-06 00:03:34Yuyang Deng, Samory KpotufeAbstract
Theoretical works on supervised transfer learning (STL) -- where the learner has access to labeled samples from both source and target distributions -- have for the most part focused on statistical aspects of the problem, while efficient optimization has received less attention. We consider the problem of designing an SGD procedure for STL that alternates sampling between source and target data, while maintaining statistical transfer guarantees without prior knowledge of the quality of the source data. A main algorithmic difficulty is in understanding how to design such an adaptive sub-sampling mechanism at each SGD step, to automatically gain from the source when it is informative, or bias towards the target and avoid negative transfer when the source is less informative. We show that, such a mixed-sample SGD procedure is feasible for general prediction tasks with convex losses, rooted in tracking an abstract sequence of constrained convex programs that serve to maintain the desired transfer guarantees. We instantiate these results in the concrete setting of linear regression with square loss, and show that the procedure converges, with $1/\sqrt{T}$ rate, to a solution whose statistical performance on the target is adaptive to the a priori unknown quality of the source. Experiments with synthetic and real datasets support the theory.
Abstract (translated)
关于监督迁移学习（STL）的理论工作——在这种情况下，学习者可以从源分布和目标分布中获取标记样本——大多数研究主要集中在问题的统计方面，而高效的优化方法则受到较少的关注。我们考虑设计一种针对STL的SGD过程，该过程在源数据和目标数据之间交替采样，并且在没有先验知识的情况下维持统计迁移保证。算法的一个重要困难在于如何设计适应性的子抽样机制，在每一步的SGD过程中，当源数据具有信息量时可以自动从中获益；而当源数据不太具信息量时，则偏向于目标并避免负面转移。 我们展示了一种混合样本SGD过程对于一般预测任务（使用凸损失函数）是可行的，并且该过程根植于跟踪一系列抽象的约束凸程序，这些程序有助于保持所需的迁移保证。我们在具体的线性回归和平方损失环境中实现了这些结果，并证明了这种程序以$1/\sqrt{T}$的速度收敛至一个统计性能适应未知源质量的目标解。 通过合成数据集和真实数据集进行的实验支持了这一理论。
URL
https://arxiv.org/abs/2507.04194
PDF
-
ChestGPT: Integrating Large Language Models and Vision Transformers for Disease Detection and Localization in Chest X-Rays
2025-07-04 17:58:52Shehroz S. Khan, Petar Przulj, Ahmed Ashraf, Ali AbediAbstract
The global demand for radiologists is increasing rapidly due to a growing reliance on medical imaging services, while the supply of radiologists is not keeping pace. Advances in computer vision and image processing technologies present significant potential to address this gap by enhancing radiologists' capabilities and improving diagnostic accuracy. Large language models (LLMs), particularly generative pre-trained transformers (GPTs), have become the primary approach for understanding and generating textual data. In parallel, vision transformers (ViTs) have proven effective at converting visual data into a format that LLMs can process efficiently. In this paper, we present ChestGPT, a deep-learning framework that integrates the EVA ViT with the Llama 2 LLM to classify diseases and localize regions of interest in chest X-ray images. The ViT converts X-ray images into tokens, which are then fed, together with engineered prompts, into the LLM, enabling joint classification and localization of diseases. This approach incorporates transfer learning techniques to enhance both explainability and performance. The proposed method achieved strong global disease classification performance on the VinDr-CXR dataset, with an F1 score of 0.76, and successfully localized pathologies by generating bounding boxes around the regions of interest. We also outline several task-specific prompts, in addition to general-purpose prompts, for scenarios radiologists might encounter. Overall, this framework offers an assistive tool that can lighten radiologists' workload by providing preliminary findings and regions of interest to facilitate their diagnostic process.
Abstract (translated)
全球对放射科医生的需求由于医疗成像服务的依赖日益增加而迅速增长，然而放射科医生的数量并未跟上这一需求的步伐。计算机视觉和图像处理技术的进步为缓解这一缺口提供了巨大的潜力，能够增强放射科医生的能力并提高诊断准确性。大型语言模型（LLMs），尤其是生成式预训练变换器（GPTs），已经成为理解和生成文本数据的主要方法。同时，视觉变换器（ViTs）已被证明能够有效将视觉数据转换成适合LLMs高效处理的格式。在这篇论文中，我们提出了ChestGPT，这是一个深度学习框架，它结合了EVA ViT和Llama 2 LLM来对胸部X光片进行疾病分类及定位感兴趣的区域。ViT将X射线图像转化为令牌，然后与工程化的提示一起输入LLM，从而实现疾病的联合分类和定位。这种方法通过采用迁移学习技术增强了可解释性和性能。我们提出的方法在VinDr-CXR数据集上的全球疾病分类任务中表现出色，取得了0.76的F1分数，并成功地生成了围绕感兴趣区域的边界框来定位病灶。此外，我们也为放射科医生可能遇到的各种场景提供了几种特定任务提示和通用提示。总体而言，该框架提供了一种辅助工具，通过提供初步发现及感兴趣的区域来减轻放射科医生的工作负担，从而方便他们的诊断过程。
URL
https://arxiv.org/abs/2507.03739
PDF
-
SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications
2025-07-04 13:48:12Yana Hasson, Pauline Luc, Liliane Momeni, Maks Ovsjanikov, Guillaume Le Moing, Alina Kuznetsova, Ira Ktena, Jennifer J. Sun, Skanda Koppula, Dilara Gokay, Joseph Heyward, Etienne Pot, Andrew ZissermanAbstract
In recent years, there has been a proliferation of spatiotemporal foundation models in different scientific disciplines. While promising, these models are often domain-specific and are only assessed within the particular applications for which they are designed. Given that many tasks can be represented as video modeling problems, video foundation models (ViFMs) hold considerable promise as general-purpose domain-agnostic approaches. However, it is not known whether the knowledge acquired on large-scale but potentially out-of-domain data can be effectively transferred across diverse scientific disciplines, and if a single, pretrained ViFM can be competitive with domain-specific baselines. To address this, we introduce SciVid, a comprehensive benchmark comprising five *Sci*entific *Vid*eo tasks, across medical computer vision, animal behavior, and weather forecasting. We adapt six leading ViFMs to SciVid using simple trainable readout modules, establishing strong baselines and demonstrating the potential for effective transfer learning. Specifically, we show that state-of-the-art results can be obtained in several applications by leveraging the general-purpose representations from ViFM backbones. Furthermore, our results reveal the limitations of existing ViFMs, and highlight opportunities for the development of generalizable models for high-impact scientific applications. We release our code at this https URL to facilitate further research in the development of ViFMs.
Abstract (translated)
近年来，不同科学领域中涌现出了许多时空基础模型。尽管这些模型前景广阔，但它们往往是特定领域的，并且仅在其设计的应用场景中进行评估。鉴于许多任务可以被表示为视频建模问题，视频基础模型（Video Foundation Models, ViFMs）作为通用的、不受限于特定领域的解决方案展现了巨大的潜力。然而，尚不清楚在大规模但可能超出领域范围的数据上获得的知识能否有效地跨多个科学学科传递，以及单一的预训练ViFM是否能与特定领域的基准方法相竞争。 为了解决这些问题，我们引入了SciVid，这是一个全面的基准测试集，涵盖了五个*科研视频任务*（Scientific Video Tasks），涉及医学计算机视觉、动物行为和天气预报。我们将六个领先的ViFMs适配到SciVid中，使用简单的可训练输出模块，并建立了强大的基线模型，展示了有效迁移学习的可能性。具体来说，我们证明了通过利用来自ViFM骨干网络的通用表示形式，在多个应用中可以获得最先进的结果。此外，我们的结果显示现有ViFMs的局限性，并强调为具有高影响力的应用开发泛化模型的机会。 为了促进进一步的研究和视频基础模型（Video Foundation Models, ViFMs）的发展，我们公开了代码，网址是[此处]（请将"this https URL"替换为实际的链接）。
URL
https://arxiv.org/abs/2507.03578
PDF
-
Understanding Knowledge Transferability for Transfer Learning: A Survey
2025-07-03 21:06:30Haohua Wang, Jingge Wang, Zijie Zhao, Yang Tan, Yanru Wu, Hanbing Liu, Jingyun Yang, Enming Zhang, Xiangyu Chen, Zhengze Rong, Shanxin Guo, Yang LiAbstract
Transfer learning has become an essential paradigm in artificial intelligence, enabling the transfer of knowledge from a source task to improve performance on a target task. This approach, particularly through techniques such as pretraining and fine-tuning, has seen significant success in fields like computer vision and natural language processing. However, despite its widespread use, how to reliably assess the transferability of knowledge remains a challenge. Understanding the theoretical underpinnings of each transferability metric is critical for ensuring the success of transfer learning. In this survey, we provide a unified taxonomy of transferability metrics, categorizing them based on transferable knowledge types and measurement granularity. This work examines the various metrics developed to evaluate the potential of source knowledge for transfer learning and their applicability across different learning paradigms emphasizing the need for careful selection of these metrics. By offering insights into how different metrics work under varying conditions, this survey aims to guide researchers and practitioners in selecting the most appropriate metric for specific applications, contributing to more efficient, reliable, and trustworthy AI systems. Finally, we discuss some open challenges in this field and propose future research directions to further advance the application of transferability metrics in trustworthy transfer learning.
Abstract (translated)
迁移学习已经成为人工智能领域的一个重要范式，它通过将知识从源任务转移到目标任务来提升性能。这种方法在计算机视觉和自然语言处理等领域中，尤其是在预训练和微调技术的帮助下取得了显著成功。然而，尽管迁移学习得到了广泛应用，如何可靠地评估知识的可转移性仍然是一项挑战。理解每种可转移性度量的理论基础对于确保迁移学习的成功至关重要。在这项调查中，我们提供了一个统一的可转移性度量分类体系，基于可转移的知识类型和测量粒度进行分类。这项工作审查了各种用于评估源知识在迁移学习中的潜力以及这些度量在不同学习范式中的适用性的指标，并强调了谨慎选择这些度量的重要性。 通过揭示这些不同的度量在不同条件下的工作机制，本调查旨在指导研究人员和实践者为特定应用选择最合适的度量标准，从而促进更高效、可靠且值得信赖的人工智能系统的开发。最后，我们讨论了一些开放性挑战并提出未来研究方向，以进一步推进可转移性度量在可信迁移学习中的应用。
URL
https://arxiv.org/abs/2507.03175
PDF
-
Wildlife Target Re-Identification Using Self-supervised Learning in Non-Urban Settings
2025-07-03 07:56:54Mufhumudzi Muthivhi, Terence L. van ZylAbstract
Wildlife re-identification aims to match individuals of the same species across different observations. Current state-of-the-art (SOTA) models rely on class labels to train supervised models for individual classification. This dependence on annotated data has driven the curation of numerous large-scale wildlife datasets. This study investigates self-supervised learning Self-Supervised Learning (SSL) for wildlife re-identification. We automatically extract two distinct views of an individual using temporal image pairs from camera trap data without supervision. The image pairs train a self-supervised model from a potentially endless stream of video data. We evaluate the learnt representations against supervised features on open-world scenarios and transfer learning in various wildlife downstream tasks. The analysis of the experimental results shows that self-supervised models are more robust even with limited data. Moreover, self-supervised features outperform supervision across all downstream tasks. The code is available here this https URL.
Abstract (translated)
野生动物再识别的目标是在不同的观测中匹配同一物种的个体。当前最先进的（SOTA）模型依赖于类别标签来训练用于个体分类的监督模型。这种对标注数据的依赖推动了大量大型野生动物数据集的整理。本研究探讨了无监督自监督学习（Self-Supervised Learning, SSL）在野生动物再识别中的应用。我们利用摄像陷阱数据中成对的时间图像，自动提取每个个体的两个不同视角，而无需任何监督。这些图像对可以从潜在无限的视频流数据集中训练出一个自监督模型。 我们在开放世界场景和各种野生动物下游任务的迁移学习上评估了所学表示与监督特征的表现。实验结果分析表明，在有限的数据下，自监督模型更加稳健，并且在所有下游任务中，无监督特征的表现均优于监督方法。相关代码可在[此处](此链接应为一个指向具体代码仓库或项目的URL)获取。 请注意，原文中的“this https URL”是一个占位符，你需要将它替换为实际的链接地址以便访问相关资料。
URL
https://arxiv.org/abs/2507.02403
PDF
-
Generalized Adaptive Transfer Network: Enhancing Transfer Learning in Reinforcement Learning Across Domains
2025-07-02 21:33:48Abhishek Verma, Nallarasan V, Balaraman RavindranAbstract
Transfer learning in Reinforcement Learning (RL) enables agents to leverage knowledge from source tasks to accelerate learning in target tasks. While prior work, such as the Attend, Adapt, and Transfer (A2T) framework, addresses negative transfer and selective transfer, other critical challenges remain underexplored. This paper introduces the Generalized Adaptive Transfer Network (GATN), a deep RL architecture designed to tackle task generalization across domains, robustness to environmental changes, and computational efficiency in transfer. GATN employs a domain-agnostic representation module, a robustness-aware policy adapter, and an efficient transfer scheduler to achieve these goals. We evaluate GATN on diverse benchmarks, including Atari 2600, MuJoCo, and a custom chatbot dialogue environment, demonstrating superior performance in cross-domain generalization, resilience to dynamic environments, and reduced computational overhead compared to baselines. Our findings suggest GATN is a versatile framework for real-world RL applications, such as adaptive chatbots and robotic control.
Abstract (translated)
在强化学习（RL）中，迁移学习使智能体能够利用来自源任务的知识来加速目标任务的学习过程。尽管之前的工作，如“Attend, Adapt, and Transfer”(A2T) 框架已经解决了负向迁移和选择性迁移的问题，但其他关键挑战仍然没有得到充分探索。本文介绍了广义自适应传输网络（GATN），这是一种深度强化学习架构，旨在解决跨领域任务泛化、对环境变化的鲁棒性和迁移过程中的计算效率等问题。 GATN采用了一个域无关表示模块，一个感知鲁棒性的策略适配器以及一个高效的转移调度程序来实现这些目标。我们在包括Atari 2600游戏、MuJoCo物理模拟器和一个自定义聊天机器人对话环境在内的多种基准测试中评估了GATN的性能，结果显示它在跨域泛化能力、对动态环境的适应能力和计算开销方面都优于基线方法。 我们的研究结果表明，GATN是一个适合现实世界强化学习应用（如自适应聊天机器人和机器人控制）的多功能框架。
URL
https://arxiv.org/abs/2507.03026
PDF
-
Natural language processing for African languages
2025-06-30 22:26:36David Ifeoluwa AdelaniAbstract
Recent advances in word embeddings and language models use large-scale, unlabelled data and self-supervised learning to boost NLP performance. Multilingual models, often trained on web-sourced data like Wikipedia, face challenges: few low-resource languages are included, their data is often noisy, and lack of labeled datasets makes it hard to evaluate performance outside high-resource languages like English. In this dissertation, we focus on languages spoken in Sub-Saharan Africa where all the indigenous languages in this region can be regarded as low-resourced in terms of the availability of labelled data for NLP tasks and unlabelled data found on the web. We analyse the noise in the publicly available corpora, and curate a high-quality corpus, demonstrating that the quality of semantic representations learned in word embeddings does not only depend on the amount of data but on the quality of pre-training data. We demonstrate empirically the limitations of word embeddings, and the opportunities the multilingual pre-trained language model (PLM) offers especially for languages unseen during pre-training and low-resource scenarios. We further study how to adapt and specialize multilingual PLMs to unseen African languages using a small amount of monolingual texts. To address the under-representation of the African languages in NLP research, we developed large scale human-annotated labelled datasets for 21 African languages in two impactful NLP tasks: named entity recognition and machine translation. We conduct an extensive empirical evaluation using state-of-the-art methods across supervised, weakly-supervised, and transfer learning settings.
Abstract (translated)
近期在词嵌入和语言模型方面的进展利用大规模无标注数据及自监督学习来提升自然语言处理（NLP）的性能。多语种模型通常基于如维基百科等网络来源的数据进行训练，但面临着几个挑战：少数低资源语言未被包括、其使用的数据往往质量不高且含有噪声，缺乏标记的数据集使得评估除英语这类高资源语言之外的语言表现变得困难。在这篇博士论文中，我们专注于撒哈拉以南非洲地区所使用的所有土著语言，在NLP任务的标注数据和网络上可用的无标注数据方面均被视为低资源语言。我们分析了公开语料库中的噪声，并整理出高质量的语料库，表明词嵌入学习到的语义表示不仅依赖于数据量还取决于预训练数据的质量。通过实证展示了词嵌入的局限性以及多语种预训练语言模型（PLM）在未见过的语言及低资源场景中提供的机会。我们进一步研究了如何使用少量单语文本来适应和专门化多语种PLM以应对非洲未知语言。为了弥补非洲语言在NLP研究中的代表性不足，我们为21个非洲语言开发了一整套大规模的人工注释标注数据集，在两个具有影响力的任务中：命名实体识别与机器翻译任务中使用这些数据集进行测试。我们采用最新的方法进行了广泛的实证评估，涵盖了监督学习、弱监督学习及迁移学习设置。
URL
https://arxiv.org/abs/2507.00297
PDF
-
Pruning by Block Benefit: Exploring the Properties of Vision Transformer Blocks during Domain Adaptation
2025-06-30 09:58:25Patrick Glandorf, Bodo RosenhahnAbstract
Vision Transformer have set new benchmarks in several tasks, but these models come with the lack of high computational costs which makes them impractical for resource limited hardware. Network pruning reduces the computational complexity by removing less important operations while maintaining performance. However, pruning a model on an unseen data domain, leads to a misevaluation of weight significance, resulting in suboptimal resource assignment. In this work, we find that task-sensitive layers initially fail to improve the feature representation on downstream tasks, leading to performance loss for early pruning decisions. To address this problem, we introduce Pruning by Block Benefit (P3B), a pruning method that utilizes the relative contribution on block level to globally assign parameter resources. P3B identifies low-impact components to reduce parameter allocation while preserving critical ones. Classical pruning mask optimization struggles to reactivate zero-mask-elements. In contrast, P3B sets a layerwise keep ratio based on global performance metrics, ensuring the reactivation of late-converging blocks. We show in extensive experiments that P3B is a state of the art pruning method with most noticeable gains in transfer learning tasks. Notably, P3B is able to conserve high performance, even in high sparsity regimes of 70% parameter reduction while only losing 0.64% in accuracy.
Abstract (translated)
视觉变压器已经在多个任务中设立了新的基准，但这些模型伴随着高昂的计算成本，这使得它们在资源有限的硬件上变得不切实际。网络剪枝通过移除不太重要的操作来减少计算复杂性，同时保持性能不变。然而，在未见过的数据域上进行剪枝会导致权重重要性的评估失误，从而导致次优的资源分配。在这项工作中，我们发现任务敏感层最初未能改善下游任务中的特征表示能力，导致早期剪枝决策下的性能下降。 为了解决这个问题，我们引入了一种名为块收益剪枝（Pruning by Block Benefit, P3B）的方法，该方法利用块级别的相对贡献来全局分配参数资源。P3B识别低影响组件以减少参数分配同时保留关键部分。传统的剪枝掩码优化很难重新激活被设为零的元素。相比之下，P3B根据整体性能指标设定逐层保持比例，确保晚期收敛模块的重新激活。 在广泛的实验中我们展示了P3B是目前最先进的剪枝方法，在迁移学习任务中表现出最显著的优势。值得注意的是，即使是在高达70%参数减少（高稀疏度）的情况下，P3B仍能够维持高性能，仅损失了0.64%的准确性。
URL
https://arxiv.org/abs/2506.23675
PDF
-
AI-Generated Lecture Slides for Improving Slide Element Detection and Retrieval
2025-06-30 08:11:31Suyash Maniyar, Vishvesh Trivedi, Ajoy Mondal, Anand Mishra, C. V. JawaharAbstract
Lecture slide element detection and retrieval are key problems in slide understanding. Training effective models for these tasks often depends on extensive manual annotation. However, annotating large volumes of lecture slides for supervised training is labor intensive and requires domain expertise. To address this, we propose a large language model (LLM)-guided synthetic lecture slide generation pipeline, SynLecSlideGen, which produces high-quality, coherent and realistic slides. We also create an evaluation benchmark, namely RealSlide by manually annotating 1,050 real lecture slides. To assess the utility of our synthetic slides, we perform few-shot transfer learning on real data using models pre-trained on them. Experimental results show that few-shot transfer learning with pretraining on synthetic slides significantly improves performance compared to training only on real data. This demonstrates that synthetic data can effectively compensate for limited labeled lecture slides. The code and resources of our work are publicly available on our project website: this https URL.
Abstract (translated)
讲义幻灯片元素检测和检索是理解幻灯片的关键问题。为了有效地训练用于这些任务的模型，通常需要大量的手动注释。然而，对大量讲座幻灯片进行监督训练的手动标注既耗时又需要领域专业知识。为了解决这个问题，我们提出了一种由大语言模型（LLM）引导的合成讲座幻灯片生成管道SynLecSlideGen，该管道可以产生高质量、连贯且真实的幻灯片。此外，我们创建了一个评估基准——RealSlide，通过手动注释1,050张真实讲座幻灯片来构建。为了评估我们的合成幻灯片的实用性，我们在真实数据上进行了少量样本迁移学习，并使用在合成幻灯片上预训练的模型进行测试。实验结果表明，与仅在真实数据上训练相比，在合成幻灯片上进行预训练后的少量样本迁移学习显著提高了性能。这证明了合成数据可以有效地弥补标记讲座幻灯片数量不足的问题。我们的工作代码和资源可以在项目网站上公开获取：[此URL]。
URL
https://arxiv.org/abs/2506.23605
PDF
-
FedRef: Communication-Efficient Bayesian Fine Tuning with Reference Model
2025-06-29 12:41:11Taehwan Yoon, Bongjun ChoiAbstract
Federated learning(FL) is used for distributed scenarios to train artificial intelligence(AI) models while ensuring users' privacy. In federated learning scenario, the server generally never knows about users' data. This type of concept makes the AI training process efficient in terms of data privacy. However, regarding model performance, federated AI models may not sufficiently satisfy AI users' expectations. Furthermore, AI users have a wide range of different needs. It is not easy to satisfy the whole users needs. These types of issues can be addressed through AI model optimization, fine-tuning, or personalization to achieve optimal model performance. To address model optimization challenges, we propose reference model-based federated learning for optimal fine-tuning, which overcomes catastrophic forgetting in each round. This method is derived from Bayesian parameter-efficient transfer learning, which includes an optimal proximal term and enables overcoming the catastrophic forgetting issue in each round by utilizing a reference model that incorporates previous model parameters. As a result, this method achieves both high model performance and low computing cost.
Abstract (translated)
联邦学习（FL）用于分布式场景下训练人工智能（AI）模型，同时确保用户的隐私安全。在联邦学习中，服务器通常不会接触到用户的数据。这种概念使得AI的训练过程在数据隐私方面变得更加高效。然而，在模型性能方面，联邦AI模型可能无法完全满足AI用户的需求和期望。此外，AI用户有着广泛的多样化需求，难以一一满足。这些问题可以通过AI模型优化、微调或个性化来实现最佳模型性能。 为了应对模型优化的挑战，我们提出了基于参考模型的联邦学习方法来进行最优微调，这种方法可以克服每一轮中的灾难性遗忘问题。该方法源自贝叶斯参数高效迁移学习，包括一个最优近似项，并通过使用包含先前模型参数的参考模型来解决每轮中出现的灾难性遗忘问题。因此，这种技术不仅实现了高模型性能，还降低了计算成本。
URL
https://arxiv.org/abs/2506.23210
PDF
-
Self-Supervised Contrastive Learning for Multi-Label Images
2025-06-29 09:29:37Jiale ChenAbstract
Self-supervised learning (SSL) has demonstrated its effectiveness in learning representations through comparison methods that align with human intuition. However, mainstream SSL methods heavily rely on high body datasets with single label, such as ImageNet, resulting in intolerable pre-training overhead. Besides, more general multi-label images are frequently overlooked in SSL, despite their potential for richer semantic information and broader applicability in downstream scenarios. Therefore, we tailor the mainstream SSL approach to guarantee excellent representation learning capabilities using fewer multi-label images. Firstly, we propose a block-wise augmentation module aimed at extracting additional potential positive view pairs from multi-label images. Subsequently, an image-aware contrastive loss is devised to establish connections between these views, thereby facilitating the extraction of semantically consistent representations. Comprehensive linear fine-tuning and transfer learning validate the competitiveness of our approach despite challenging sample quality and quantity.
Abstract (translated)
自监督学习（SSL）通过与人类直觉相符合的比较方法展示了其在表示学习方面的有效性。然而，主流的SSL方法严重依赖于具有单一标签的大规模数据集，如ImageNet，这导致了无法接受的预训练开销。此外，尽管多标签图像具有更丰富的语义信息和下游应用场景中的广泛适用性，但在SSL中却常常被忽视。因此，我们将主流的SSL方法进行调整，以利用较少的多标签图像确保卓越的表示学习能力。 首先，我们提出了一种基于块增强模块的方法，旨在从多标签图像中提取额外潜在的正向视图对。随后，设计了一种感知图像对比损失来建立这些视图之间的联系，从而促进语义一致性的表示抽取。通过全面的线性微调和迁移学习验证了我们的方法在具有挑战性的样本质量和数量条件下仍具备竞争力。
URL
https://arxiv.org/abs/2506.23156
PDF
-
An efficient plant disease detection using transfer learning approach
2025-06-28 13:47:27Bosubabu Sambana, Hillary Sunday Nnadi, Mohd Anas Wajid, Nwosu Ogochukwu Fidelia, Claudia Camacho-Zu\~niga, Henry Dozie Ajuzie, Edeh Michael OnyemaAbstract
Plant diseases pose significant challenges to farmers and the agricultural sector at large. However, early detection of plant diseases is crucial to mitigating their effects and preventing widespread damage, as outbreaks can severely impact the productivity and quality of crops. With advancements in technology, there are increasing opportunities for automating the monitoring and detection of disease outbreaks in plants. This study proposed a system designed to identify and monitor plant diseases using a transfer learning approach. Specifically, the study utilizes YOLOv7 and YOLOv8, two state-ofthe-art models in the field of object detection. By fine-tuning these models on a dataset of plant leaf images, the system is able to accurately detect the presence of Bacteria, Fungi and Viral diseases such as Powdery Mildew, Angular Leaf Spot, Early blight and Tomato mosaic virus. The model's performance was evaluated using several metrics, including mean Average Precision (mAP), F1-score, Precision, and Recall, yielding values of 91.05, 89.40, 91.22, and 87.66, respectively. The result demonstrates the superior effectiveness and efficiency of YOLOv8 compared to other object detection methods, highlighting its potential for use in modern agricultural practices. The approach provides a scalable, automated solution for early any plant disease detection, contributing to enhanced crop yield, reduced reliance on manual monitoring, and supporting sustainable agricultural practices.
Abstract (translated)
植物疾病对农民和整个农业部门构成了重大挑战。然而，早期检测植物病害对于减轻其影响和防止广泛损害至关重要，因为疫情爆发会严重影响作物的产量和质量。随着技术的进步，越来越多的机会出现可以自动化监测和识别植物病害的发生。这项研究提出了一种系统，旨在使用迁移学习方法来识别和监控植物疾病。具体来说，该研究利用了YOLOv7和YOLOv8，这两种是目标检测领域中的最新模型。通过在包含植物叶片图像的数据集上微调这些模型，该系统能够准确地检测细菌、真菌和病毒性疾病的出现，如白粉病、角斑病、早疫病以及番茄花叶病毒。模型的性能使用多个指标进行评估，包括平均精度（mAP）、F1分数、精确度和召回率，分别得到了91.05、89.40、91.22和87.66的成绩。 结果表明，YOLOv8在与其他目标检测方法相比时表现出优越的有效性和效率，突显了其在现代农业实践中的应用潜力。该方法为早期任何植物疾病的自动检测提供了一个可扩展的解决方案，有助于提高作物产量、减少对人工监控的依赖，并支持可持续的农业生产方式。
URL
https://arxiv.org/abs/2507.00070
PDF
-
ReasonBridge: Efficient Reasoning Transfer from Closed to Open-Source Language Models
2025-06-28 12:22:55Ziqi Zhong, Xunzhu TangAbstract
Recent advancements in Large Language Models (LLMs) have revealed a significant performance gap between closed-source and open-source models, particularly in tasks requiring complex reasoning and precise instruction following. This paper introduces ReasonBridge, a methodology that efficiently transfers reasoning capabilities from powerful closed-source to open-source models through a novel hierarchical knowledge distillation framework. We develop a tailored dataset Reason1K with only 1,000 carefully curated reasoning traces emphasizing difficulty, diversity, and quality. These traces are filtered from across multiple domains using a structured multi-criteria selection algorithm. Our transfer learning approach incorporates: (1) a hierarchical distillation process capturing both strategic abstraction and tactical implementation patterns, (2) a sparse reasoning-focused adapter architecture requiring only 0.3% additional trainable parameters, and (3) a test-time compute scaling mechanism using guided inference interventions. Comprehensive evaluations demonstrate that ReasonBridge improves reasoning capabilities in open-source models by up to 23% on benchmark tasks, significantly narrowing the gap with closed-source models. Notably, the enhanced Qwen2.5-14B outperforms Claude-Sonnet3.5 on MATH500 and matches its performance on competition-level AIME problems. Our methodology generalizes effectively across diverse reasoning domains and model architectures, establishing a sample-efficient approach to reasoning enhancement for instruction following.
Abstract (translated)
近期在大型语言模型（LLMs）领域的进展揭示了闭源模型和开源模型之间存在显著的性能差距，尤其是在需要复杂推理和精准指令执行的任务中。本文介绍了ReasonBridge，一种通过新颖的分层知识蒸馏框架高效地将强大闭源模型的推理能力转移到开源模型的方法论。我们开发了一个特制的数据集Reason1K，仅包含精心策划的1,000个推理痕迹，这些痕迹强调难度、多样性和质量，并且是从多个领域中筛选出来的，使用的是结构化多准则选择算法。我们的迁移学习方法包括： 1. 一种分层蒸馏过程，捕捉策略抽象和战术实施模式。 2. 一个专注于稀疏推理的适配器架构，只需要增加0.3%的可训练参数。 3. 在测试时采用计算缩放机制，使用引导式推断干预。 全面评估表明，ReasonBridge在基准任务上将开源模型的推理能力提高了高达23%，显著缩小了与闭源模型之间的差距。特别值得注意的是，经过增强的Qwen2.5-14B在MATH500和竞赛级别的AIME问题上表现优于Claude-Sonnet3.5，并且在后者的问题上与其性能相当。 我们的方法论能够在广泛的推理领域和不同架构的模型中有效推广，建立了一种高效的推理增强方法来改进指令遵循。
URL
https://arxiv.org/abs/2506.22865
PDF
-
Multi-View Contrastive Learning for Robust Domain Adaptation in Medical Time Series Analysis
2025-06-27 17:06:16YongKyung Oh, Alex BuiAbstract
Adapting machine learning models to medical time series across different domains remains a challenge due to complex temporal dependencies and dynamic distribution shifts. Current approaches often focus on isolated feature representations, limiting their ability to fully capture the intricate temporal dynamics necessary for robust domain adaptation. In this work, we propose a novel framework leveraging multi-view contrastive learning to integrate temporal patterns, derivative-based dynamics, and frequency-domain features. Our method employs independent encoders and a hierarchical fusion mechanism to learn feature-invariant representations that are transferable across domains while preserving temporal coherence. Extensive experiments on diverse medical datasets, including electroencephalogram (EEG), electrocardiogram (ECG), and electromyography (EMG) demonstrate that our approach significantly outperforms state-of-the-art methods in transfer learning tasks. By advancing the robustness and generalizability of machine learning models, our framework offers a practical pathway for deploying reliable AI systems in diverse healthcare settings.
Abstract (translated)
将机器学习模型适应不同领域中的医疗时间序列数据仍然是一个挑战，原因在于复杂的时间依赖性和动态分布变化。目前的方法往往集中在孤立的特征表示上，这限制了它们捕捉稳健域适应所必需的复杂时间动态的能力。在本工作中，我们提出了一种利用多视角对比学习的新框架，以整合时间模式、基于导数的动力学和频域特征。我们的方法采用独立编码器和分层融合机制来学习跨领域可迁移且保持时间一致性的不变性表示。我们在包括脑电图（EEG）、心电图（ECG）和肌电图（EMG）在内的多样化医疗数据集上进行了广泛的实验，结果表明我们的方法在迁移学习任务中显著优于最先进的方法。通过增强机器学习模型的鲁棒性和泛化能力，我们的框架为在各种医疗环境中部署可靠的AI系统提供了实用路径。
URL
https://arxiv.org/abs/2506.22393
PDF
-
Visual Content Detection in Educational Videos with Transfer Learning and Dataset Enrichment
2025-06-27 04:43:05Dipayan Biswas, Shishir Shah, Jaspal SubhlokAbstract
Video is transforming education with online courses and recorded lectures supplementing and replacing classroom teaching. Recent research has focused on enhancing information retrieval for video lectures with advanced navigation, searchability, summarization, as well as question answering chatbots. Visual elements like tables, charts, and illustrations are central to comprehension, retention, and data presentation in lecture videos, yet their full potential for improving access to video content remains underutilized. A major factor is that accurate automatic detection of visual elements in a lecture video is challenging; reasons include i) most visual elements, such as charts, graphs, tables, and illustrations, are artificially created and lack any standard structure, and ii) coherent visual objects may lack clear boundaries and may be composed of connected text and visual components. Despite advancements in deep learning based object detection, current models do not yield satisfactory performance due to the unique nature of visual content in lectures and scarcity of annotated datasets. This paper reports on a transfer learning approach for detecting visual elements in lecture video frames. A suite of state of the art object detection models were evaluated for their performance on lecture video datasets. YOLO emerged as the most promising model for this task. Subsequently YOLO was optimized for lecture video object detection with training on multiple benchmark datasets and deploying a semi-supervised auto labeling strategy. Results evaluate the success of this approach, also in developing a general solution to the problem of object detection in lecture videos. Paper contributions include a publicly released benchmark of annotated lecture video frames, along with the source code to facilitate future research.
Abstract (translated)
视频正在通过在线课程和录制的讲座改变教育，这些内容正补充甚至取代传统的课堂教学。近期的研究重点在于利用高级导航、搜索能力、摘要生成以及问答聊天机器人来增强对视频讲座的信息检索功能。视觉元素如表格、图表和插图在讲义视频的理解、记忆以及数据展示中至关重要，但它们在整个视频内容的访问改善方面的潜力尚未得到充分利用。一个主要因素是准确自动检测讲义视频中的视觉元素具有挑战性；原因包括：一）大多数视觉元素（例如图表、图形、表格和插图）都是人工创建且缺乏任何标准结构；二）连贯的视觉对象可能没有明确边界，并由连接的文本和视觉组件组成。尽管基于深度学习的对象检测技术已经取得了进展，但由于讲义视频中视觉内容的独特性和标注数据集的稀缺性，现有模型的表现并不令人满意。 本文报道了一种用于识别讲座视频帧中的视觉元素的迁移学习方法。评估了一系列最先进的对象检测模型在讲座视频数据集上的表现，YOLO（You Only Look Once）被认为是最有前途的模型。随后，通过对多个基准数据集进行训练并部署半监督自动标注策略对YOLO进行了优化以适应讲座视频的对象检测需求。实验结果评估了这种方法的成功，并为解决讲义视频中的对象检测问题开发了一种通用解决方案提供了支持。 本文贡献包括公开发布了一个包含注释的讲义视频帧的基准测试，以及源代码，以促进未来的相关研究。
URL
https://arxiv.org/abs/2506.21903
PDF
