Transfer_Learning
-
Signals vs. Videos: Advancing Motion Intention Recognition for Human-Robot Collaboration in Construction
2025-08-25 14:58:07Charan Gajjala Chenchu, Kinam Kim, Gao Lu, Zia Ud DinAbstract
Human-robot collaboration (HRC) in the construction industry depends on precise and prompt recognition of human motion intentions and actions by robots to maximize safety and workflow efficiency. There is a research gap in comparing data modalities, specifically signals and videos, for motion intention recognition. To address this, the study leverages deep learning to assess two different modalities in recognizing workers' motion intention at the early stage of movement in drywall installation tasks. The Convolutional Neural Network - Long Short-Term Memory (CNN-LSTM) model utilizing surface electromyography (sEMG) data achieved an accuracy of around 87% with an average time of 0.04 seconds to perform prediction on a sample input. Meanwhile, the pre-trained Video Swin Transformer combined with transfer learning harnessed video sequences as input to recognize motion intention and attained an accuracy of 94% but with a longer average time of 0.15 seconds for a similar prediction. This study emphasizes the unique strengths and trade-offs of both data formats, directing their systematic deployments to enhance HRC in real-world construction projects.
Abstract (translated)
人类与机器人协作（HRC）在建筑行业中依赖于机器人对人类运动意图和动作的精确且及时识别，以最大化安全性和工作流程效率。目前的研究中存在一个缺口，即比较不同数据模态（特别是信号和视频）对于运动意图识别的效果。为了解决这一问题，研究利用深度学习技术评估了两种不同的模态在干墙安装任务早期阶段工人运动意图识别中的表现。 使用表面肌电信号(sEMG)数据的卷积神经网络-长短时记忆(CNN-LSTM)模型实现了约87%的准确率，并且平均只需要0.04秒的时间来预测单一样本输入。另一方面，预训练的视频Swin Transformer结合了迁移学习方法，利用视频序列作为输入识别运动意图，达到了94%的准确率，但其平均处理时间较长，为0.15秒。 这项研究强调了两种数据格式的独特优势和权衡，并指出了它们在实际建筑项目中系统部署的方向，以进一步提升人类与机器人协作的效果。
URL
https://arxiv.org/abs/2509.07990
PDF
-
AMELIA: A Family of Multi-task End-to-end Language Models for Argumentation
2025-08-25 11:51:39Henri Savigny, Bruno YunAbstract
Argument mining is a subfield of argumentation that aims to automatically extract argumentative structures and their relations from natural language texts. This paper investigates how a single large language model can be leveraged to perform one or several argument mining tasks. Our contributions are two-fold. First, we construct a multi-task dataset by surveying and converting 19 well-known argument mining datasets from the literature into a unified format. Second, we explore various training strategies using Meta AI's Llama-3.1-8B-Instruct model: (1) fine-tuning on individual tasks, (2) fine-tuning jointly on multiple tasks, and (3) merging models fine-tuned separately on individual tasks. Our experiments show that task-specific fine-tuning significantly improves individual performance across all tasks. Moreover, multi-task fine-tuning maintains strong performance without degradation, suggesting effective transfer learning across related tasks. Finally, we demonstrate that model merging offers a viable compromise: it yields competitive performance while mitigating the computational costs associated with full multi-task fine-tuning.
Abstract (translated)
论证挖掘是论辩学的一个子领域，旨在自动从自然语言文本中提取论证结构及其关系。本文探讨了如何利用单一的大规模语言模型来执行一项或多项论证挖掘任务。我们的贡献主要有两个方面：首先，我们通过调查并转换文献中的19个知名论证挖掘数据集，构建了一个多任务数据集，并将其统一为一种格式。其次，我们探索了使用Meta AI的Llama-3.1-8B-Instruct模型的各种训练策略：(1)在单一任务上进行微调；(2)在多个任务上联合进行微调；以及(3)将单独针对每个任务进行微调后的模型合并。我们的实验表明，特定任务的微调显著提升了所有任务上的个体性能。此外，多任务微调能够保持强劲的表现而不会出现退化现象，这表明跨相关任务的有效迁移学习。最后，我们证明了模型合并提供了一个可行的折中方案：它在降低完整多任务微调相关的计算成本的同时还能达到竞争性的表现。
URL
https://arxiv.org/abs/2508.17926
PDF
-
Sketchpose: Learning to Segment Cells with Partial Annotations
2025-08-25 08:43:32Cl\'ement Cazorla, Nathana\"el Munier, Renaud Morin, Pierre WeissAbstract
The most popular networks used for cell segmentation (e.g. Cellpose, Stardist, HoverNet,...) rely on a prediction of a distance map. It yields unprecedented accuracy but hinges on fully annotated datasets. This is a serious limitation to generate training sets and perform transfer learning. In this paper, we propose a method that still relies on the distance map and handles partially annotated objects. We evaluate the performance of the proposed approach in the contexts of frugal learning, transfer learning and regular learning on regular databases. Our experiments show that it can lead to substantial savings in time and resources without sacrificing segmentation quality. The proposed algorithm is embedded in a user-friendly Napari plugin.
Abstract (translated)
用于细胞分割的最流行的网络（例如 Cellpose、Stardist、HoverNet 等）依赖于距离图预测。这种方法实现了前所未有的准确性，但需要完全注释的数据集。这在生成训练集和进行迁移学习方面是一个严重的限制。在这篇论文中，我们提出了一种方法，该方法仍然基于距离图，并能够处理部分标注的对象。我们在节俭学习、迁移学习和常规数据库上的常规学习环境中评估了所提方法的性能。实验结果显示，在不牺牲分割质量的情况下，它可以在时间和资源上实现显著节省。提出的算法被嵌入到一个用户友好的 Napari 插件中。
URL
https://arxiv.org/abs/2508.17798
PDF
-
Towards Optimal Convolutional Transfer Learning Architectures for Breast Lesion Classification and ACL Tear Detection
2025-08-25 00:33:43Daniel Frees, Moritz Bolling, Aditri BhagirathAbstract
Modern computer vision models have proven to be highly useful for medical imaging classification and segmentation tasks, but the scarcity of medical imaging data often limits the efficacy of models trained from scratch. Transfer learning has emerged as a pivotal solution to this, enabling the fine-tuning of high-performance models on small data. Mei et al. (2022) found that pre-training CNNs on a large dataset of radiologist-labeled images (RadImageNet) enhanced model performance on downstream tasks compared to ImageNet pretraining. The present work extends Mei et al. (2022) by conducting a comprehensive investigation to determine optimal CNN architectures for breast lesion malignancy detection and ACL tear detection, as well as performing statistical analysis to compare the effect of RadImageNet and ImageNet pre-training on downstream model performance. Our findings suggest that 1-dimensional convolutional classifiers with skip connections, ResNet50 pre-trained backbones, and partial backbone unfreezing yields optimal downstream medical classification performance. Our best models achieve AUCs of 0.9969 for ACL tear detection and 0.9641 for breast nodule malignancy detection, competitive with the results reported by Mei et al. (2022) and surpassing other previous works. We do not find evidence confirming RadImageNet pre-training to provide superior downstream performance for ACL tear and breast lesion classification tasks.
Abstract (translated)
现代计算机视觉模型已被证明在医学影像分类和分割任务中非常有用，但医疗影像数据的稀缺性往往限制了从头开始训练的模型的效果。迁移学习已经成为解决这一问题的关键方案，它使得在小规模数据集上对高性能模型进行微调成为可能。Mei等人（2022年）发现，在一个大规模的放射科医生标注图像的数据集（RadImageNet）上预训练卷积神经网络（CNN），相比使用ImageNet预训练，可以提高下游任务的表现。 本研究在Mei等人的基础上进行了扩展，通过全面调查来确定最优的CNN架构用于乳腺病变恶性检测和前交叉韧带撕裂检测，并进行统计分析以比较RadImageNet与ImageNet预训练对下游模型性能的影响。我们的发现表明，具备跳跃连接的一维卷积分类器、使用ResNet50预训练骨干网络以及部分解冻的骨干网络能够为下游医学分类任务提供最佳性能。 我们最好的模型在前交叉韧带撕裂检测中达到了AUC值为0.9969，在乳腺结节恶性检测中的AUC值为0.9641，与Mei等人（2022年）报告的结果相当，并且超越了其他先前的工作。然而，我们没有找到证据支持RadImageNet预训练在前交叉韧带撕裂和乳腺病变分类任务中提供比ImageNet预训练更好的下游性能。
URL
https://arxiv.org/abs/2508.17567
PDF
-
Deep Learning-Assisted Detection of Sarcopenia in Cross-Sectional Computed Tomography Imaging
2025-08-24 09:53:56Manish Bhardwaj, Huizhi Liang, Ashwin Sivaharan, Sandip Nandhra, Vaclav Snasel, Tamer El-Sayed, Varun OjhaAbstract
Sarcopenia is a progressive loss of muscle mass and function linked to poor surgical outcomes such as prolonged hospital stays, impaired mobility, and increased mortality. Although it can be assessed through cross-sectional imaging by measuring skeletal muscle area (SMA), the process is time-consuming and adds to clinical workloads, limiting timely detection and management; however, this process could become more efficient and scalable with the assistance of artificial intelligence applications. This paper presents high-quality three-dimensional cross-sectional computed tomography (CT) images of patients with sarcopenia collected at the Freeman Hospital, Newcastle upon Tyne Hospitals NHS Foundation Trust. Expert clinicians manually annotated the SMA at the third lumbar vertebra, generating precise segmentation masks. We develop deep-learning models to measure SMA in CT images and automate this task. Our methodology employed transfer learning and self-supervised learning approaches using labelled and unlabeled CT scan datasets. While we developed qualitative assessment models for detecting sarcopenia, we observed that the quantitative assessment of SMA is more precise and informative. This approach also mitigates the issue of class imbalance and limited data availability. Our model predicted the SMA, on average, with an error of +-3 percentage points against the manually measured SMA. The average dice similarity coefficient of the predicted masks was 93%. Our results, therefore, show a pathway to full automation of sarcopenia assessment and detection.
Abstract (translated)
肌少症是一种进行性的肌肉质量和功能丧失，与较差的手术结果相关，如住院时间延长、行动能力受损和死亡率增加。虽然可以通过测量骨骼肌面积（SMA）来利用横断面成像评估肌少症，但这一过程耗时且增加了临床工作量，从而限制了及时发现和管理；然而，在人工智能应用的帮助下，该流程可以变得更高效和可扩展。 本文介绍了由纽卡斯尔泰恩河畔弗里曼医院收集的患有肌少症患者高质量三维横断面CT图像。专家医生手动在第三腰椎水平标注了SMA，生成了精确的分割掩模。我们开发了一种深度学习模型来测量CT图像中的骨骼肌面积（SMA）并自动化这一任务。我们的方法采用了迁移学习和自监督学习方法，使用标记和未标记的CT扫描数据集。尽管我们为检测肌少症开发了定性评估模型，但我们发现定量评估SMA更为准确且信息量更大。这种方法还缓解了类不平衡和数据可用性有限的问题。 我们的模型预测的骨骼肌面积（SMA）平均误差与手动测量的结果相比约为正负3个百分点。预测掩模的平均Dice相似系数为93%。因此，我们的结果表明了一条实现肌少症评估和检测完全自动化的路径。
URL
https://arxiv.org/abs/2508.17275
PDF
-
SugarcaneShuffleNet: A Very Fast, Lightweight Convolutional Neural Network for Diagnosis of 15 Sugarcane Leaf Diseases
2025-08-23 18:39:25Shifat E. Arman, Hasan Muhammad Abdullah, Syed Nazmus Sakib, RM Saiem, Shamima Nasrin Asha, Md Mehedi Hasan, Shahrear Bin Amin, S M Mahin AbrarAbstract
Despite progress in AI-based plant diagnostics, sugarcane farmers in low-resource regions remain vulnerable to leaf diseases due to the lack of scalable, efficient, and interpretable tools. Many deep learning models fail to generalize under real-world conditions and require substantial computational resources, limiting their use in resource-constrained regions. In this paper, we present SugarcaneLD-BD, a curated dataset for sugarcane leaf-disease classification; SugarcaneShuffleNet, an optimized lightweight model for rapid on-device diagnosis; and SugarcaneAI, a Progressive Web Application for field deployment. SugarcaneLD-BD contains 638 curated images across five classes, including four major sugarcane diseases, collected in Bangladesh under diverse field conditions and verified by expert pathologists. To enhance diversity, we combined SugarcaneLD-BD with two additional datasets, yielding a larger and more representative corpus. Our optimized model, SugarcaneShuffleNet, offers the best trade-off between speed and accuracy for real-time, on-device diagnosis. This 9.26 MB model achieved 98.02% accuracy, an F1-score of 0.98, and an average inference time of 4.14 ms per image. For comparison, we fine-tuned five other lightweight convolutional neural networks: MnasNet, EdgeNeXt, EfficientNet-Lite, MobileNet, and SqueezeNet via transfer learning and Bayesian optimization. MnasNet and EdgeNeXt achieved comparable accuracy to SugarcaneShuffleNet, but required significantly more parameters, memory, and computation, limiting their suitability for low-resource deployment. We integrate SugarcaneShuffleNet into SugarcaneAI, delivering Grad-CAM-based explanations in the field. Together, these contributions offer a diverse benchmark, efficient models for low-resource environments, and a practical tool for sugarcane disease classification. It spans varied lighting, backgrounds and devices used on-farm
Abstract (translated)
尽管基于人工智能的植物诊断技术取得了进展，但资源匮乏地区甘蔗种植者仍易受到叶片疾病的威胁，因为缺乏可扩展、高效且易于解释的工具。许多深度学习模型在实际条件下的泛化能力较差，并需要大量计算资源，这限制了它们在资源受限地区的应用。 本文介绍了SugarcaneLD-BD（用于甘蔗叶病分类的数据集）、SugarcaneShuffleNet（一种优化后的轻量级模型，可在设备上快速进行诊断）和SugarcaneAI（一款适用于现场部署的渐进式Web应用程序）。SugarcaneLD-BD包含638张经过精心挑选的图像，涵盖五个类别，包括四种主要的甘蔗疾病，在孟加拉国的各种田间条件下收集，并由专家病理学家进行了验证。为了增强多样性，我们结合了两个额外的数据集，形成了一个更大、更具代表性的数据集合。 我们的优化模型SugarcaneShuffleNet在实时设备上诊断的速度和准确性之间提供了最佳的权衡。这款9.26MB大小的模型达到了98.02%的准确率，F1评分为0.98，并且每张图像的平均推理时间为4.14毫秒。作为比较，我们通过迁移学习和贝叶斯优化对另外五种轻量级卷积神经网络（MnasNet、EdgeNeXt、EfficientNet-Lite、MobileNet 和 SqueezeNet）进行了微调。尽管MnasNet和EdgeNeXt在准确率方面与SugarcaneShuffleNet相当，但它们需要更多的参数、内存及计算资源，限制了其在低资源配置环境中的适用性。 我们将SugarcaneShuffleNet集成到SugarcaneAI中，在现场提供了基于Grad-CAM的解释。这些贡献共同提供了一个多样化的基准测试、适用于低资源环境的有效模型以及一个实用的甘蔗疾病分类工具。它涵盖了各种照明条件、背景和田间使用的设备。
URL
https://arxiv.org/abs/2508.17107
PDF
-
Transfer Learning via Lexical Relatedness: A Sarcasm and Hate Speech Case Study
2025-08-22 17:23:08Angelly Cabrera, Linus Lei, Antonio OrtegaAbstract
Detecting hate speech in non-direct forms, such as irony, sarcasm, and innuendos, remains a persistent challenge for social networks. Although sarcasm and hate speech are regarded as distinct expressions, our work explores whether integrating sarcasm as a pre-training step improves implicit hate speech detection and, by extension, explicit hate speech detection. Incorporating samples from ETHOS, Sarcasm on Reddit, and Implicit Hate Corpus, we devised two training strategies to compare the effectiveness of sarcasm pre-training on a CNN+LSTM and BERT+BiLSTM model. The first strategy is a single-step training approach, where a model trained only on sarcasm is then tested on hate speech. The second strategy uses sequential transfer learning to fine-tune models for sarcasm, implicit hate, and explicit hate. Our results show that sarcasm pre-training improved the BERT+BiLSTM's recall by 9.7%, AUC by 7.8%, and F1-score by 6% on ETHOS. On the Implicit Hate Corpus, precision increased by 7.8% when tested only on implicit samples. By incorporating sarcasm into the training process, we show that models can more effectively detect both implicit and explicit hate.
Abstract (translated)
在社交媒体平台上，识别非直接形式的仇恨言论（如讽刺、讽刺性言论和暗讽）仍然是一个持续的挑战。尽管讽刺和仇恨言论被视为不同的表达方式，但我们的研究探讨了将讽刺作为预训练步骤是否能改善隐含仇恨言论的检测，并进而提升明确仇恨言论的检测效果。我们利用ETHOS数据集、Reddit上的讽刺语料库以及隐含仇恨语料库设计了两种训练策略，以比较在CNN+LSTM和BERT+BiLSTM模型中使用讽刺预训练的有效性。 第一种策略采用单步训练方法，在此方法下仅用讽刺样本训练的模型随后会被用于检测仇恨言论。第二种策略则利用顺序迁移学习技术，先对模型进行讽刺语料库的微调，再依次针对隐含仇恨和明确仇恨进行进一步调整。我们的实验结果表明，通过将讽刺纳入预训练过程，BERT+BiLSTM模型在ETHOS数据集上的召回率提高了9.7%，AUC（曲线下面积）提升了7.8%，F1分数增加了6%。 而在处理隐含仇恨语料库的测试样本时，仅使用隐含样本进行检测的精度则提升了7.8%。这表明将讽刺纳入训练过程可以有效增强模型在识别隐含和明确仇恨言论方面的能力。
URL
https://arxiv.org/abs/2508.16555
PDF
-
STA-GANN: A Valid and Generalizable Spatio-Temporal Kriging Approach
2025-08-22 07:33:12Yujie Li, Zezhi Shao, Chengqing Yu, Tangwen Qian, Zhao Zhang, Yifan Du, Shaoming He, Fei Wang, Yongjun XuAbstract
Spatio-temporal tasks often encounter incomplete data arising from missing or inaccessible sensors, making spatio-temporal kriging crucial for inferring the completely missing temporal information. However, current models struggle with ensuring the validity and generalizability of inferred spatio-temporal patterns, especially in capturing dynamic spatial dependencies and temporal shifts, and optimizing the generalizability of unknown sensors. To overcome these limitations, we propose Spatio-Temporal Aware Graph Adversarial Neural Network (STA-GANN), a novel GNN-based kriging framework that improves spatio-temporal pattern validity and generalization. STA-GANN integrates (i) Decoupled Phase Module that senses and adjusts for timestamp shifts. (ii) Dynamic Data-Driven Metadata Graph Modeling to update spatial relationships using temporal data and metadata; (iii) An adversarial transfer learning strategy to ensure generalizability. Extensive validation across nine datasets from four fields and theoretical evidence both demonstrate the superior performance of STA-GANN.
Abstract (translated)
空间时间任务常常遇到由于传感器缺失或无法访问而导致的不完整数据，使得时空克里金法（一种用于推断完全丢失的时间信息的技术）变得至关重要。然而，目前的模型在确保所推测的空间时间模式的有效性和泛化能力方面存在困难，尤其是在捕捉动态空间依赖关系和时间变化以及优化未知传感器的泛化性方面。为了解决这些限制，我们提出了一种基于图神经网络（GNN）的新框架——时空感知图对抗神经网络（STA-GANN），该框架能够改进时空模式的有效性和泛化能力。 STA-GANN整合了以下三个关键部分： (i) 解耦相位模块：可以检测并调整时间戳的偏移； (ii) 动态数据驱动元数据图建模：利用时间数据和元数据来更新空间关系； (iii) 对抗性迁移学习策略，以确保模型具有良好的泛化能力。 通过在四个领域的九个不同数据集上的广泛验证以及理论证据都证明了STA-GANN的优越性能。
URL
https://arxiv.org/abs/2508.16161
PDF
-
Transfer Learning for Minimum Operating Voltage Prediction in Advanced Technology Nodes: Leveraging Legacy Data and Silicon Odometer Sensing
2025-08-21 23:13:55Yuxuan Yin, Rebecca Chen, Boxun Xu, Chen He, Peng LiAbstract
Accurate prediction of chip performance is critical for ensuring energy efficiency and reliability in semiconductor manufacturing. However, developing minimum operating voltage ($V_{min}$) prediction models at advanced technology nodes is challenging due to limited training data and the complex relationship between process variations and $V_{min}$. To address these issues, we propose a novel transfer learning framework that leverages abundant legacy data from the 16nm technology node to enable accurate $V_{min}$ prediction at the advanced 5nm node. A key innovation of our approach is the integration of input features derived from on-chip silicon odometer sensor data, which provide fine-grained characterization of localized process variations -- an essential factor at the 5nm node -- resulting in significantly improved prediction accuracy.
Abstract (translated)
准确预测芯片性能对于确保半导体制造中的能源效率和可靠性至关重要。然而，在先进工艺节点上开发最小操作电压($V_{min}$)的预测模型面临挑战，原因是可用训练数据有限以及加工变化与$V_{min}$之间复杂的关系。为解决这些问题，我们提出了一种新的迁移学习框架，利用来自16纳米技术节点的大量历史数据来实现5纳米先进节点上的准确$V_{min}$预测。我们的方法的一个关键创新点是整合了从芯片硅里程表传感器数据中提取的输入特征，这些特征能够精细地描述局部加工变化——这是在5纳米节点上至关重要的因素，从而显著提高了预测准确性。
URL
https://arxiv.org/abs/2509.00035
PDF
-
Futurity as Infrastructure: A Techno-Philosophical Interpretation of the AI Lifecycle
Abstract
This paper argues that a techno-philosophical reading of the EU AI Act provides insight into the long-term dynamics of data in AI systems, specifically, how the lifecycle from ingestion to deployment generates recursive value chains that challenge existing frameworks for Responsible AI. We introduce a conceptual tool to frame the AI pipeline, spanning data, training regimes, architectures, feature stores, and transfer learning. Using cross-disciplinary methods, we develop a technically grounded and philosophically coherent analysis of regulatory blind spots. Our central claim is that what remains absent from policymaking is an account of the dynamic of becoming that underpins both the technical operation and economic logic of AI. To address this, we advance a formal reading of AI inspired by Simondonian philosophy of technology, reworking his concept of individuation to model the AI lifecycle, including the pre-individual milieu, individuation, and individuated AI. To translate these ideas, we introduce futurity: the self-reinforcing lifecycle of AI, where more data enhances performance, deepens personalisation, and expands application domains. Futurity highlights the recursively generative, non-rivalrous nature of data, underpinned by infrastructures like feature stores that enable feedback, adaptation, and temporal recursion. Our intervention foregrounds escalating power asymmetries, particularly the tech oligarchy whose infrastructures of capture, training, and deployment concentrate value and decision-making. We argue that effective regulation must address these infrastructural and temporal dynamics, and propose measures including lifecycle audits, temporal traceability, feedback accountability, recursion transparency, and a right to contest recursive reuse.
Abstract (translated)
这篇论文主张，从技术哲学的角度解读欧盟《人工智能法案》可以为理解AI系统中数据的长期动态提供洞见，特别是数据从摄取到部署的生命循环如何生成递归价值链，从而挑战现有的负责任AI框架。我们引入了一种概念工具来构建涵盖数据、训练制度、架构、特征存储和迁移学习的人工智能管道模型。通过跨学科的方法，我们在技术实证的基础上进行哲学上的分析，揭示监管的盲点所在。我们的核心观点是，在政策制定中缺乏对支撑人工智能技术和经济逻辑的发展过程的理解。 为了解决这个问题，我们借鉴吉伦·菲利蒙（Simondonian）的技术哲学思想提出了一个形式化的解读视角，并重新定义了他的个体化概念以建模AI生命周期，包括前个人环境、个体化以及个体化的人工智能。为了表达这些观念，我们将引入“未来性”：即人工智能自我强化的生命循环，在这种循环中，更多的数据可以提高性能，加深个性化并扩展应用场景。“未来性”突出了基于特征存储等基础设施支持的反馈机制和时间递归下所体现的数据的递归生成、非竞争性质。 我们的研究强调了不断加剧的权力不对称问题，尤其是技术寡头通过捕获、训练及部署的基础设施集中价值与决策权。我们主张有效的监管必须解决这些基础设施和时间动态的问题，并提出包括生命周期审计、时序可追溯性、反馈问责制、递归透明度以及针对循环再用的质疑权利在内的措施。
URL
https://arxiv.org/abs/2508.15680
PDF
-
Transfer learning optimization based on evolutionary selective fine tuning
2025-08-21 08:51:43Jacinto Colan, Ana Davila, Yasuhisa HasegawaAbstract
Deep learning has shown substantial progress in image analysis. However, the computational demands of large, fully trained models remain a consideration. Transfer learning offers a strategy for adapting pre-trained models to new tasks. Traditional fine-tuning often involves updating all model parameters, which can potentially lead to overfitting and higher computational costs. This paper introduces BioTune, an evolutionary adaptive fine-tuning technique that selectively fine-tunes layers to enhance transfer learning efficiency. BioTune employs an evolutionary algorithm to identify a focused set of layers for fine-tuning, aiming to optimize model performance on a given target task. Evaluation across nine image classification datasets from various domains indicates that BioTune achieves competitive or improved accuracy and efficiency compared to existing fine-tuning methods such as AutoRGN and LoRA. By concentrating the fine-tuning process on a subset of relevant layers, BioTune reduces the number of trainable parameters, potentially leading to decreased computational cost and facilitating more efficient transfer learning across diverse data characteristics and distributions.
Abstract (translated)
深度学习在图像分析方面取得了显著进展。然而，大型全训练模型的计算需求仍然需要考虑。迁移学习为将预训练模型适应新任务提供了一种策略。传统的微调方法通常涉及更新所有模型参数，这可能会导致过拟合和更高的计算成本。本文介绍了BioTune，这是一种基于进化算法的自适应微调技术，能够选择性地对特定层进行微调以增强迁移学习效率。BioTune通过使用进化算法来识别一组聚焦的层来进行微调，旨在优化模型在给定目标任务上的性能。跨九个来自不同领域的图像分类数据集的评估结果表明，与现有的AutoRGN和LoRA等微调方法相比，BioTune在精度和效率方面达到了竞争或改进的效果。通过将微调过程集中在相关层的一个子集中，BioTune减少了可训练参数的数量，这可能会降低计算成本，并促进跨不同数据特征和分布的更高效的迁移学习。
URL
https://arxiv.org/abs/2508.15367
PDF
-
DeepEmoNet: Building Machine Learning Models for Automatic Emotion Recognition in Human Speeches
Abstract
Speech emotion recognition (SER) has been a challenging problem in spoken language processing research, because it is unclear how human emotions are connected to various components of sounds such as pitch, loudness, and energy. This paper aims to tackle this problem using machine learning. Particularly, we built several machine learning models using SVMs, LTSMs, and CNNs to classify emotions in human speeches. In addition, by leveraging transfer learning and data augmentation, we efficiently trained our models to attain decent performances on a relatively small dataset. Our best model was a ResNet34 network, which achieved an accuracy of $66.7\%$ and an F1 score of $0.631$.
Abstract (translated)
语音情感识别（SER）一直是口语语言处理研究中的一个难题，因为人类情绪与声音的各个组成部分（如音高、响度和能量）之间的联系尚不清楚。本文旨在通过机器学习方法解决这一问题。具体而言，我们构建了若干使用支持向量机(SVMs)、长短期记忆网络(LTSMs) 和卷积神经网络(CNNs) 的机器学习模型来分类人类语音中的情感。此外，通过利用迁移学习和数据增强技术，我们在相对较小的数据集上高效地训练了我们的模型，并取得了不错的性能结果。我们最佳的模型是一个ResNet34 网络，在测试中达到了66.7%的准确率和0.631的F1分数。
URL
https://arxiv.org/abs/2509.00025
PDF
-
A Systematic Study of Deep Learning Models and xAI Methods for Region-of-Interest Detection in MRI Scans
2025-08-19 17:42:45Justin Yiu, Kushank Arora, Daniel Steinberg, Rohit GhiyaAbstract
Magnetic Resonance Imaging (MRI) is an essential diagnostic tool for assessing knee injuries. However, manual interpretation of MRI slices remains time-consuming and prone to inter-observer variability. This study presents a systematic evaluation of various deep learning architectures combined with explainable AI (xAI) techniques for automated region of interest (ROI) detection in knee MRI scans. We investigate both supervised and self-supervised approaches, including ResNet50, InceptionV3, Vision Transformers (ViT), and multiple U-Net variants augmented with multi-layer perceptron (MLP) classifiers. To enhance interpretability and clinical relevance, we integrate xAI methods such as Grad-CAM and Saliency Maps. Model performance is assessed using AUC for classification and PSNR/SSIM for reconstruction quality, along with qualitative ROI visualizations. Our results demonstrate that ResNet50 consistently excels in classification and ROI identification, outperforming transformer-based models under the constraints of the MRNet dataset. While hybrid U-Net + MLP approaches show potential for leveraging spatial features in reconstruction and interpretability, their classification performance remains lower. Grad-CAM consistently provided the most clinically meaningful explanations across architectures. Overall, CNN-based transfer learning emerges as the most effective approach for this dataset, while future work with larger-scale pretraining may better unlock the potential of transformer models.
Abstract (translated)
磁共振成像（MRI）是评估膝关节损伤的重要诊断工具。然而，手动解释MRI切片既耗时又容易出现观察者之间的差异。本研究系统地评估了各种深度学习架构与可解释人工智能(xAI)技术的结合，用于自动检测膝部MRI扫描中的感兴趣区域(ROI)。我们探讨了监督和自监督方法，包括ResNet50、InceptionV3、Vision Transformers (ViT)，以及多个通过多层感知器（MLP）分类器增强的U-Net变体。为了提高可解释性和临床相关性，我们整合了诸如Grad-CAM和显著图(Saliency Maps)等xAI方法。模型性能通过分类的AUC值及重建质量的PSNR/SSIM指标进行评估，并辅以定性的ROI可视化结果。 我们的研究结果显示，在MRNet数据集的约束下，ResNet50在分类和ROI识别方面始终表现出色，优于基于变换器（transformer）的模型。尽管混合U-Net + MLP方法显示出利用空间特征于重建和可解释性方面的潜力，但它们的分类性能仍较低。Grad-CAM在整个架构中持续提供了最具临床意义的解释。总体而言，基于卷积神经网络(CNN)的迁移学习对于该数据集是最有效的策略，而未来使用更大规模预训练的研究可能会更好地解锁变换器模型的潜在优势。
URL
https://arxiv.org/abs/2508.14151
PDF
-
Extracting Structured Requirements from Unstructured Building Technical Specifications for Building Information Modeling
2025-08-19 13:55:41Insaf Nahri, Romain Pinqui\'e, Philippe V\'eron, Nicolas Bus, Mathieu ThorelAbstract
This study explores the integration of Building Information Modeling (BIM) with Natural Language Processing (NLP) to automate the extraction of requirements from unstructured French Building Technical Specification (BTS) documents within the construction industry. Employing Named Entity Recognition (NER) and Relation Extraction (RE) techniques, the study leverages the transformer-based model CamemBERT and applies transfer learning with the French language model Fr\_core\_news\_lg, both pre-trained on a large French corpus in the general domain. To benchmark these models, additional approaches ranging from rule-based to deep learning-based methods are developed. For RE, four different supervised models, including Random Forest, are implemented using a custom feature vector. A hand-crafted annotated dataset is used to compare the effectiveness of NER approaches and RE models. Results indicate that CamemBERT and Fr\_core\_news\_lg exhibited superior performance in NER, achieving F1-scores over 90\%, while Random Forest proved most effective in RE, with an F1 score above 80\%. The outcomes are intended to be represented as a knowledge graph in future work to further enhance automatic verification systems.
Abstract (translated)
这项研究探讨了将建筑信息模型（BIM）与自然语言处理（NLP）相结合，以自动化提取法国建筑工程技术规范（BTS）文档中的需求。这些文档是未结构化的，并且是在建筑业中使用的。该研究利用命名实体识别（NER）和关系抽取（RE）的技术手段，采用基于变换器的CamemBERT模型，并使用法语语言模型Fr_core_news_lg进行迁移学习，后者在广泛的通用领域大型语料库上进行了预训练。为了对这些模型进行基准测试，开发了从规则基础方法到深度学习方法的一系列额外方法。 对于关系抽取（RE），采用了一个自定义特征向量，实现了包括随机森林在内的四种不同的监督式模型。使用手工注释的数据集来比较命名实体识别（NER）的方法和关系抽取（RE）的模型的有效性。结果表明，在命名实体识别方面，CamemBERT和Fr_core_news_lg表现出色，F1分数超过90%；而在关系抽取方面，随机森林方法最为有效，其F1分数超过了80%。 研究结果旨在未来的进一步工作中以知识图谱的形式表示出来，从而增强自动验证系统。
URL
https://arxiv.org/abs/2508.13833
PDF
-
ITL-LIME: Instance-Based Transfer Learning for Enhancing Local Explanations in Low-Resource Data Settings
2025-08-19 09:18:20Rehan Raza, Guanjin Wang, Kevin Wong, Hamid Laga, Marco FisichellaAbstract
Explainable Artificial Intelligence (XAI) methods, such as Local Interpretable Model-Agnostic Explanations (LIME), have advanced the interpretability of black-box machine learning models by approximating their behavior locally using interpretable surrogate models. However, LIME's inherent randomness in perturbation and sampling can lead to locality and instability issues, especially in scenarios with limited training data. In such cases, data scarcity can result in the generation of unrealistic variations and samples that deviate from the true data manifold. Consequently, the surrogate model may fail to accurately approximate the complex decision boundary of the original model. To address these challenges, we propose a novel Instance-based Transfer Learning LIME framework (ITL-LIME) that enhances explanation fidelity and stability in data-constrained environments. ITL-LIME introduces instance transfer learning into the LIME framework by leveraging relevant real instances from a related source domain to aid the explanation process in the target domain. Specifically, we employ clustering to partition the source domain into clusters with representative prototypes. Instead of generating random perturbations, our method retrieves pertinent real source instances from the source cluster whose prototype is most similar to the target instance. These are then combined with the target instance's neighboring real instances. To define a compact locality, we further construct a contrastive learning-based encoder as a weighting mechanism to assign weights to the instances from the combined set based on their proximity to the target instance. Finally, these weighted source and target instances are used to train the surrogate model for explanation purposes.
Abstract (translated)
可解释的人工智能（XAI）方法，例如局部可解释模型无关解释（LIME），通过使用可解释的替代模型来近似黑盒机器学习模型的行为，从而提高了模型的透明度。然而，LIME在扰动和采样过程中固有的随机性可能会导致局部性和不稳定问题，尤其是在训练数据有限的情况下。在这种情况下，数据稀缺可能导致生成不现实的变化和样本，这些样本偏离了真实的数据流形。因此，替代模型可能无法准确地近似原始模型的复杂决策边界。 为了解决这些问题，我们提出了一种基于实例的迁移学习LIME框架（ITL-LIME），该框架在数据受限环境中提高了解释的精确性和稳定性。通过利用相关源域中的实际样本来帮助目标领域的解释过程，ITL-LIME将实例转移学习引入到LIME框架中。具体来说，我们采用聚类方法将源领域划分成具有代表原型的集群。我们的方法不是生成随机扰动，而是从与目标样本最相似的源簇中检索相关的实际源样本，并将其与目标样本附近的实际样本相结合。 为了定义一个紧凑的空间局部区域，我们进一步构建了一个基于对比学习的编码器作为加权机制，用于根据结合集合中的实例相对于目标样本的距离分配权重。最后，使用这些加权后的源和目标实例来训练替代模型以供解释用途。这种框架旨在提高在数据稀少情况下的模型可解释性的准确性和稳定性。
URL
https://arxiv.org/abs/2508.13672
PDF
-
Is Transfer Learning Necessary for Violin Transcription?
2025-08-19 05:20:24Yueh-Po Peng, Ting-Kang Wang, Li Su, Vincent K. M. CheungAbstract
Automatic music transcription (AMT) has achieved remarkable progress for instruments such as the piano, largely due to the availability of large-scale, high-quality datasets. In contrast, violin AMT remains underexplored due to limited annotated data. A common approach is to fine-tune pretrained models for other downstream tasks, but the effectiveness of such transfer remains unclear in the presence of timbral and articulatory differences. In this work, we investigate whether training from scratch on a medium-scale violin dataset can match the performance of fine-tuned piano-pretrained models. We adopt a piano transcription architecture without modification and train it on the MOSA dataset, which contains about 30 hours of aligned violin recordings. Our experiments on URMP and Bach10 show that models trained from scratch achieved competitive or even superior performance compared to fine-tuned counterparts. These findings suggest that strong violin AMT is possible without relying on pretrained piano representations, highlighting the importance of instrument-specific data collection and augmentation strategies.
Abstract (translated)
自动音乐转录（AMT）在钢琴等乐器上取得了显著进展，这主要得益于大规模、高质量数据集的可用性。相比之下，由于标注数据有限，小提琴 AMT 的研究相对较少。一种常见的方法是微调为其他下游任务预训练的模型，但在音色和发音差异存在的情况下，这种迁移的有效性仍不清楚。在这项工作中，我们调查了在中等规模的小提琴数据集上从头开始训练是否可以与钢琴预训练模型的微调性能相匹配。我们采用未经修改的钢琴转录架构，并将其在包含约30小时对齐小提琴录音的MOSA数据集上进行训练。我们在URMP和Bach10上的实验表明，从零开始训练的模型达到了与微调版本相当甚至更优的表现。这些发现表明，在不依赖于预训练钢琴表示的情况下，可以实现强大的小提琴AMT，这强调了乐器特定数据收集和增强策略的重要性。
URL
https://arxiv.org/abs/2508.13516
PDF
-
Integrating Feedback Loss from Bi-modal Sarcasm Detector for Sarcastic Speech Synthesis
2025-08-18 15:44:54Zhu Li, Yuqing Zhang, Xiyuan Gao, Devraj Raghuvanshi, Nagendra Kumar, Shekhar Nayak, Matt ColerAbstract
Sarcastic speech synthesis, which involves generating speech that effectively conveys sarcasm, is essential for enhancing natural interactions in applications such as entertainment and human-computer interaction. However, synthesizing sarcastic speech remains a challenge due to the nuanced prosody that characterizes sarcasm, as well as the limited availability of annotated sarcastic speech data. To address these challenges, this study introduces a novel approach that integrates feedback loss from a bi-modal sarcasm detection model into the TTS training process, enhancing the model's ability to capture and convey sarcasm. In addition, by leveraging transfer learning, a speech synthesis model pre-trained on read speech undergoes a two-stage fine-tuning process. First, it is fine-tuned on a diverse dataset encompassing various speech styles, including sarcastic speech. In the second stage, the model is further refined using a dataset focused specifically on sarcastic speech, enhancing its ability to generate sarcasm-aware speech. Objective and subjective evaluations demonstrate that our proposed methods improve the quality, naturalness, and sarcasm-awareness of synthesized speech.
Abstract (translated)
讽刺语调的语音合成，即生成能够有效传达讽刺意味的语音，在娱乐和人机交互应用中提高自然互动体验方面至关重要。然而，由于讽刺表达具有复杂的语调特点以及可用的标注讽刺语料数据有限，因此在实际操作中实现这一目标仍面临诸多挑战。 为了应对这些挑战，本研究提出了一种创新方法，该方法将双模态讽刺检测模型提供的反馈损失融入到文本转语音（TTS）训练过程中。通过这种方式，可以增强模型捕捉和传达讽刺信息的能力。此外，在进行迁移学习的基础上，先对大量朗读语料预训练的语音合成模型进行了两阶段微调处理：第一阶段基于涵盖各种语言风格（包括讽刺表达）的数据集；第二阶段则专门使用聚焦于讽刺表达的数据集进一步优化模型，使其能够生成带有讽刺意味的语音。 通过客观和主观评估，我们证明了所提出的方法可以提高合成语音的质量、自然度以及其对讽刺语境的理解力。
URL
https://arxiv.org/abs/2508.13028
PDF
-
Preserve and Sculpt: Manifold-Aligned Fine-tuning of Vision-Language Models for Few-Shot Learning
2025-08-18 12:28:43Dexia Chen, Qianjie Zhu, Weibing Li, Yue Yu, Tong Zhang, Ruixuan WangAbstract
Pretrained vision-language models (VLMs), such as CLIP, have shown remarkable potential in few-shot image classification and led to numerous effective transfer learning strategies. These methods leverage the pretrained knowledge of VLMs to enable effective domain adaptation while mitigating overfitting through parameter-efficient tuning or instance-based consistency constraints. However, such regularizations often neglect the geometric structure of data distribution, which may lead to distortion of the overall semantic representation. To overcome this limitation, we propose a novel fine-tuning method, Manifold-Preserving and Sculpting Tuning (MPS-Tuning). Regarding the data distribution in feature space as a semantic manifold, MPS-Tuning explicitly constrains the intrinsic geometry of this manifold while further sculpting it to enhance class separability. Specifically, MPS-Tuning preserves both macroscopic and microscopic topological structures of the original manifold by aligning Gram matrices of features before and after fine-tuning. Theoretically, this constraint is shown to approximate an upper bound of the Gromov-Wasserstein distance. Furthermore, features from the image and text modalities are paired, and pairwise similarities are optimized to enhance the manifold's class discriminability. Extensive experiments demonstrate that MPS-Tuning significantly improves model performance while effectively preserving the structure of the semantic manifold. The code will be released.
Abstract (translated)
预训练的视觉-语言模型（VLMs），如CLIP，在少量样本图像分类中表现出显著潜力，并促成了许多有效的迁移学习策略。这些方法利用了VLMs的预训练知识，以实现有效的领域适应性并通过参数高效的微调或基于实例的一致性约束来减轻过拟合问题。然而，这种正则化通常忽视了数据分布的几何结构，这可能导致整体语义表示失真。为了克服这一局限，我们提出了一种新的微调方法：保形雕塑微调（MPS-Tuning）。将特征空间中的数据分布视为语义流形，MPS-Tuning 明确地约束了这种流形的内在几何形状，并进一步对其进行塑形以增强类别可分离性。具体来说，通过在微调前后的特征Gram矩阵对齐来保留原始流形的宏观和微观拓扑结构。从理论上讲，这一限制显示其接近Gromov-Wasserstein距离的一个上限。此外，图像和文本模式中的特征被配对，并且成对相似度被优化以增强流形的类别判别能力。广泛的实验表明，MPS-Tuning 显著提高了模型性能，同时有效地保持了语义流形的结构。代码将会发布。
URL
https://arxiv.org/abs/2508.12877
PDF
-
Cross-Domain Few-Shot Learning via Multi-View Collaborative Optimization with Vision-Language Models
2025-08-18 12:00:09Dexia Chen, Wentao Zhang, Qianjie Zhu, Ping Hu, Weibing Li, Tong Zhang, Ruixuan WangAbstract
Vision-language models (VLMs) pre-trained on natural image and language data, such as CLIP, have exhibited significant potential in few-shot image recognition tasks, leading to development of various efficient transfer learning methods. These methods exploit inherent pre-learned knowledge in VLMs and have achieved strong performance on standard image datasets. However, their effectiveness is often limited when confronted with cross-domain tasks where imaging domains differ from natural images. To address this limitation, we propose Consistency-guided Multi-view Collaborative Optimization (CoMuCo), a novel fine-tuning strategy for VLMs. This strategy employs two functionally complementary expert modules to extract multi-view features, while incorporating prior knowledge-based consistency constraints and information geometry-based consensus mechanisms to enhance the robustness of feature learning. Additionally, a new cross-domain few-shot benchmark is established to help comprehensively evaluate methods on imaging domains distinct from natural images. Extensive empirical evaluations on both existing and newly proposed benchmarks suggest CoMuCo consistently outperforms current methods in few-shot tasks. The code and benchmark will be released.
Abstract (translated)
预训练于自然图像和语言数据上的视觉-语言模型（VLMs，如CLIP），在少量样本的图像识别任务中展现了巨大的潜力，并促进了各种高效迁移学习方法的发展。这些方法利用了VLMs内部预先学习的知识，在标准图像数据集上取得了强劲的表现。然而，当面对跨域任务时，即成像领域与自然图像不同，它们的有效性往往受到限制。 为了解决这一局限性，我们提出了一种新的微调策略：一致性引导的多视图协同优化（CoMuCo）。该策略采用两个功能互补的专家模块来提取多视图特征，并通过引入基于先验知识的一致性约束以及基于信息几何的共识机制，增强了特征学习的鲁棒性。此外，我们还建立了一个新的跨域少量样本基准测试集，以便全面评估那些与自然图像领域不同的成像领域的方法。 在现有和新提出的基准上进行广泛的经验评测表明，CoMuCo在少量样本任务中始终优于当前的方法。代码和基准将在不久的将来发布。
URL
https://arxiv.org/abs/2508.12861
PDF
-
Skin Cancer Classification: Hybrid CNN-Transformer Models with KAN-Based Fusion
2025-08-17 19:57:34Shubhi Agarwal, Amulya Kumar MahtoAbstract
Skin cancer classification is a crucial task in medical image analysis, where precise differentiation between malignant and non-malignant lesions is essential for early diagnosis and treatment. In this study, we explore Sequential and Parallel Hybrid CNN-Transformer models with Convolutional Kolmogorov-Arnold Network (CKAN). Our approach integrates transfer learning and extensive data augmentation, where CNNs extract local spatial features, Transformers model global dependencies, and CKAN facilitates nonlinear feature fusion for improved representation learning. To assess generalization, we evaluate our models on multiple benchmark datasets (HAM10000,BCN20000 and PAD-UFES) under varying data distributions and class imbalances. Experimental results demonstrate that hybrid CNN-Transformer architectures effectively capture both spatial and contextual features, leading to improved classification performance. Additionally, the integration of CKAN enhances feature fusion through learnable activation functions, yielding more discriminative representations. Our proposed approach achieves competitive performance in skin cancer classification, demonstrating 92.81% accuracy and 92.47% F1-score on the HAM10000 dataset, 97.83% accuracy and 97.83% F1-score on the PAD-UFES dataset, and 91.17% accuracy with 91.79% F1- score on the BCN20000 dataset highlighting the effectiveness and generalizability of our model across diverse datasets. This study highlights the significance of feature representation and model design in advancing robust and accurate medical image classification.
Abstract (translated)
皮肤癌分类是医学图像分析中的关键任务，准确地区分恶性病变和非恶性病变对于早期诊断和治疗至关重要。在这项研究中，我们探讨了结合卷积柯尔莫哥洛夫-阿恩德网络（CKAN）的序列和并行混合CNN-Transformer模型。我们的方法整合了迁移学习和广泛的数据增强技术，其中CNN用于提取局部空间特征，Transformer用于建模全局依赖关系，而CKAN通过可学习的激活函数促进非线性特征融合，以改进表示学习。 为了评估泛化能力，我们使用多个基准数据集（HAM10000、BCN20000和PAD-UFES）在不同的数据分布和类别不平衡情况下对我们的模型进行评价。实验结果显示，混合CNN-Transformer架构能够有效捕捉空间特征和上下文信息，从而提高分类性能。此外，CKAN的集成通过可学习的激活函数增强了特征融合功能，生成了更具区分性的表示。 我们提出的这种方法在皮肤癌分类中表现出色，在HAM10000数据集上达到了92.81%的准确率和92.47%的F1分数，在PAD-UFES数据集上分别达到97.83%的准确率和97.83%的F1分数，以及在BCN20000数据集上实现了91.17%的准确率和91.79%的F1分数。这些结果突显了我们模型在不同数据集中具有有效性和泛化能力。 这项研究强调了特征表示和模型设计对于提高医学图像分类鲁棒性和准确性的重要性。
URL
https://arxiv.org/abs/2508.12484
PDF
