Abstract
Respiratory diseases are a leading cause of death globally, highlighting the urgent need for early and accessible screening methods. While some lung auscultation analysis has been automated and machine learning audio based models are able to predict respiratory pathologies, there remains a critical gap: the lack of intelligent systems that can interact in real-time consultations using natural language. Unlike other clinical domains, such as electronic health records, radiological images, and biosignals, where numerous question-answering (QA) datasets and models have been established, audio-based modalities remain notably underdeveloped. We curated and harmonized data from 11 diverse respiratory audio datasets to construct the first Respiratory Audio Question Answering (RA-QA) dataset. As the first multimodal QA resource of its kind focused specifically on respiratory health, RA-QA bridges clinical audio and natural language in a structured, scalable format. This new data resource contains about 7.5 million QA pairs spanning more than 60 attributes and three question types: single verification, multiple choice, and open-ended questions. Building upon this dataset, we introduce a novel benchmark that compares audio-text generation models with traditional audio classifiers to evaluate their respective performance.\\Our experiments reveal interesting performance variations across different attributes and question types, establishing a baseline and paving the way for more advanced architectures that could further improve the performance. By bridging machine learning with real-world clinical dialogue, our work opens the door to the development of more interactive, intelligent, and accessible diagnostic tools in respiratory healthcare.
Abstract (translated)
呼吸系统疾病是全球死亡的主要原因之一，这凸显了早期和易于获取的筛查方法的迫切需求。虽然一些肺部听诊分析已经被自动化，并且基于音频的机器学习模型能够预测呼吸道病理性问题，但仍存在一个关键缺口：缺乏能够在实时咨询中使用自然语言进行交互的智能系统。与电子健康记录、放射图像和生物信号等其他临床领域相比，这些领域的问答（QA）数据集和模型已经建立得相当多，而基于音频模态的数据则明显处于欠开发状态。我们从11个多样化的呼吸音数据集中整理并协调数据，构建了首个呼吸音问答（RA-QA）数据集。作为首个专注于呼吸道健康的多媒体问答资源，RA-QA以结构化和可扩展的格式将临床音频与自然语言连接起来。这一新的数据资源包含了大约750万个问答对，涵盖了超过60个属性以及三种问题类型：单一验证、多项选择题及开放式问题。 基于这个数据集，我们引入了一个新颖的基准测试，对比了音频-文本生成模型和传统音频分类器在不同性能指标上的表现。我们的实验揭示了各种属性和问题类型的性能差异，建立了基线，并为未来能够进一步提升性能的更先进架构铺平道路。通过将机器学习与现实世界中的临床对话相结合，我们的工作为开发更具互动性、智能化且易于访问的诊断工具打开了大门，在呼吸系统医疗领域中具有重要意义。
URL
https://arxiv.org/abs/2602.18452
