Paper Reading AI Learner

MultiADE: A Multi-domain Benchmark for Adverse Drug Event Extraction

2024-05-28 09:57:28
Xiang Dai, Sarvnaz Karimi, Abeed Sarker, Ben Hachey, Cecile Paris

Abstract

Objective. Active adverse event surveillance monitors Adverse Drug Events (ADE) from different data sources, such as electronic health records, medical literature, social media and search engine logs. Over years, many datasets are created, and shared tasks are organised to facilitate active adverse event surveillance. However, most-if not all-datasets or shared tasks focus on extracting ADEs from a particular type of text. Domain generalisation-the ability of a machine learning model to perform well on new, unseen domains (text types)-is under-explored. Given the rapid advancements in natural language processing, one unanswered question is how far we are from having a single ADE extraction model that are effective on various types of text, such as scientific literature and social media posts}. Methods. We contribute to answering this question by building a multi-domain benchmark for adverse drug event extraction, which we named MultiADE. The new benchmark comprises several existing datasets sampled from different text types and our newly created dataset-CADECv2, which is an extension of CADEC (Karimi, et al., 2015), covering online posts regarding more diverse drugs than CADEC. Our new dataset is carefully annotated by human annotators following detailed annotation guidelines. Conclusion. Our benchmark results show that the generalisation of the trained models is far from perfect, making it infeasible to be deployed to process different types of text. In addition, although intermediate transfer learning is a promising approach to utilising existing resources, further investigation is needed on methods of domain adaptation, particularly cost-effective methods to select useful training instances.

Abstract (translated)

目标. 主动不良反应监视器从不同数据源(如电子病历、医学文献、社交媒体和搜索引擎日志)监测ADEs(不良药物事件)。多年来,许多数据集都被创建,并组织了分享任务以促进主动不良反应监视。然而,大多数数据集或共享任务都集中在从特定类型的文本中提取ADEs。领域泛化-机器学习模型在新、未见过的域中表现良好的能力(文本类型)尚无充分利用。考虑到自然语言处理技术的快速发展,一个未回答的问题是我们距离拥有一个有效的多种文本类型的ADE提取模型还有多远。方法。为了回答这个问题,我们通过构建一个多领域 benchmark for adverse drug event extraction(MultiADE)来回答这个问题。这个新基准包括从不同文本类型和来源的现有数据集中的几个数据样本,以及我们新创建的数据集-CADECv2(CADEC的扩展版),涵盖了比CADEC更为广泛的在线药物帖子。我们的新数据集是由人类标注者仔细标注的,并遵循详细的注释指南。结论。我们的基准结果表明,训练模型的泛化程度还有很大的提升,这使得将其用于处理不同类型的文本变得不合实际。此外,虽然中间迁移学习是一种有前途的方法,以便利用现有资源,但还需要进一步研究领域适应的方法,特别是成本效益高的方法来选择有用的训练实例。

URL

https://arxiv.org/abs/2405.18015

PDF

https://arxiv.org/pdf/2405.18015.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot