Abstract
Objective: Natural language processing can help minimize human intervention in identifying patients meeting eligibility criteria for clinical trials, but there is still a long way to go to obtain a general and systematic approach that is useful for researchers. We describe two methods taking a step in this direction and present their results obtained during the n2c2 challenge on cohort selection for clinical trials. Materials and Methods: The first method is a weakly supervised method using an unlabeled corpus (MIMIC) to build a silver standard, by producing semi-automatically a small and very precise set of rules to detect some samples of positive and negative patients. This silver standard is then used to train a traditional supervised model. The second method is a terminology-based approach where a medical expert selects the appropriate concepts, and a procedure is defined to search the terms and check the structural or temporal constraints. Results: On the n2c2 dataset containing annotated data about 13 selection criteria on 288 patients, we obtained an overall F1-measure of 0.8969, which is the third best result out of 45 participant teams, with no statistically significant difference with the best-ranked team. Discussion: Both approaches obtained very encouraging results and apply to different types of criteria. The weakly supervised method requires explicit descriptions of positive and negative examples in some reports. The terminology-based method is very efficient when medical concepts carry most of the relevant information. Conclusion: It is unlikely that much more annotated data will be soon available for the task of identifying a wide range of patient phenotypes. One must focus on weakly or non-supervised learning methods using both structured and unstructured data and relying on a comprehensive representation of the patients.
Abstract (translated)
目的：自然语言处理有助于尽可能减少人为干预，以确定符合临床试验合格标准的患者，但要获得一种对研究人员有用的通用、系统的方法还有很长的路要走。我们描述了两种朝这个方向迈出一步的方法，并介绍了在临床试验队列选择的N2C2挑战中获得的结果。材料和方法：第一种方法是一种弱监督方法，使用未标记的语料库（模拟）建立银标准，通过半自动生成一组小的、非常精确的规则来检测一些阳性和阴性患者的样本。该银标准随后用于培训传统的监督模型。第二种方法是基于术语的方法，其中医学专家选择适当的概念，并定义搜索术语和检查结构或时间约束的过程。结果：在对288例患者的13项选择标准进行注释后的n2c2数据集上，我们获得了0.8969的总F1测量值，这是45个参与者团队中的第三个最佳结果，与排名最佳的团队没有统计学显著差异。讨论：这两种方法都获得了非常令人鼓舞的结果，并适用于不同类型的标准。弱监督方法要求在某些报告中明确描述正负示例。当医学概念包含大部分相关信息时，基于术语的方法是非常有效的。结论：不太可能很快就有更多的注释数据用于鉴定广泛的患者表型。必须关注使用结构化和非结构化数据的弱或非监督学习方法，并依赖于患者的全面表示。
URL
https://arxiv.org/abs/1903.07879
