Abstract
Electronic medical records (EMRs) are stored in relational databases. It can be challenging to access the required information if the user is unfamiliar with the database schema or general database fundamentals. Hence, researchers have explored text-to-SQL generation methods that provide healthcare professionals direct access to EMR data without needing a database expert. However, currently available datasets have been essentially "solved" with state-of-the-art models achieving accuracy greater than or near 90%. In this paper, we show that there is still a long way to go before solving text-to-SQL generation in the medical domain. To show this, we create new splits of the existing medical text-to-SQL dataset MIMICSQL that better measure the generalizability of the resulting models. We evaluate state-of-the-art language models on our new split showing substantial drops in performance with accuracy dropping from up to 92% to 28%, thus showing substantial room for improvement. Moreover, we introduce a novel data augmentation approach to improve the generalizability of the language models. Overall, this paper is the first step towards developing more robust text-to-SQL models in the medical domain.\footnote{The dataset and code will be released upon acceptance.
Abstract (translated)
电子医疗记录(EMRs)存储在关系型数据库中。如果用户不熟悉数据库表 schema或一般数据库基础结构，那么访问所需的信息可能会非常困难。因此，研究人员已经探索了文本到SQL生成方法，以便提供医疗保健专业人员直接访问EMR数据，而不需要数据库专家。然而，目前可用的数据集基本上已经“解决”，最先进的模型准确率超过或接近于90%。在本文中，我们表明，在医疗领域中解决文本到SQL生成问题还有很长的路要走。为了展示这一点，我们创造了新的医疗文本到SQL数据集MIMICSQL的分集，更好地衡量结果模型的通用性。我们评估了最先进的语言模型在我们的新分集中的表现，显示性能大幅度下降，准确率从高达92%降至28%，因此表明有很大的改进空间。此外，我们引入了一种新的数据增强方法，以提高语言模型的通用性。总的来说，本文是开发医疗领域中更稳定的文本到SQL模型的第一步。
URL
https://arxiv.org/abs/2303.12898
