Abstract
The performance of current supervised AI systems is tightly connected to the availability of annotated datasets. Annotations are usually collected through annotation tools, which are often designed for specific tasks and are difficult to customize. Moreover, existing annotation tools with an active learning mechanism often only support limited use cases. To address these limitations, we present EASE, an Easily-Customized Annotation System Powered by Efficiency Enhancement Mechanisms. \sysname provides modular annotation units for building customized annotation interfaces and also provides multiple back-end options that suggest annotations using (1) multi-task active learning; (2) demographic feature based active learning; (3) a prompt system that can query the API of large language models. We conduct multiple experiments and user studies to evaluate our system's flexibility and effectiveness. Our results show that our system can meet the diverse needs of NLP researchers and significantly accelerate the annotation process.
Abstract (translated)
当前监督人工智能系统的性能与标注数据可用性密切相关。通常，标注数据是通过标注工具收集的，这些工具通常专为特定任务设计，难以自定义。此外，现有的具有主动学习机制的标注工具通常只支持有限的使用场景。为了解决这些限制，我们提出了Ease，一个 easily- customized Annotation System 动力于效率增强机制的自定义标注系统。sysname提供了模块化的标注单元，以构建自定义标注接口，并提供了多个后端选项，建议使用(1)多任务主动学习；(2)基于年龄特征的主动学习；(3)一个可以查询大型语言模型 API 的提示系统。我们进行了多个实验和用户研究，以评估我们的系统的灵活性和效果。我们的结果表明，我们的系统可以满足不同NLP研究人员的需求，并显著加速标注过程。
URL
https://arxiv.org/abs/2305.14169
