Abstract
UnScientify, a system designed to detect scientific uncertainty in scholarly full text. The system utilizes a weakly supervised technique to identify verbally expressed uncertainty in scientific texts and their authorial references. The core methodology of UnScientify is based on a multi-faceted pipeline that integrates span pattern matching, complex sentence analysis and author reference checking. This approach streamlines the labeling and annotation processes essential for identifying scientific uncertainty, covering a variety of uncertainty expression types to support diverse applications including information retrieval, text mining and scientific document processing. The evaluation results highlight the trade-offs between modern large language models (LLMs) and the UnScientify system. UnScientify, which employs more traditional techniques, achieved superior performance in the scientific uncertainty detection task, attaining an accuracy score of 0.808. This finding underscores the continued relevance and efficiency of UnScientify's simple rule-based and pattern matching strategy for this specific application. The results demonstrate that in scenarios where resource efficiency, interpretability, and domain-specific adaptability are critical, traditional methods can still offer significant advantages.
Abstract (translated)
UnScientify 是一个设计用于检测学术全文中科学不确定性的系统。该系统利用弱监督技术来识别科学文本和作者引用中的口头表达的不确定性。UnScientify 的核心方法基于一个多方面的管道，整合了跨度模式匹配、复杂句子分析以及作者引用检查等功能。这种策略简化了标记和注释过程，有助于识别科学不确定性，并涵盖了各种不确定性的表达类型，支持信息检索、文本挖掘和科学文档处理等多种应用。 评估结果强调了现代大型语言模型（LLMs）与 UnScientify 系统之间的权衡取舍。采用传统技术的 UnScientify 在检测科学不确定性任务中表现出色，准确率达到 0.808。这一发现突显了对于此类特定应用而言，简单规则基础和模式匹配策略的相关性和效率。结果表明，在资源效率、可解释性以及领域特定适应能力至关重要的场景下，传统方法仍能提供显著优势。
URL
https://arxiv.org/abs/2503.11376
