Abstract
Evidence data for automated fact-checking (AFC) can be in multiple modalities such as text, tables, images, audio, or video. While there is increasing interest in using images for AFC, previous works mostly focus on detecting manipulated or fake images. We propose a novel task, chart-based fact-checking, and introduce ChartBERT as the first model for AFC against chart evidence. ChartBERT leverages textual, structural and visual information of charts to determine the veracity of textual claims. For evaluation, we create ChartFC, a new dataset of 15, 886 charts. We systematically evaluate 75 different vision-language (VL) baselines and show that ChartBERT outperforms VL models, achieving 63.8% accuracy. Our results suggest that the task is complex yet feasible, with many challenges ahead.
Abstract (translated)
自动事实检查证据数据可以采用多种模式,如文本、表格、图像、音频或视频。虽然使用图像进行自动事实检查的兴趣不断增加,但以前的工作主要关注检测操纵或伪造的图像。我们提出了一项全新的任务,基于图表的事实检查,并介绍了 ChartBERT 作为第一个针对图表证据的自动事实检查模型。 ChartBERT利用图表的文本、结构和视觉信息来确定文本声称的真实性。为了评估,我们创建了一个新的数据集 ChartFC,包含 15,886 个图表。我们系统地评估了 75 种视觉语言(VL)基线,并表明 ChartBERT比VL模型表现更好,达到了 63.8%的准确率。我们的结果表明,任务非常复杂,但仍可行,前方有许多挑战。
URL
https://arxiv.org/abs/2301.11843
