Abstract
Misinformation poses a critical societal challenge, and current approaches have yet to produce an effective solution. We propose focusing on generalization, soft classification, and leveraging recent large language models to create more practical tools in contexts where perfect predictions remain unattainable. We begin by demonstrating that GPT-4 and other language models can outperform existing methods in the literature. Next, we explore their generalization, revealing that GPT-4 and RoBERTa-large exhibit critical differences in failure modes, which offer potential for significant performance improvements. Finally, we show that these models can be employed in soft classification frameworks to better quantify uncertainty. We find that models with inferior hard classification results can achieve superior soft classification performance. Overall, this research lays groundwork for future tools that can drive real-world progress on misinformation.
Abstract (translated)
虚假信息是一个关键的社会挑战，而当前的方法尚未能够提供有效的解决方案。我们建议专注于泛化、软分类和利用最近的大型语言模型来创造在无法准确预测的情况下更实用的工具。我们首先证明，GPT-4和其他语言模型可以在文献中比现有方法表现更好。接下来，我们探讨它们的泛化，揭示GPT-4和RoBERTa-large在故障模式方面存在关键差异，这些差异有可能导致显著的性能改进。最后，我们表明这些模型可以在软分类框架中应用，更好地量化不确定性。我们发现，比硬分类结果较差的模型可以实现更好的软分类性能。总的来说，这项研究为未来工具，能够推动虚假信息实际进展的工作奠定了基础。
URL
https://arxiv.org/abs/2305.14928
