Abstract
Multimodal fake news detection typically demands complex architectures and substantial computational resources, posing deployment challenges in real-world settings. We introduce UNITE-FND, a novel framework that reframes multimodal fake news detection as a unimodal text classification task. We propose six specialized prompting strategies with Gemini 1.5 Pro, converting visual content into structured textual descriptions, and enabling efficient text-only models to preserve critical visual information. To benchmark our approach, we introduce Uni-Fakeddit-55k, a curated dataset family of 55,000 samples each, each processed through our multimodal-to-unimodal translation framework. Experimental results demonstrate that UNITE-FND achieves 92.52% accuracy in binary classification, surpassing prior multimodal models while reducing computational costs by over 10x (TinyBERT variant: 14.5M parameters vs. 250M+ in SOTA models). Additionally, we propose a comprehensive suite of five novel metrics to evaluate image-to-text conversion quality, ensuring optimal information preservation. Our results demonstrate that structured text-based representations can replace direct multimodal processing with minimal loss of accuracy, making UNITE-FND a practical and scalable alternative for resource-constrained environments.
Abstract (translated)
多模态假新闻检测通常需要复杂的架构和大量的计算资源，这在实际部署中带来了挑战。我们提出了UNITE-FND框架，该框架将多模态假新闻检测重新定义为单模态文本分类任务。我们与Gemini 1.5 Pro合作，提出六种专门的提示策略，能够将视觉内容转化为结构化的文本描述，并使纯文本模型能够高效地保留关键的视觉信息。为了评估我们的方法，我们引入了Uni-Fakeddit-55k数据集家族，包含经过我们多模态到单模态转换框架处理后的55,000个样本。实验结果显示，UNITE-FND在二元分类中达到了92.52%的准确率，超过了之前的多模态模型，并将计算成本减少了超过10倍（TinyBERT变体：14.5M参数 vs 最先进的模型中的2.5亿+）。此外，我们还提出了一套五种新的评估指标来衡量图像到文本转换的质量，以确保最佳的信息保留。我们的研究结果表明，基于结构化文本的表示可以替代直接的多模态处理，并且在准确性上几乎没有损失，这使得UNITE-FND成为资源受限环境中的实用和可扩展的解决方案。
URL
https://arxiv.org/abs/2502.11132
