Abstract
Recent advances in the field of generative artificial intelligence (AI) have blurred the lines between authentic and machine-generated content, making it almost impossible for humans to distinguish between such media. One notable consequence is the use of AI-generated images for fake profiles on social media. While several types of disinformation campaigns and similar incidents have been reported in the past, a systematic analysis has been lacking. In this work, we conduct the first large-scale investigation of the prevalence of AI-generated profile pictures on Twitter. We tackle the challenges of a real-world measurement study by carefully integrating various data sources and designing a multi-stage detection pipeline. Our analysis of nearly 15 million Twitter profile pictures shows that 0.052% were artificially generated, confirming their notable presence on the platform. We comprehensively examine the characteristics of these accounts and their tweet content, and uncover patterns of coordinated inauthentic behavior. The results also reveal several motives, including spamming and political amplification campaigns. Our research reaffirms the need for effective detection and mitigation strategies to cope with the potential negative effects of generative AI in the future.
Abstract (translated)
近年来，在生成人工智能（AI）领域的发展已经使真实和机器生成的内容之间的界限变得模糊，使得人类很难区分这些媒体。一个著名的后果是在社交媒体上使用AI生成的图像作为虚假个人资料。虽然过去的报道中已经提到了几种形式的虚假信息活动和类似事件，但缺乏系统性的分析。在这项工作中，我们对Twitter上AI生成个人资料图片的普及进行了首次大型调查。我们通过仔细整合各种数据来源并设计了一个多阶段检测管道，解决了真实世界测量研究的挑战。我们对近1500万Twitter个人资料图片的分析显示，0.052%是人工生成的，证实了它们在平台上的显著存在。我们全面检查了这些账户的性质和推特内容，并揭示了协同伪造行为的模式。结果还显示了几个动机，包括垃圾邮件和政治放大宣传活动。我们的研究确认了在应对生成AI对未来潜在负面影响方面需要有效的检测和减轻策略。
URL
https://arxiv.org/abs/2404.14244
