Abstract
Phishing, a prevalent cybercrime tactic for decades, remains a significant threat in today's digital world. By leveraging clever social engineering elements and modern technology, cybercrime targets many individuals, businesses, and organizations to exploit trust and security. These cyber-attackers are often disguised in many trustworthy forms to appear as legitimate sources. By cleverly using psychological elements like urgency, fear, social proof, and other manipulative strategies, phishers can lure individuals into revealing sensitive and personalized information. Building on this pervasive issue within modern technology, this paper aims to analyze the effectiveness of 15 Large Language Models (LLMs) in detecting phishing attempts, specifically focusing on a randomized set of "419 Scam" emails. The objective is to determine which LLMs can accurately detect phishing emails by analyzing a text file containing email metadata based on predefined criteria. The experiment concluded that the following models, ChatGPT 3.5, GPT-3.5-Turbo-Instruct, and ChatGPT, were the most effective in detecting phishing emails.
Abstract (translated)
网络钓鱼，这种了几十年来的普遍网络犯罪手段，在当今数字世界中仍然是一个重要的威胁。通过利用聪明的社交工程要素和现代技术，网络犯罪目标众多个人、企业和组织，以利用信任和安全性。这些网络攻击者通常以多种可信形式伪装自己，伪装成合法来源。通过巧妙地使用心理要素，如紧迫感、恐惧、社交证据等操纵策略，网络钓鱼者可以将个人引入透露敏感和个人信息。在现代技术的普遍问题基础上，本文旨在分析15个大型语言模型（LLMs）在检测网络钓鱼尝试方面的有效性，特别关注一个预定义的“419诈骗”电子邮件随机集。目标是要确定哪些LLM可以准确检测到网络钓鱼电子邮件，通过分析包含邮件元数据的文本文件来确定预定义标准。实验结果表明，ChatGPT 3.5、GPT-3.5-Turbo-Instruct和ChatGPT是最有效的检测网络钓鱼电子邮件的模型。
URL
https://arxiv.org/abs/2404.15485
