Abstract
Recent advances in large language models (LLMs) have led to the development of multimodal LLMs (MLLMs) in the fields of natural language processing (NLP) and computer vision. Although these models allow for integrated visual and language understanding, they present challenges such as opaque internal processing and the generation of hallucinations and misinformation. Therefore, there is a need for a method to clarify the location of knowledge in MLLMs. In this study, we propose a method to identify neurons associated with specific knowledge using MiniGPT-4, a Transformer-based MLLM. Specifically, we extract knowledge neurons through two stages: activation differences filtering using inpainting and gradient-based filtering using GradCAM. Experiments on the image caption generation task using the MS COCO 2017 dataset, BLEU, ROUGE, and BERTScore quantitative evaluation, and qualitative evaluation using an activation heatmap showed that our method is able to locate knowledge with higher accuracy than existing methods. This study contributes to the visualization and explainability of knowledge in MLLMs and shows the potential for future knowledge editing and control.
Abstract (translated)
近期，在自然语言处理（NLP）和计算机视觉领域，大型语言模型（LLM）的进展促使了多模态大型语言模型（MLLMs）的发展。尽管这些模型能够实现集成化的视觉与语言理解，但它们也带来了诸如内部处理不透明、生成幻觉及错误信息等挑战。因此，需要一种方法来明确多模态大语言模型中知识的位置。本研究提出了一种利用基于Transformer架构的MiniGPT-4模型识别特定知识相关神经元的方法。具体而言，我们通过两个阶段提取知识神经元：第一阶段使用修复（inpainting）技术进行激活差异过滤；第二阶段采用GradCAM方法进行梯度基过滤。 在图像描述生成任务中，我们利用MS COCO 2017数据集进行了实验，并使用BLEU、ROUGE和BERTScore等定量评估指标以及通过激活热图的定性评估来验证我们的方法。结果显示，相较于现有技术，本研究的方法能够以更高的准确性定位知识位置。 这项研究为多模态大语言模型的知识可视化及可解释性做出了贡献，并展示了未来在知识编辑与控制方面的潜在应用。
URL
https://arxiv.org/abs/2503.22941
