Abstract
This report provide a detailed description of the method that we explored and proposed in the WECIA Emotion Prediction Competition (EPC), which predicts a person's emotion through an artistic work with a comment. The dataset of this competition is ArtELingo, designed to encourage work on diversity across languages and cultures. The dataset has two main challenges, namely modal imbalance problem and language-cultural differences problem. In order to address this issue, we propose a simple yet effective approach called single-multi modal with Emotion-Cultural specific prompt(ECSP), which focuses on using the single modal message to enhance the performance of multimodal models and a well-designed prompt to reduce cultural differences problem. To clarify, our approach contains two main blocks: (1)XLM-R\cite{conneau2019unsupervised} based unimodal model and X$^2$-VLM\cite{zeng2022x} based multimodal model (2) Emotion-Cultural specific prompt. Our approach ranked first in the final test with a score of 0.627.
Abstract (translated)
本报告详细描述了我们参加WECIA情感预测竞赛（EPC）时所探索和提出的方法，该竞赛通过一件艺术作品来预测一个人的情感。比赛的數據集是ArtELingo，旨在鼓励跨語言和文化的作品。比赛數據集有两个主要挑戰，即模态不平衡問題和語言-文化差異問題。为了应对这个问题，我们提出了一个简单而有效的方案，称为情感文化特定提示（ECSP）单一模态与多模态模型。该方案重点使用单一模态信息来提高多模态模型的性能，并设计了一个精心设计的提示来减少文化差异问题。为了明确，我们的方法包含两个主要部分：（1）基于unimodal的XLM-R模型和基于multimodal的X$^2$-VLM模型（2）情感-文化特定提示。我们的方法在决赛测试中排名第一，得分为0.627。
URL
https://arxiv.org/abs/2403.17683
