Abstract
Personality analysis from online short videos has gained prominence due to its applications in personalized recommendation systems, sentiment analysis, and human-computer interaction. Traditional assessment methods, such as questionnaires based on the Big Five Personality Framework, are limited by self-report biases and are impractical for large-scale or real-time analysis. Leveraging the rich, multi-modal data present in short videos offers a promising alternative for more accurate personality inference. However, integrating these diverse and asynchronous modalities poses significant challenges, particularly in aligning time-varying data and ensuring models generalize well to new domains with limited labeled data. In this paper, we propose a novel multi-modal personality analysis framework that addresses these challenges by synchronizing and integrating features from multiple modalities and enhancing model generalization through domain adaptation. We introduce a timestamp-based modality alignment mechanism that synchronizes data based on spoken word timestamps, ensuring accurate correspondence across modalities and facilitating effective feature integration. To capture temporal dependencies and inter-modal interactions, we employ Bidirectional Long Short-Term Memory networks and self-attention mechanisms, allowing the model to focus on the most informative features for personality prediction. Furthermore, we develop a gradient-based domain adaptation method that transfers knowledge from multiple source domains to improve performance in target domains with scarce labeled data. Extensive experiments on real-world datasets demonstrate that our framework significantly outperforms existing methods in personality prediction tasks, highlighting its effectiveness in capturing complex behavioral cues and robustness in adapting to new domains.
Abstract (translated)
从在线短视频中进行人格分析因其在个性化推荐系统、情感分析和人机交互中的应用而变得越来越重要。传统的评估方法，如基于大五人格框架的问卷调查，受限于自我报告偏差，并且对于大规模或实时分析来说并不实际。利用短视频中存在的丰富多模态数据为更准确的人格推断提供了有希望的替代方案。然而，整合这些多样性和异步模式带来了重大挑战，特别是在对齐随时间变化的数据和确保模型在新领域中有限标注数据的情况下能够很好地泛化方面。本文提出了一种新颖的多模态人格分析框架，通过同步和集成来自多个模态的功能，并通过域适应增强模型的泛化能力来解决这些挑战。我们引入了基于时间戳的模式对齐机制，根据所说单词的时间戳同步数据，确保跨模态之间的准确对应并促进有效特征整合。为了捕捉时序依赖性和多模态交互，我们采用双向长短时记忆网络和自注意力机制，使模型能够专注于人格预测中最具信息量的特征。此外，我们开发了一种基于梯度的领域适应方法，将多个源域的知识转移到目标域以改善在标注数据稀缺的情况下的性能。在真实世界数据集上的广泛实验表明，我们的框架在人格预测任务中显著优于现有方法，突显了其捕捉复杂行为线索和适应新领域的稳健性。
URL
https://arxiv.org/abs/2411.00813
