Abstract
Human engagement estimation in conversational scenarios is essential for applications such as adaptive tutoring, remote healthcare assessment, and socially aware human--computer interaction. Engagement is a dynamic, multimodal signal conveyed by facial expressions, speech, gestures, and behavioral cues over time. In this work we introduce DA-Mamba, a dialogue-aware multimodal architecture that replaces attention-heavy dialogue encoders with Mamba-based selective state-space processing to achieve linear time and memory complexity while retaining expressive cross-modal reasoning. We design a Mamba dialogue-aware selective state-space model composed of three core modules: a Dialogue-Aware Encoder, and two Mamba-based fusion mechanisms: Modality-Group Fusion and Partner-Group Fusion, these modules achieve expressive dialogue understanding. Extensive experiments on three standard benchmarks (NoXi, NoXi-Add, and MPIIGI) show that DA-Mamba surpasses prior state-of-the-art (SOTA) methods in concordance correlation coefficient (CCC), while reducing training time and peak memory; these gains enable processing much longer sequences and facilitate real-time deployment in resource-constrained, multi-party conversational settings. The source code will be available at: this https URL.
Abstract (translated)
在对话场景中的人类参与度估计对于自适应辅导、远程医疗评估和社会意识型人机交互等应用至关重要。参与度是一个随时间变化的多模态信号，通过面部表情、言语、手势和行为线索传达。 在这项工作中，我们引入了DA-Mamba（Dialogue-Aware Multimodal Architecture with Mamba-based Selective State-Space Processing），这是一种对话感知的多模式架构，它用基于Mamba的选择性状态空间处理方法替代了注重注意力的对话编码器。这种架构在保持跨模态推理能力的同时实现了线性的时间和内存复杂度。我们设计了一个由三个核心模块组成的DA-Mamba对话感知选择性状态空间模型：一个对话感知编解码器，以及两种基于Mamba的融合机制——模式组融合（Modality-Group Fusion）和伙伴组融合（Partner-Group Fusion）。这些模块共同实现了具有表达性的对话理解。 在三个标准基准数据集（NoXi、NoXi-Add 和 MPIIGI）上的广泛实验表明，DA-Mamba 在一致性相关系数 (CCC) 方面超越了先前的最先进方法，并且减少了训练时间和峰值内存使用量。这种改进使得处理更长序列成为可能，并有助于在资源受限的多方对话场景中实现实时部署。 该研究的源代码将在以下网址提供：this https URL.
URL
https://arxiv.org/abs/2509.17711
