Abstract
Conceptual models such as Concept Bottleneck Models (CBMs) have driven substantial progress in improving interpretability for image classification by leveraging human-interpretable concepts. However, extending these models from static images to sequences of images, such as video data, introduces a significant challenge due to the temporal dependencies inherent in videos, which are essential for capturing actions and events. In this work, we introduce MoTIF (Moving Temporal Interpretable Framework), an architectural design inspired by a transformer that adapts the concept bottleneck framework for video classification and handles sequences of arbitrary length. Within the video domain, concepts refer to semantic entities such as objects, attributes, or higher-level components (e.g., 'bow', 'mount', 'shoot') that reoccur across time - forming motifs collectively describing and explaining actions. Our design explicitly enables three complementary perspectives: global concept importance across the entire video, local concept relevance within specific windows, and temporal dependencies of a concept over time. Our results demonstrate that the concept-based modeling paradigm can be effectively transferred to video data, enabling a better understanding of concept contributions in temporal contexts while maintaining competitive performance. Code available at this http URL.
Abstract (translated)
概念模型，如概念瓶颈模型（CBM），通过利用人类可解释的概念，在提高图像分类的可解释性方面取得了显著进展。然而，将这些模型从静态图像扩展到序列图像（如视频数据）带来了重大挑战，因为视频中的时间依赖性对于捕捉动作和事件至关重要。在本工作中，我们介绍了MoTIF（移动时序可解释框架），这是一种受变压器架构启发的设计，它适应了概念瓶颈框架以适用于视频分类，并能够处理任意长度的序列。在视频领域中，概念指的是语义实体，如物体、属性或更高层次的组件（例如，“拉弓”、“上箭”、“射击”）这些成分随着时间反复出现，共同描述和解释动作。 我们的设计明确支持三个互补的角度：整个视频中的全局概念重要性、特定窗口内的局部概念相关性和时间跨度上的概念时序依赖。研究结果表明，基于概念的建模范式可以有效地应用于视频数据，在保持竞争力的同时，使我们更好地理解在时间上下文中概念的作用和贡献。 代码可在该链接获取（请将“this http URL”替换为实际提供的GitHub或任何版本控制系统中的URL）。
URL
https://arxiv.org/abs/2509.20899
