Paper Reading AI Learner

Diffusion Forcing for Multi-Agent Interaction Sequence Modeling

2025-12-19 18:59:02
Vongani H. Maluleke, Kie Horiuchi, Lea Wilken, Evonne Ng, Jitendra Malik, Angjoo Kanazawa

Abstract

Understanding and generating multi-person interactions is a fundamental challenge with broad implications for robotics and social computing. While humans naturally coordinate in groups, modeling such interactions remains difficult due to long temporal horizons, strong inter-agent dependencies, and variable group sizes. Existing motion generation methods are largely task-specific and do not generalize to flexible multi-agent generation. We introduce MAGNet (Multi-Agent Diffusion Forcing Transformer), a unified autoregressive diffusion framework for multi-agent motion generation that supports a wide range of interaction tasks through flexible conditioning and sampling. MAGNet performs dyadic prediction, partner inpainting, and full multi-agent motion generation within a single model, and can autoregressively generate ultra-long sequences spanning hundreds of v. Building on Diffusion Forcing, we introduce key modifications that explicitly model inter-agent coupling during autoregressive denoising, enabling coherent coordination across agents. As a result, MAGNet captures both tightly synchronized activities (e.g, dancing, boxing) and loosely structured social interactions. Our approach performs on par with specialized methods on dyadic benchmarks while naturally extending to polyadic scenarios involving three or more interacting people, enabled by a scalable architecture that is agnostic to the number of agents. We refer readers to the supplemental video, where the temporal dynamics and spatial coordination of generated interactions are best appreciated. Project page: this https URL

Abstract (translated)

理解并生成多人互动是机器人技术和社交计算中一个基本且具有广泛影响的挑战。虽然人类在群体活动中自然地进行协调,但由于长时间跨度、强烈的代理间依赖关系以及变化不定的群组规模,建模这些互动仍然非常困难。现有的运动生成方法大多针对特定任务,并不能很好地泛化到灵活多代理体的生成上。 我们引入了MAGNet(多代理扩散强迫变压器),这是一种统一的自回归扩散框架,用于多代理体动作生成,支持通过灵活条件和采样进行多种交互任务。MAGNet能够在单个模型中执行二元预测、伙伴补全以及完全多代理体运动生成,并能自回归地生成跨越数百帧的超长序列。基于扩散强迫技术,我们引入了关键修改,这些修改明确地在自回归去噪过程中建模代理间耦合关系,从而使得不同代理间的协调更加连贯。因此,MAGNet能够捕捉到紧密同步的活动(如跳舞、拳击)和松散结构化的社交互动。 我们的方法在二元基准测试中与专用方法表现相当,并且自然地扩展到了涉及三个或更多相互作用的人的多边场景,这是由一个不依赖于代理体数量的可伸缩架构支持的。我们鼓励读者参考项目页面上的补充视频,在那里可以更好地欣赏生成互动中的时间动态和空间协调。 项目页面: [此处应为实际链接,请参见原始文献获取]

URL

https://arxiv.org/abs/2512.17900

PDF

https://arxiv.org/pdf/2512.17900.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot