Paper Reading AI Learner

Towards aligned body representations in vision models

2025-11-29 07:25:32
Andrey Gizdov, Andrea Procopio, Yichen Li, Daniel Harari, Tomer Ullman

Abstract

Human physical reasoning relies on internal "body" representations - coarse, volumetric approximations that capture an object's extent and support intuitive predictions about motion and physics. While psychophysical evidence suggests humans use such coarse representations, their internal structure remains largely unknown. Here we test whether vision models trained for segmentation develop comparable representations. We adapt a psychophysical experiment conducted with 50 human participants to a semantic segmentation task and test a family of seven segmentation networks, varying in size. We find that smaller models naturally form human-like coarse body representations, whereas larger models tend toward overly detailed, fine-grain encodings. Our results demonstrate that coarse representations can emerge under limited computational resources, and that machine representations can provide a scalable path toward understanding the structure of physical reasoning in the brain.

Abstract (translated)

人类的物理推理依赖于内部“身体”表示形式——这是一种粗糙、体积化的近似,能够捕捉到物体的范围,并支持对运动和物理现象的直观预测。虽然心理物理学证据表明人类使用这种粗略的表示方法,但其内在结构仍然很大程度上未知。在此研究中,我们测试了视觉模型在训练分割任务时是否发展出了类似的表示形式。我们将一项与50名人类参与者进行的心理物理学实验改编为一个语义分割任务,并对七种不同大小的分割网络进行了测试。我们发现较小规模的模型自然地形成了类似人体粗略的身体表示,而较大规模的模型则倾向于产生过于详细、细粒度的编码。我们的研究结果表明,在有限计算资源下可以出现粗糙的表示形式,并且机器表示为理解大脑物理推理结构提供了一条可扩展路径。

URL

https://arxiv.org/abs/2512.00365

PDF

https://arxiv.org/pdf/2512.00365.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot