Abstract
Recent studies have uncovered an interesting phenomenon: unimodal foundation models tend to learn convergent representations, regardless of differences in architecture, training objectives, or data modalities. However, these representations are essentially internal abstractions of samples that characterize samples independently, leading to limited expressiveness. In this paper, we propose The Indra Representation Hypothesis, inspired by the philosophical metaphor of Indra's Net. We argue that representations from unimodal foundation models are converging to implicitly reflect a shared relational structure underlying reality, akin to the relational ontology of Indra's Net. We formalize this hypothesis using the V-enriched Yoneda embedding from category theory, defining the Indra representation as a relational profile of each sample with respect to others. This formulation is shown to be unique, complete, and structure-preserving under a given cost function. We instantiate the Indra representation using angular distance and evaluate it in cross-model and cross-modal scenarios involving vision, language, and audio. Extensive experiments demonstrate that Indra representations consistently enhance robustness and alignment across architectures and modalities, providing a theoretically grounded and practical framework for training-free alignment of unimodal foundation models. Our code is available at this https URL.
Abstract (translated)
近期研究揭示了一个有趣现象：无论架构、训练目标或数据模态有何差异，单模态基础模型倾向于学习收敛表示。然而这些表示本质上是独立刻画样本的内部抽象，导致表达能力受限。本文受因陀罗网哲学隐喻启发，提出因陀罗表示假说。我们认为单模态基础模型的表示正在收敛以隐式反映现实背后的共享关系结构，类似于因陀罗网的关系本体论。我们运用范畴论中的V-丰富化永田嵌入对该假说进行形式化，将因陀罗表示定义为每个样本相对于其他样本的关系轮廓。在给定代价函数下，该形式化被证明具有唯一性、完备性和结构保持性。我们通过角距离实例化因陀罗表示，并在涉及视觉、语言和音频的跨模型与跨模态场景中进行评估。大量实验表明，因陀罗表示能持续增强跨架构与跨模态的鲁棒性与对齐性，为单模态基础模型的免训练对齐提供了理论严谨且实用的框架。代码已公开于此https链接。
URL
https://arxiv.org/abs/2604.04496
