Abstract
Learning interpretable multimodal representations inherently relies on uncovering the conditional dependencies between heterogeneous features. However, sparse graph estimation techniques, such as Graphical Lasso (GLasso), to visual-linguistic domains is severely bottlenecked by high-dimensional noise, modality misalignment, and the confounding of shared versus category-specific topologies. In this paper, we propose Cross-Modal Graphical Lasso (CM-GLasso) that overcomes these fundamental limitations. By coupling a novel text-visualization strategy with a unified vision-language encoder, we strictly align multimodal features into a shared latent space. We introduce a cross-attention distillation mechanism that condenses high-dimensional patches into explicit semantic nodes, naturally extracting spatial-aware cross-modal priors. Furthermore, we unify tailored GLasso estimation and Common-Specific Structure Learning (CSSL) into a joint objective optimized via the Alternating Direction Method of Multiplier (ADMM). This formulation guarantees the simultaneous disentanglement of invariant and class-specific precision matrices without multi-step error accumulation. Extensive experiments across eight benchmarks covering both natural and medical domains demonstrate that CM-GLasso establishes a new state-of-the-art in generative classification and dense semantic segmentation tasks.
Abstract (translated)
学习可解释的多模态表示本质上依赖于挖掘异构特征间的条件依赖关系。然而，将稀疏图估计技术（如图形套索GLasso）应用于视觉-语言领域时，会受到高维噪声、模态错位以及共享拓扑与类别特定拓扑混淆的严重制约。本文提出跨模态图形套索（CM-GLasso）以克服这些根本性限制。通过将新颖的文本-可视化策略与统一视觉-语言编码器相结合，我们将多模态特征严格对齐至共享潜在空间。引入的跨注意力蒸馏机制能将高维图像块压缩为显式语义节点，自然提取空间感知的跨模态先验。此外，我们将定制化图形套索估计与共同-特定结构学习（CSSL）统一为联合目标函数，并通过乘子交替方向法（ADMM）进行优化。该 formulation 保证了不变与类别特定精度矩阵的同步解耦，且避免了多步骤误差累积。在涵盖自然与医学领域的八个基准数据集上的大量实验表明，CM-GLasso 在生成式分类与密集语义分割任务中建立了新的最先进水平。
URL
https://arxiv.org/abs/2604.03953
