Abstract
Text-attributed graphs (TAGs) have emerged as a powerful representation for modeling complex relationships across diverse domains. With the rise of large language models (LLMs), there is growing interest in leveraging their capabilities for graph learning. However, current approaches face significant challenges in embedding structural information into LLM-compatible formats, requiring either computationally expensive alignment mechanisms or manual graph verbalization techniques that often lose critical structural details. Moreover, these methods typically require labeled data from source domains for effective transfer learning, significantly constraining their adaptability. We propose STAG, a novel self-supervised framework that directly quantizes graph structural information into discrete tokens using a frozen codebook. Unlike traditional quantization approaches, our method employs soft assignment and KL divergence guided quantization to address the unique challenges of graph data, which lacks natural tokenization structures. Our framework enables both LLM-based and traditional learning approaches, supporting true zero-shot transfer learning without requiring labeled data even in the source domain. Extensive experiments demonstrate state-of-the-art performance across multiple node classification benchmarks while maintaining compatibility with different LLM architectures, offering an elegant solution to bridging graph learning with LLMs.
Abstract (translated)
基于属性的图（TAG）作为一种强大的表示方法，已被用于跨多个领域的复杂关系建模。随着大规模语言模型（LLMs）的发展，人们对利用其能力进行图学习的兴趣日益增加。然而，目前的方法在将结构信息嵌入到与LLM兼容的格式中面临重大挑战，这要么需要计算成本高昂的对齐机制，要么需要手动图转文本的技术，而这些技术往往会导致关键结构细节的丢失。此外，这些方法通常需要来自源域的标注数据以实现有效的迁移学习，从而大大限制了其适应性。 我们提出了STAG，一种新颖的自我监督框架，该框架直接利用冻结代码簿将图的结构信息量化为离散标记。与传统的量化方法不同，我们的方法采用软分配和KL散度引导的量化来解决图数据特有的挑战，即缺乏自然的分词结构。我们的框架同时支持基于LLM的方法和传统学习方法，并且能够在不依赖源域标注数据的情况下实现真正的零样本迁移学习。广泛的实验表明，STAG在多个节点分类基准测试中表现出色，同时与不同的LLM架构兼容，为图学习与大规模语言模型之间的桥梁提供了一种优雅的解决方案。
URL
https://arxiv.org/abs/2507.19526
