Abstract
Time, cost, and energy efficiency are critical considerations in Deep-Learning (DL), particularly when processing long texts. Transformers, which represent the current state of the art, exhibit quadratic computational complexity relative to input length, making them inefficient for extended documents. This study introduces a novel model architecture that combines Graph Neural Networks (GNNs) and Convolutional Neural Networks (CNNs), integrated with a real-time, end-to-end graph generation mechanism. The model processes compact batches of character-level inputs without requiring padding or truncation. To enhance performance while maintaining high speed and efficiency, the model incorporates information from Large Language Models (LLMs), such as token embeddings and sentiment polarities, through efficient dictionary lookups. It captures local contextual patterns using CNNs, expands local receptive fields via lattice-based graph structures, and employs small-world graphs to aggregate document-level information. The generated graphs exhibit structural properties indicative of meaningful semantic organization, with an average clustering coefficient of approximately 0.45 and an average shortest path length ranging between 4 and 5. The model is evaluated across multiple text classification tasks, including sentiment analysis and news-categorization, and is compared against state-of-the-art models. Experimental results confirm the proposed model's efficiency and competitive performance.
Abstract (translated)
时间、成本和能源效率是深度学习（DL）中的关键考虑因素，尤其是在处理长文本时。当前最先进的模型——变压器表现出与输入长度呈二次关系的计算复杂性，这使得它们在处理较长文档时效率低下。本研究提出了一种结合图神经网络（GNNs）和卷积神经网络（CNNs）的新颖架构，并整合了实时、端到端的图生成机制。该模型以字符级输入的小批量为单位进行处理，无需填充或截断。 为了在保持高速度和高效率的同时提升性能，该模型通过高效的字典查找方式集成了大型语言模型（LLMs）的信息，包括token嵌入和情感极性等。具体来说，它使用CNN捕捉局部上下文模式，并利用基于格子的图结构扩展了局部感受野；同时采用小世界图来汇总文档级别的信息。 生成的图表显示有意义的语义组织特征，其平均聚类系数约为0.45，而最短路径长度则在4到5之间。该模型在多项文本分类任务上进行了评估，包括情感分析和新闻分类，并与其他最先进的模型进行了比较。实验结果证实了所提出模型的有效性和竞争性性能。
URL
https://arxiv.org/abs/2507.07414
