Abstract
We propose SemImage, a novel method for representing a text document as a two-dimensional semantic image to be processed by convolutional neural networks (CNNs). In a SemImage, each word is represented as a pixel in a 2D image: rows correspond to sentences and an additional boundary row is inserted between sentences to mark semantic transitions. Each pixel is not a typical RGB value but a vector in a disentangled HSV color space, encoding different linguistic features: the Hue with two components H_cos and H_sin to account for circularity encodes the topic, Saturation encodes the sentiment, and Value encodes intensity or certainty. We enforce this disentanglement via a multi-task learning framework: a ColorMapper network maps each word embedding to the HSV space, and auxiliary supervision is applied to the Hue and Saturation channels to predict topic and sentiment labels, alongside the main task objective. The insertion of dynamically computed boundary rows between sentences yields sharp visual boundaries in the image when consecutive sentences are semantically dissimilar, effectively making paragraph breaks salient. We integrate SemImage with standard 2D CNNs (e.g., ResNet) for document classification. Experiments on multi-label datasets (with both topic and sentiment annotations) and single-label benchmarks demonstrate that SemImage can achieve competitive or better accuracy than strong text classification baselines (including BERT and hierarchical attention networks) while offering enhanced interpretability. An ablation study confirms the importance of the multi-channel HSV representation and the dynamic boundary rows. Finally, we present visualizations of SemImage that qualitatively reveal clear patterns corresponding to topic shifts and sentiment changes in the generated image, suggesting that our representation makes these linguistic features visible to both humans and machines.
Abstract (translated)
我们提出了一种名为SemImage的新方法，用于将文本文档表示为二维语义图像，以便使用卷积神经网络（CNN）进行处理。在SemImage中，每个单词都作为2D图像中的一个像素来表示：行对应于句子，并且会在每两个句子之间插入额外的边界行以标记语义转换。每个像素不是典型的RGB值，而是在解耦合HSV颜色空间中的向量，编码不同的语言特征：色调（包括用于处理循环性的H_cos和H_sin两部分）表示主题，饱和度表示情感，亮度表示强度或确定性。我们通过多任务学习框架强制执行这种解耦：ColorMapper网络将每个词嵌入映射到HSV空间，并对色调和饱和度通道进行辅助监督以预测主题和情感标签，同时实现主要的任务目标。在句子之间插入动态计算的边界行会产生当连续句子语义不相似时图像中的清晰视觉边界，从而有效地使段落之间的分隔变得显著。我们将SemImage与标准2D CNN（例如ResNet）集成用于文档分类。在具有主题和情感标注的多标签数据集以及单标签基准上的实验表明，SemImage可以达到甚至优于强大的文本分类基线（包括BERT和层次注意力网络）的准确性，并且提供增强的可解释性。消融研究表明HSV多通道表示和动态边界行的重要性。最后，我们展示了SemImage的可视化效果，这些可视化的定性结果揭示了生成图像中对应于主题转换和情感变化的明显模式，表明我们的表示方法使语言特征对人和机器都可见。
URL
https://arxiv.org/abs/2512.00088
