Abstract
Semantic text classification has undergone significant advances in recent years due to the rise of large language models (LLMs) and their high dimensional embeddings. While LLM-embeddings are frequently used to store and retrieve text by semantic similarity in vector databases, the global structure semantic relationships in text corpora often remains opaque. Herein we propose a nested density clustering approach, to infer hierarchical trees of semantically related texts. The method starts by identifying texts of strong semantic similarity as it searches for dense clusters in LLM embedding space. As the density criterion is gradually relaxed, these dense clusters merge into more diffuse clusters, until the whole dataset is represented by a single cluster - the root of the tree. By embedding dense clusters into increasingly diffuse ones, we construct a tree structure that captures hierarchical semantic relationships among texts. We outline how this approach can be used to classify textual data for abstracts of scientific abstracts as a case study. This enables the data-driven discovery research areas and their subfields without predefined categories. To evaluate the general applicability of the method, we further apply it to established benchmark datasets such as the 20 News- groups and IMDB 50k Movie Reviews, demonstrating its robustness across domains. Finally we discuss possible applications on scientometrics, topic evolution, highlighting how nested density trees can reveal semantic structure and evolution in textual datasets.
Abstract (translated)
近年来，由于大型语言模型（LLMs）和高维嵌入的兴起，语义文本分类取得了显著进展。尽管LLM-嵌入常用于通过向量数据库存储和检索具有语义相似性的文本，但文本语料库中的全局结构化语义关系往往仍然是不透明的。在此我们提出了一种嵌套密度聚类方法，以推断相关文本的语义层次树。该方法从在LLM嵌入空间中搜索密集簇开始，识别出具有强烈语义相似性的文本。随着密度标准逐渐放松，这些密集簇合并为更稀疏的簇，直到整个数据集由一个单一的簇表示——即树的根部。通过将密集簇嵌入到越来越稀疏的簇中，我们构建了一种结构，该结构捕捉了文本之间的层次语义关系。 本文阐述了如何使用这种方法作为案例研究来对科学摘要的文本来进行分类，这使数据驱动地发现研究领域及其子领域的过程不再需要预定义类别。为了评估该方法的一般适用性，我们将进一步将其应用于20新闻组和IMDb 5万电影评论等已建立基准数据集上，证明了其在不同领域中的鲁棒性。 最后我们讨论了可能的应用场景，包括科学计量学、主题演变等方面，强调嵌套密度树如何揭示文本数据集中的语义结构及其演化。
URL
https://arxiv.org/abs/2512.23471
