Abstract
Languages change over time. Computational models can be trained to recognize such changes enabling them to estimate the publication date of texts. Despite recent advancements in Large Language Models (LLMs), their performance on automatic dating of texts, also known as Temporal Text Classification (TTC), has not been explored. This study provides the first systematic evaluation of leading proprietary (Claude 3.5, GPT-4o, Gemini 1.5) and open-source (LLaMA 3.2, Gemma 2, Mistral, Nemotron 4) LLMs on TTC using three historical corpora, two in English and one in Portuguese. We test zero-shot and few-shot prompting, and fine-tuning settings. Our results indicate that proprietary models perform well, especially with few-shot prompting. They also indicate that fine-tuning substantially improves open-source models but that they still fail to match the performance delivered by proprietary LLMs.
Abstract (translated)
语言随着时间的推移而发生变化。计算模型可以被训练来识别这些变化，从而能够估计文本的出版日期。尽管近年来大型语言模型（LLM）取得了显著进展，但它们在自动确定文本年代上的表现——即时间文本分类（TTC）方面尚未得到充分研究。本研究首次系统地评估了领先的专有模型（Claude 3.5、GPT-4o、Gemini 1.5）和开源模型（LLaMA 3.2、Gemma 2、Mistral、Nemotron 4）在TTC上的表现，使用三个历史语料库进行评估，其中包括两个英语语料库和一个葡萄牙语文料库。我们测试了零样本和少量样本提示以及微调设置的表现。我们的结果显示，专有模型表现良好，特别是在少量样本提示下。此外，结果还表明开源模型在经过微调后性能显著提升，但仍然无法达到专有LLM的水平。
URL
https://arxiv.org/abs/2603.11295
