Abstract
Table-to-text generation (insight generation from tables) is a challenging task that requires precision in analyzing the data. In addition, the evaluation of existing benchmarks is affected by contamination of Large Language Model (LLM) training data as well as domain imbalance. We introduce FreshTab, an on-the-fly table-to-text benchmark generation from Wikipedia, to combat the LLM data contamination problem and enable domain-sensitive evaluation. While non-English table-to-text datasets are limited, FreshTab collects datasets in different languages on demand (we experiment with German, Russian and French in addition to English). We find that insights generated by LLMs from recent tables collected by our method appear clearly worse by automatic metrics, but this does not translate into LLM and human evaluations. Domain effects are visible in all evaluations, showing that a~domain-balanced benchmark is more challenging.
Abstract (translated)
表格到文本生成（从表格中生成洞察）是一项需要精确数据分析的任务。此外，现有基准的评估受到大型语言模型（LLM）训练数据污染以及领域不平衡的影响。我们引入了FreshTab，这是一个基于维基百科实时生成表格到文本基准的数据集，以解决LLM数据污染问题，并实现领域敏感性评估。尽管非英语的表格到文本数据集有限，但FreshTab可以根据需求收集不同语言的数据集（我们在实验中使用了德语、俄语和法语，以及英语）。我们发现，从最近通过我们的方法收集的表格中生成的LLM洞察在自动评分指标上明显较差，但这并未反映在LLM和人类评估中。所有评估中的领域效应表明，一个领域的平衡基准更具挑战性。
URL
https://arxiv.org/abs/2510.13598
