Text_Generation
-
LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals
2026-01-15 18:54:50Gilat Toker, Nitay Calderon, Ohad Amosy, Roi ReichartAbstract
Concept-based explanations quantify how high-level concepts (e.g., gender or experience) influence model behavior, which is crucial for decision-makers in high-stakes domains. Recent work evaluates the faithfulness of such explanations by comparing them to reference causal effects estimated from counterfactuals. In practice, existing benchmarks rely on costly human-written counterfactuals that serve as an imperfect proxy. To address this, we introduce a framework for constructing datasets containing structural counterfactual pairs: LIBERTy (LLM-based Interventional Benchmark for Explainability with Reference Targets). LIBERTy is grounded in explicitly defined Structured Causal Models (SCMs) of the text generation, interventions on a concept propagate through the SCM until an LLM generates the counterfactual. We introduce three datasets (disease detection, CV screening, and workplace violence prediction) together with a new evaluation metric, order-faithfulness. Using them, we evaluate a wide range of methods across five models and identify substantial headroom for improving concept-based explanations. LIBERTy also enables systematic analysis of model sensitivity to interventions: we find that proprietary LLMs show markedly reduced sensitivity to demographic concepts, likely due to post-training mitigation. Overall, LIBERTy provides a much-needed benchmark for developing faithful explainability methods.
Abstract (translated)
基于概念的解释量化了高层次概念（如性别或经验）如何影响模型的行为，这对高风险领域的决策者来说至关重要。最近的研究通过将这些解释与从反事实估计出的参考因果效应进行比较来评估其准确性。然而，在实践中，现有的基准测试依赖于成本高昂且不完美的手工编写反事实数据作为替代方案。为了解决这个问题，我们引入了一个框架用于构建包含结构化反事实对的数据集：LIBERTy（基于LLM的可解释性干预基准，带有参考目标）。LIBERTy 以明确定义的文本生成结构因果模型（SCMs）为基础，在概念上进行干预时，这些变化会沿着 SCM 传播直至 LLM 生成反事实。我们介绍了三个数据集（疾病检测、简历筛选和工作场所暴力预测），并提出了一种新的评估指标：顺序忠实度。通过使用这些数据集，我们在五种模型中广泛的方法范围内进行了评价，并发现提高基于概念的解释的空间仍然很大。此外，LIBERTy 还允许对模型对干预的敏感性进行系统分析：我们发现在专有 LLM 中，人口统计学概念上的敏感度显著降低，这可能是由于训练后的缓解措施所导致。总体而言，LIBERTy 为开发准确的可解释方法提供了一个急需的基准测试标准。
URL
https://arxiv.org/abs/2601.10700
PDF
-
LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning
2026-01-15 07:14:24Linquan Wu, Tianxiang Jiang, Yifei Dong, Haoyu Yang, Fengji Zhang, Shichaang Meng, Ai Xuan, Linqi Song, Jacky KeungAbstract
Current multimodal latent reasoning often relies on external supervision (e.g., auxiliary images), ignoring intrinsic visual attention dynamics. In this work, we identify a critical Perception Gap in distillation: student models frequently mimic a teacher's textual output while attending to fundamentally divergent visual regions, effectively relying on language priors rather than grounded perception. To bridge this, we propose LaViT, a framework that aligns latent visual thoughts rather than static embeddings. LaViT compels the student to autoregressively reconstruct the teacher's visual semantics and attention trajectories prior to text generation, employing a curriculum sensory gating mechanism to prevent shortcut learning. Extensive experiments show that LaViT significantly enhances visual grounding, achieving up to +16.9% gains on complex reasoning tasks and enabling a compact 3B model to outperform larger open-source variants and proprietary models like GPT-4o.
Abstract (translated)
当前的多模态潜在推理通常依赖于外部监督（例如，辅助图像），而忽略了内在的视觉注意力动态。在本工作中，我们识别出蒸馏过程中一个关键的认知差距：学生模型经常模仿教师模型的文字输出，但关注的根本是不同的视觉区域，这实际上依赖于语言先验而不是基于感知的理解。为解决这个问题，我们提出了LaViT框架，该框架对齐的是潜在的视觉思考而非静态嵌入。 LaViT 强制要求学生在生成文本之前自回归地重构教师模型的视觉语义和注意力轨迹，并采用课程感官门控机制来防止捷径学习的发生。大量的实验表明，LaViT 显著增强了视觉基础能力，在复杂的推理任务中取得了高达+16.9% 的收益，并使得较小的3B 模型能够超越更大的开源变体以及专有的模型如GPT-4o。
URL
https://arxiv.org/abs/2601.10129
PDF
-
MVSS: A Unified Framework for Multi-View Structured Survey Generation
2026-01-14 14:11:39Yinqi Liu, Yueqi Zhu, Yongkang Zhang, Xinfeng Li, Feiran Liu, Yufei Sun, Xin Wang, Renzhao Liang, Yidong Wang, Cunxiang WangAbstract
Scientific surveys require not only summarizing large bodies of literature, but also organizing them into clear and coherent conceptual structures. Existing automatic survey generation methods typically focus on linear text generation and struggle to explicitly model hierarchical relations among research topics and structured methodological comparisons, resulting in gaps in structural organization compared to expert-written surveys. We propose MVSS, a multi-view structured survey generation framework that jointly generates and aligns citation-grounded hierarchical trees, structured comparison tables, and survey text. MVSS follows a structure-first paradigm: it first constructs a conceptual tree of the research domain, then generates comparison tables constrained by the tree, and finally uses both as structural constraints for text generation. This enables complementary multi-view representations across structure, comparison, and narrative. We introduce an evaluation framework assessing structural quality, comparative completeness, and citation fidelity. Experiments on 76 computer science topics show MVSS outperforms existing methods in organization and evidence grounding, achieving performance comparable to expert surveys.
Abstract (translated)
科学研究的综述不仅需要总结大量的文献，还需要将这些文献组织成清晰连贯的概念结构。现有的自动综述生成方法通常集中在线性文本生成上，并且难以明确地建模研究主题之间的层次关系和有结构的方法比较，这导致与专家撰写的综述相比，在结构组织方面存在差距。 为此，我们提出了一种名为MVSS的多视角结构化综述生成框架。该框架旨在同时生成并对齐基于引用的层级树、结构化的对比表格以及综述文本。MVSS遵循一种“以结构为先”的方法：首先构建研究领域的概念树，然后在此基础上生成受树约束的比较表，并最终将这两者作为结构限制来辅助文本生成。这种做法可以实现跨越结构、对比和叙述的互补多视角表示。 我们还引入了一种评估框架，用于评价综述在结构性质量、对比完整性和引用忠实度方面的表现。实验结果显示，在针对76个计算机科学主题进行测试时，MVSS相较于现有方法在组织和证据支撑方面表现出色，并且其整体性能可以与专家撰写的综述相媲美。
URL
https://arxiv.org/abs/2601.09504
PDF
-
Test-Time Detoxification without Training or Learning Anything
2026-01-14 11:54:23Baturay Saglam, Dionysis KalogeriasAbstract
Large language models can produce toxic or inappropriate text even for benign inputs, creating risks when deployed at scale. Detoxification is therefore important for safety and user trust, particularly when we want to reduce harmful content without sacrificing the model's generation quality. Many existing approaches rely on model retraining, gradients, or learned auxiliary components, which can be costly and may not transfer across model families or to truly black-box settings. We introduce a test-time procedure that approximates the gradient of completion toxicity with respect to the input embeddings and uses a small number of descent steps to steer generation toward less toxic continuations. This is achieved with zeroth-order optimization that requires only access to input embeddings, a toxicity scoring function, and forward evaluations of the model. Empirically, the approach delivers robust toxicity reductions across models and prompts and, in most settings, achieves the best overall toxicity-quality trade-off. More broadly, our work positions word embeddings as effective control variables and encourages wider use of black-box optimization to guide autoregressive language models toward scalable, safer text generation, without requiring any training or access to intermediate computations.
Abstract (translated)
大型语言模型即使在接收良性输入时，也可能生成有毒或不适当的内容，在大规模部署时会带来风险。因此，为了确保安全性和用户信任，去毒化工作尤为重要，尤其是在减少有害内容的同时又不牺牲模型的生成质量的情况下。现有的许多方法依赖于重新训练、梯度计算或学习辅助组件，这些方法成本较高，并且可能无法跨不同模型家族迁移或应用于真正意义上的黑盒设置中。 我们介绍了一种在测试时进行的过程，该过程通过输入嵌入（input embeddings）来近似完成任务的毒性相对于输入变化的梯度，并使用少量下降步骤将生成引导到更少毒害性的延续。这一方法是基于零阶优化实现的，仅需访问输入嵌入、一个有毒性评分函数和模型的前向评估即可。 从经验来看，在大多数情况下，这种方法能够有效地减少各种模型和提示中的毒性内容，并且在大多数设置下实现了最佳的整体毒性与质量权衡。更为广泛而言，我们的工作将词嵌入作为有效的控制变量进行了定位，并鼓励更广泛应用黑盒优化来引导自回归语言模型实现可扩展、安全的文本生成，而无需任何训练或对中间计算的访问需求。
URL
https://arxiv.org/abs/2602.02498
PDF
-
Efficient Multilingual Dialogue Processing via Translation Pipelines and Distilled Language Models
2026-01-14 01:02:06Santiago Mart\'inez Novoa, Nicol\'as Rozo Fajardo, Diego Alejandro Gonz\'alez Vargas, Nicol\'as Bedoya FigueroaAbstract
This paper presents team Kl33n3x's multilingual dialogue summarization and question answering system developed for the NLPAI4Health 2025 shared task. The approach employs a three-stage pipeline: forward translation from Indic languages to English, multitask text generation using a 2.55B parameter distilled language model, and reverse translation back to source languages. By leveraging knowledge distillation techniques, this work demonstrates that compact models can achieve highly competitive performance across nine languages. The system achieved strong win rates across the competition's tasks, with particularly robust performance on Marathi (86.7% QnA), Tamil (86.7% QnA), and Hindi (80.0% QnA), demonstrating the effectiveness of translation-based approaches for low-resource language processing without task-specific fine-tuning.
Abstract (translated)
本文介绍了团队Kl33n3x为NLPAI4Health 2025共享任务开发的多语言对话摘要和问答系统。该方法采用三阶段管道：将印地语系语言翻译成英语，使用一个2.55B参数精简的语言模型进行多任务文本生成，以及返回源语言的逆向翻译。通过利用知识蒸馏技术，这项工作展示了紧凑型模型在九种语言中可以实现非常有竞争力的表现。该系统在整个竞赛任务中取得了强大的胜率，在马拉地语（问答部分86.7%）、泰卢固语（问答部分86.7%）和印地语（问答部分80.0%）上的表现尤为出色，证明了基于翻译的方法在处理资源不足的语言时的有效性，并且无需针对特定任务进行微调。
URL
https://arxiv.org/abs/2601.09059
PDF
-
OpenMic: A Multi-Agent-Based Stand-Up Comedy Generation System
2026-01-13 07:26:23Yuyang Wu, Hanzhong Cao, Jianhao Chen, Yufei LiAbstract
Chinese stand-up comedy generation goes beyond plain text generation, requiring culturally grounded humor, precise timing, stage-performance cues, and implicit multi-step reasoning. Moreover, commonly used Chinese humor datasets are often better suited for humor understanding and evaluation than for long-form stand-up generation, making direct supervision misaligned with the target task. To address these challenges, we present OpenMic, an end-to-end multi-agent system built on AutoGen that transforms a user-provided life topic into a 3-5 minute Chinese stand-up performance and further produces a narrated comedy video. OpenMic orchestrates multiple specialized agents in a multi-round iterative loop-planning to jointly optimize humor, timing, and performability. To mitigate the dataset-task mismatch, we augment generation with retrieval-augmented generation (RAG) for material grounding and idea expansion, and we fine-tune a dedicated JokeWriter to better internalize stand-up-specific setup-punchline structures and long-range callbacks.
Abstract (translated)
中文脱口秀的生成超越了单纯的文字生成，需要具备文化根基的幽默感、精准的节奏把握、舞台表演提示以及隐含的多步骤推理能力。此外，常用的中文幽默数据集往往更适合于理解和评估幽默，而非长期形式的脱口秀创作，这使得直接监督方法与目标任务不匹配。为了解决这些挑战，我们提出了OpenMic系统，这是一个基于AutoGen构建的端到端多代理系统，它可以将用户提供的生活话题转化为3-5分钟的中文脱口秀表演，并进一步制作成有声笑料视频。 OpenMic通过在多个回合迭代规划中协调多个专业化的代理来共同优化幽默感、节奏和可执行性。为了缓解数据集与任务之间的不匹配问题，我们在生成过程中引入了检索增强生成（RAG）技术，用于材料的定位以及观点扩展，并且训练了一个专门的JokeWriter以更好地内化脱口秀特有的铺垫-笑点结构及长程回扣技巧。 这种综合方法不仅提高了中文脱口秀内容的质量和吸引力，还确保了系统输出具有高度的文化相关性和娱乐性。
URL
https://arxiv.org/abs/2601.08288
PDF
-
Dynamical Systems Analysis Reveals Functional Regimes in Large Language Models
2026-01-11 21:57:52Hassan Ugail, Newton HowardAbstract
Large language models perform text generation through high-dimensional internal dynamics, yet the temporal organisation of these dynamics remains poorly understood. Most interpretability approaches emphasise static representations or causal interventions, leaving temporal structure largely unexplored. Drawing on neuroscience, where temporal integration and metastability are core markers of neural organisation, we adapt these concepts to transformer models and discuss a composite dynamical metric, computed from activation time-series during autoregressive generation. We evaluate this metric in GPT-2-medium across five conditions: structured reasoning, forced repetition, high-temperature noisy sampling, attention-head pruning, and weight-noise injection. Structured reasoning consistently exhibits elevated metric relative to repetitive, noisy, and perturbed regimes, with statistically significant differences confirmed by one-way ANOVA and large effect sizes in key comparisons. These results are robust to layer selection, channel subsampling, and random seeds. Our findings demonstrate that neuroscience-inspired dynamical metrics can reliably characterise differences in computational organisation across functional regimes in large language models. We stress that the proposed metric captures formal dynamical properties and does not imply subjective experience.
Abstract (translated)
大型语言模型通过高维内部动力学进行文本生成，但这些动力学的时序组织仍然鲜为人知。大多数可解释性方法侧重于静态表示或因果干预，而忽略了对时间结构的研究。借鉴神经科学中的时间整合和亚稳态是神经组织的核心标志这一概念，我们将这些概念应用于变换器模型，并讨论一种基于自回归生成过程中激活时间序列计算的复合动力学指标。我们在GPT-2-medium模型上评估了五种条件下的该指标：结构化推理、强制重复、高温噪声采样、注意力头剪枝和权重噪声注入。结构化推理在所有条件下均表现出高于重复性、噪音性和受干扰状态的显著更高指标值，这种差异通过单因素方差分析（ANOVA）得到统计学确认，并且关键比较中具有较大的效应量。这些结果对层数选择、通道子采样和随机种子的选择具有鲁棒性。我们的研究结果表明，受神经科学启发的动力学度量能够可靠地表征大型语言模型在不同功能状态下计算组织的差异。我们强调，所提出的度量捕捉的是形式化的动力学属性，并不意味着主观体验的存在。
URL
https://arxiv.org/abs/2601.11622
PDF
-
Codified Foreshadowing-Payoff Text Generation
2026-01-11 19:05:37Longfei Yun, Kun Zhou, Yupeng Hou, Letian Peng, Jingbo ShangAbstract
Foreshadowing and payoff are ubiquitous narrative devices through which authors introduce commitments early in a story and resolve them through concrete, observable outcomes. However, despite advances in story generation, large language models (LLMs) frequently fail to bridge these long-range narrative dependencies, often leaving "Chekhov's guns" unfired even when the necessary context is present. Existing evaluations largely overlook this structural failure, focusing on surface-level coherence rather than the logical fulfillment of narrative setups. In this paper, we introduce Codified Foreshadowing-Payoff Generation (CFPG), a novel framework that reframes narrative quality through the lens of payoff realization. Recognizing that LLMs struggle to intuitively grasp the "triggering mechanism" of a foreshadowed event, CFPG transforms narrative continuity into a set of executable causal predicates. By mining and encoding Foreshadow-Trigger-Payoff triples from the BookSum corpus, we provide structured supervision that ensures foreshadowed commitments are not only mentioned but also temporally and logically fulfilled. Experiments demonstrate that CFPG significantly outperforms standard prompting baselines in payoff accuracy and narrative alignment. Our findings suggest that explicitly codifying narrative mechanics is essential for moving LLMs from surface-level fluency to genuine narrative competence.
Abstract (translated)
预示和回报是作者在故事早期引入承诺并在后续通过具体可观察结果解决这些承诺的普遍叙述手段。然而，尽管故事情节生成技术有所进步，大型语言模型（LLM）仍然经常无法跨越这些长跨度叙事依赖关系，即使存在必要背景时也常常未能触发“契诃夫之枪”，导致故事中的线索未被利用。现有的评估方法大多忽视这种结构性失败，关注的是表面连贯性而非叙述设定的逻辑实现。 在本文中，我们引入了编码预示-回报生成（Codified Foreshadowing-Payoff Generation, CFPG），这是一个通过实现回报来重新定义叙事质量的新框架。认识到LLM难以直观地理解预示事件的“触发机制”，CFPG将叙事连贯性转化为一组可执行的因果谓词。通过对BookSum语料库中的预示-触发-回报三元组进行挖掘和编码，我们提供了一种结构化监督方法，确保不仅提到预示承诺，而且在时间和逻辑上实现它们。实验结果表明，CFPG在回报准确性和叙事一致性方面显著优于标准提示基准模型。 我们的研究结果表明，明确编码叙述机制对于将LLM从表面流畅性提升到真正的叙事能力至关重要。
URL
https://arxiv.org/abs/2601.07033
PDF
-
Characterising Toxicity in Generative Large Language Models
2026-01-10 21:50:05Zhiyao Zhang, Yazan Mash'Al, Yuhan WuAbstract
In recent years, the advent of the attention mechanism has significantly advanced the field of natural language processing (NLP), revolutionizing text processing and text generation. This has come about through transformer-based decoder-only architectures, which have become ubiquitous in NLP due to their impressive text processing and generation capabilities. Despite these breakthroughs, language models (LMs) remain susceptible to generating undesired outputs: inappropriate, offensive, or otherwise harmful responses. We will collectively refer to these as ``toxic'' outputs. Although methods like reinforcement learning from human feedback (RLHF) have been developed to align model outputs with human values, these safeguards can often be circumvented through carefully crafted prompts. Therefore, this paper examines the extent to which LLMs generate toxic content when prompted, as well as the linguistic factors -- both lexical and syntactic -- that influence the production of such outputs in generative models.
Abstract (translated)
近年来，注意力机制的出现显著推动了自然语言处理（NLP）领域的发展，彻底革新了文本处理和文本生成。这得益于基于变压器的解码器架构，由于其出色的文本处理和生成能力而成为NLP领域的主流选择。尽管取得了这些突破性进展，语言模型（LMs）仍然容易产生不希望出现的内容：不合适、冒犯性的或具有潜在危害的响应。我们将这些统称为“有毒”输出。虽然诸如从人类反馈中进行强化学习（RLHF）等方法已经开发出来以使模型输出与人类价值观保持一致，但通过精心设计的提示语，这些防护措施常常可以被规避。因此，本文探讨了大型语言模型在接收到特定提示时产生有害内容的程度，以及影响生成此类输出的语言因素——包括词汇和句法方面的因素。
URL
https://arxiv.org/abs/2601.06700
PDF
-
Can a Unimodal Language Agent Provide Preferences to Tune a Multimodal Vision-Language Model?
2026-01-10 04:28:53Sazia Tabasum Mim, Jack Morris, Manish Dhakal, Yanming Xiu, Maria Gorlatova, Yi DingAbstract
To explore a more scalable path for adding multimodal capabilities to existing LLMs, this paper addresses a fundamental question: Can a unimodal LLM, relying solely on text, reason about its own informational needs and provide effective feedback to optimize a multimodal model? To answer this, we propose a method that enables a language agent to give feedback to a vision-language model (VLM) to adapt text generation to the agent's preferences. Our results from different experiments affirm this hypothesis, showing that LLM preference feedback significantly enhances VLM descriptions. Using our proposed method, we find that the VLM can generate multimodal scene descriptions to help the LLM better understand multimodal context, leading to improvements of maximum 13% in absolute accuracy compared to the baseline multimodal approach. Furthermore, a human study validated our AI-driven feedback, showing a 64.6% preference alignment rate between the LLM's choices and human judgments. Extensive experiments provide insights on how and why the method works and its limitations.
Abstract (translated)
为了探索将多模态功能添加到现有大规模语言模型（LLM）的更可扩展路径，本文提出并回答了一个基本问题：一个仅依赖文本的一模态LLM是否能够根据自己的信息需求进行推理，并提供有效反馈以优化多模态模型？为解答这一问题，我们提出了一种方法，该方法使语言代理能够向视觉-语言模型（VLM）提供反馈，从而适应其偏好调整文本生成。来自不同实验的结果证实了这个假设，表明LLM偏好的反馈显著增强了VLM的描述效果。使用我们提出的这种方法，发现VLM可以生成多模态场景描述，以帮助LLM更好地理解多模态背景信息，与基线方法相比，在绝对准确性方面最多可提高13%。此外，一项人类研究验证了我们的AI驱动反馈的有效性，结果显示在LLM选择和人工判断之间达到了64.6%的偏好一致性率。广泛的实验提供了关于该方法如何以及为何有效及其局限性的见解。
URL
https://arxiv.org/abs/2601.06424
PDF
-
Left, Right, or Center? Evaluating LLM Framing in News Classification and Generation
2026-01-09 15:10:11Molly Kennedy, Ali Parker, Yihong Liu, Hinrich Sch\"utzeAbstract
Large Language Model (LLM) based summarization and text generation are increasingly used for producing and rewriting text, raising concerns about political framing in journalism where subtle wording choices can shape interpretation. Across nine state-of-the-art LLMs, we study political framing by testing whether LLMs' classification-based bias signals align with framing behavior in their generated summaries. We first compare few-shot ideology predictions against LEFT/CENTER/RIGHT labels. We then generate "steered" summaries under FAITHFUL, CENTRIST, LEFT, and RIGHT prompts, and score all outputs using a single fixed ideology evaluator. We find pervasive ideological center-collapse in both article-level ratings and generated text, indicating a systematic tendency toward centrist framing. Among evaluated models, Grok 4 is by far the most ideologically expressive generator, while Claude Sonnet 4.5 and Llama 3.1 achieve the strongest bias-rating performance among commercial and open-weight models, respectively.
Abstract (translated)
基于大型语言模型（LLM）的摘要和文本生成在制作和重写文本中的使用日益增加，引发了对新闻报道中政治框架问题的关注。细微的语言选择可能会塑造解释方式。通过对九种最先进的LLM进行研究，我们探讨了这些模型的政治框架倾向，方法是测试它们基于分类的偏见信号是否与其生成总结时的行为相符合。首先，我们将少量示例意识形态预测与LEFT/CENTER/RIGHT标签进行比较。然后，在FAITHFUL（忠实）、CENTRIST（中间派）、LEFT（左翼）和RIGHT（右翼）提示下生成“导向”摘要，并使用单一固定意识形态评估器对所有输出评分。 我们发现，无论是文章级别的评级还是生成的文本中都普遍存在意识形态上的中心化倾向，这表明这些模型倾向于采取中间立场。在经过评估的模型中，Grok 4 在表达意识形态方面表现最为突出，而Claude Sonnet 4.5 和Llama 3.1 分别是商业和开源模型中在偏见评级性能上最强的。
URL
https://arxiv.org/abs/2601.05835
PDF
-
Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs
2026-01-09 13:46:52Eilam Cohen, Itamar Bul, Danielle Inbar, Omri LoewenbachAbstract
Large language models (LLMs) enable strong text generation, and in general there is a practical tradeoff between fine-tuning and prompt engineering. We introduce Simplify-This, a comparative study evaluating both paradigms for text simplification with encoder-decoder LLMs across multiple benchmarks, using a range of evaluation metrics. Fine-tuned models consistently deliver stronger structural simplification, whereas prompting often attains higher semantic similarity scores yet tends to copy inputs. A human evaluation favors fine-tuned outputs overall. We release code, a cleaned derivative dataset used in our study, checkpoints of fine-tuned models, and prompt templates to facilitate reproducibility and future work.
Abstract (translated)
大型语言模型（LLMs）能够实现强大的文本生成，在一般情况下，微调和提示工程之间存在实际的权衡。我们介绍了一项名为“Simplify-This”的比较研究，该研究评估了使用编码器-解码器LLMs在多个基准测试中针对文本简化进行微调与提示这两种方法的效果，并采用了多种评价指标。微调模型始终能提供更强的结构化简化效果，而通过提示往往能达到更高的语义相似度得分，但却倾向于复制输入内容。从人类评估的角度来看，总体上更偏好微调后的输出结果。我们发布了代码、用于研究的清理过的衍生数据集、微调模型的检查点以及提示模板以促进可重复性和未来的研究工作。
URL
https://arxiv.org/abs/2601.05794
PDF
-
LLMs-Integrated Automatic Hate Speech Recognition Using Controllable Text Generation Models
2026-01-08 07:03:48Ryutaro Oshima, Yuya Hosoda, Youji IiguniAbstract
This paper proposes an automatic speech recognition (ASR) model for hate speech using large language models (LLMs). The proposed method integrates the encoder of the ASR model with the decoder of the LLMs, enabling simultaneous transcription and censorship tasks to prevent the exposure of harmful content. Instruction tuning of the LLM to mask hate-related words with specific tokens requires an annotated hate speech dataset, which is limited. We generate text samples using an LLM with the Chain-of-Thought (CoT) prompting technique guided by cultural context and examples and then convert them into speech samples using a text-to-speech (TTS) system. However, some of them contain non-hate speech samples with hate-related words, which degrades the censorship performance. This paper filters the samples which text classification models correctly label as hate content. By adjusting the threshold for the number of correct answer models, we can control the level of hate in the generated dataset, allowing us to train the LLMs through curriculum learning in a gradual manner. Experimental results show that the proposed method achieves a masking accuracy of 58.6\% for hate-related words, surpassing previous baselines. We also confirm that the curriculum training contributes to the efficiency of both transcription and censorship tasks.
Abstract (translated)
本文提出了一种使用大型语言模型（LLM）进行仇恨言论自动语音识别（ASR）的模型。该方法将ASR模型的编码器与LLMs的解码器相结合，使得转录和审查任务能够同时执行，以防止有害内容的传播。为了通过特定标记屏蔽与仇恨相关的词汇对LLMs进行指令微调，需要一个带有标注的仇恨言论数据集，但这样的数据集较为有限。我们使用了大型语言模型，并结合了“思维链”（CoT）提示技术，在文化背景和示例引导下生成文本样本，然后通过文字转语音（TTS）系统将其转换为语音样本。然而，部分生成的样本中包含含有仇恨相关词汇但不属于仇恨言论的非仇恨言论样本，这降低了审查性能。本文的方法筛选出被文本分类模型正确标注为仇恨内容的样本。通过调整判定为正确答案的模型数量阈值，我们可以控制生成数据集中的仇恨程度，从而允许我们逐步进行课程学习训练LLMs。实验结果显示，该方法在与仇恨相关的词汇屏蔽准确率上达到了58.6%，超过了之前的基准水平。此外，我们也确认了课程训练对于转录和审查任务的有效性有所提升。
URL
https://arxiv.org/abs/2601.04654
PDF
-
Web Retrieval-Aware Chunking for Efficient and Cost-Effective Retrieval-Augmented Generation Systems
2026-01-08 06:41:59Uday Allu, Sonu Kedia, Tanmay Odapally, Biddwan AhmedAbstract
Retrieval-Augmented Generation (RAG) systems critically depend on effective document chunking strategies to balance retrieval quality, latency, and operational cost. Traditional chunking approaches, such as fixed-size, rule-based, or fully agentic chunking, often suffer from high token consumption, redundant text generation, limited scalability, and poor debuggability, especially for large-scale web content ingestion. In this paper, we propose Web Retrieval-Aware Chunking (W-RAC), a novel, cost-efficient chunking framework designed specifically for web-based documents. W-RAC decouples text extraction from semantic chunk planning by representing parsed web content as structured, ID-addressable units and leveraging large language models (LLMs) only for retrieval-aware grouping decisions rather than text generation. This significantly reduces token usage, eliminates hallucination risks, and improves system this http URL analysis and architectural comparison demonstrate that W-RAC achieves comparable or better retrieval performance than traditional chunking approaches while reducing chunking-related LLM costs by an order of magnitude.
Abstract (translated)
检索增强生成（RAG）系统严重依赖于有效的文档分块策略，以平衡检索质量、延迟和运营成本。传统的分块方法（如固定大小、基于规则或完全智能体的分块）通常存在令牌消耗高、文本生成冗余、可扩展性有限以及可调试性差等问题，尤其在大规模网络内容摄入场景下更为突出。本文提出了一种面向网络文档的新型低成本分块框架——网络检索感知分块（W-RAC）。W-RAC通过将解析后的网络内容表示为结构化、ID可寻址的单元，并仅利用大语言模型（LLM）进行检索感知的分组决策（而非文本生成），实现了文本提取与语义分块规划的分离。这显著降低了令牌使用量，消除了幻觉风险，并提升了系统可扩展性和可调试性。系统分析与架构对比表明，W-RAC在实现与传统分块方法相当或更优的检索性能的同时，将分块相关的LLM成本降低了一个数量级。
URL
https://arxiv.org/abs/2604.04936
PDF
-
BanglaLorica: Design and Evaluation of a Robust Watermarking Algorithm for Large Language Models in Bangla Text Generation
2026-01-08 03:01:59Amit Bin Tariqul, A N M Zahid Hossain Milkan, Sahab-Al-Chowdhury, Syed Rifat Raiyan, Hasan Mahmud, Md Kamrul HasanAbstract
As large language models (LLMs) are increasingly deployed for text generation, watermarking has become essential for authorship attribution, intellectual property protection, and misuse detection. While existing watermarking methods perform well in high-resource languages, their robustness in low-resource languages remains underexplored. This work presents the first systematic evaluation of state-of-the-art text watermarking methods: KGW, Exponential Sampling (EXP), and Waterfall, for Bangla LLM text generation under cross-lingual round-trip translation (RTT) attacks. Under benign conditions, KGW and EXP achieve high detection accuracy (>88%) with negligible perplexity and ROUGE degradation. However, RTT causes detection accuracy to collapse below RTT causes detection accuracy to collapse to 9-13%, indicating a fundamental failure of token-level watermarking. To address this, we propose a layered watermarking strategy that combines embedding-time and post-generation watermarks. Experimental results show that layered watermarking improves post-RTT detection accuracy by 25-35%, achieving 40-50% accuracy, representing a 3$\times$ to 4$\times$ relative improvement over single-layer methods, at the cost of controlled semantic degradation. Our findings quantify the robustness-quality trade-off in multilingual watermarking and establish layered watermarking as a practical, training-free solution for low-resource languages such as Bangla. Our code and data will be made public.
Abstract (translated)
随着大规模语言模型（LLMs）在文本生成中的应用日益广泛，水印技术已成为作者归属认定、知识产权保护和滥用检测的重要手段。尽管现有的水印方法在高资源语言中表现良好，但在低资源语言中的鲁棒性仍需进一步探索。本文首次系统地评估了三种最先进的文本水印方法（KGW、指数采样法EXP 和 Waterfall）在孟加拉语LLM文本生成过程中跨语言往返翻译(RTT) 攻击下的性能。 在无攻击的正常情况下，KGW和EXP 方法可以实现超过88%的高检测准确率，并且不会导致困惑度(perplexity)或ROUGE值的显著下降。然而，在RTT攻击下，水印的检测准确率急剧降至9-13%，这表明基于单令牌的水印技术在跨语言环境中失效。 为解决这一问题，我们提出了一种分层水印策略，该策略结合了嵌入时间和生成后两种类型的水印方法。实验结果显示，采用分层水印可以将RTT攻击后的检测准确率提高25-35%，达到40-50%的水平，相对于单层方法提高了3到4倍的效果，但代价是可控语义退化。 我们的研究量化了多语言水印技术中鲁棒性和质量之间的权衡，并证明分层水印是一种适用于低资源语言（如孟加拉语）的实际且无需重新训练的方法。我们将在后期公开代码和数据。
URL
https://arxiv.org/abs/2601.04534
PDF
-
Evaluating Small Decoder-Only Language Models for Grammar Correction and Text Simplification
Abstract
Large language models have become extremely popular recently due to their ability to achieve strong performance on a variety of tasks, such as text generation and rewriting, but their size and computation cost make them difficult to access, deploy, and secure in many settings. This paper investigates whether small, decoder-only language models can provide an efficient alternative for the tasks of grammar correction and text simplification. The experiments in this paper focus on testing small language models out of the box, fine-tuned, and run sequentially on the JFLEG and ASSET datasets using established metrics. The results show that while SLMs may learn certain behaviors well, their performance remains below strong baselines and current LLMs. The results also show that SLMs struggle with retaining meaning and hallucinations. These findings suggest that despite their efficiency advantages, current SLMs are not yet competitive enough with modern LLMs for rewriting, and further advances in training are required for SLMs to close the performance gap between them and today's LLMs.
Abstract (translated)
近期，大型语言模型由于在文本生成、重写等多种任务上表现出色而变得非常流行。然而，它们的庞大体积和计算成本使得这些模型难以在许多环境中获取、部署和保护。本文研究了小型解码器独享的语言模型是否可以作为语法纠正和文本简化任务的有效替代方案。论文中的实验侧重于测试未经调整、经过微调以及顺序运行的小型语言模型（SLMs）在JFLEG和ASSET数据集上使用公认度量标准下的表现。 结果显示，虽然小型语言模型可能能够很好地学习某些行为，但它们的表现仍然低于强大的基准线和当前的大型语言模型。此外，实验表明，小型语言模型在保留意义和减少幻觉方面存在困难。这些发现表明，尽管小型语言模型具有效率优势，但在文本重写领域目前尚不足以与现代大型语言模型竞争，并且需要进一步的技术进步来缩小它们之间的性能差距。
URL
https://arxiv.org/abs/2601.03874
PDF
-
Evaluation of Multilingual LLMs Personalized Text Generation Capabilities Targeting Groups and Social-Media Platforms
Abstract
Capabilities of large language models to generate multilingual coherent text have continuously enhanced in recent years, which opens concerns about their potential misuse. Previous research has shown that they can be misused for generation of personalized disinformation in multiple languages. It has also been observed that personalization negatively affects detectability of machine-generated texts; however, this has been studied in the English language only. In this work, we examine this phenomenon across 10 languages, while we focus not only on potential misuse of personalization capabilities, but also on potential benefits they offer. Overall, we cover 1080 combinations of various personalization aspects in the prompts, for which the texts are generated by 16 distinct language models (17,280 texts in total). Our results indicate that there are differences in personalization quality of the generated texts when targeting demographic groups and when targeting social-media platforms across languages. Personalization towards platforms affects detectability of the generated texts in a higher scale, especially in English, where the personalization quality is the highest.
Abstract (translated)
近年来，大型语言模型生成多语种连贯文本的能力不断提升，这引发了对其潜在误用的担忧。先前的研究表明，它们可以被用于多种语言中制造个性化虚假信息。还观察到，个性化会降低机器生成文本的可检测性；然而，这一现象仅在英语中得到了研究。在这项工作中，我们在10种不同的语言环境中考察了这种现象，并不仅关注个性化能力可能带来的潜在误用问题，也探讨它们所提供的潜在益处。总体而言，我们覆盖了各种提示中的个性化方面的1080种组合，这些文本由16个不同的语言模型生成（总计17,280篇）。我们的研究结果表明，在针对不同的人口群体和社交媒体平台时，所生成文本的个性化质量在不同语言中存在差异。面向平台进行个性化尤其会对生成文本的可检测性产生较大影响，尤其是在英语环境中，其个性化质量最高。
URL
https://arxiv.org/abs/2601.03752
PDF
-
Evaluation Framework for AI Creativity: A Case Study Based on Story Generation
2026-01-07 08:31:08Pharath Sathya, Yin Jou Huang, Fei ChengAbstract
Evaluating creative text generation remains a challenge because existing reference-based metrics fail to capture the subjective nature of creativity. We propose a structured evaluation framework for AI story generation comprising four components (Novelty, Value, Adherence, and Resonance) and eleven sub-components. Using controlled story generation via ``Spike Prompting'' and a crowdsourced study of 115 readers, we examine how different creative components shape both immediate and reflective human creativity judgments. Our findings show that creativity is evaluated hierarchically rather than cumulatively, with different dimensions becoming salient at different stages of judgment, and that reflective evaluation substantially alters both ratings and inter-rater agreement. Together, these results support the effectiveness of our framework in revealing dimensions of creativity that are obscured by reference-based evaluation.
Abstract (translated)
评估创意文本生成仍然是一个挑战，因为现有的基于参考的度量标准无法捕捉到创造力的主观性质。我们提出了一种用于AI故事生成的结构化评估框架，该框架由四个组成部分（新颖性、价值、一致性以及共鸣）和十一项子组成部分构成。通过使用“脉冲提示”进行控制的故事生成，并结合115名读者的众包研究，我们探讨了不同的创意组件如何影响人类在即时与反思阶段的创造力评价。我们的发现表明，创造力是分层次而非累加地被评估的，在判断的不同阶段会出现不同维度的重要性。此外，反思性评价显著改变了评分及评判者之间的共识。这些结果共同支持了该框架的有效性，因为它揭示了基于参考评估所忽略的创意维度。
URL
https://arxiv.org/abs/2601.03698
PDF
-
eTracer: Towards Traceable Text Generation via Claim-Level Grounding
2026-01-07 07:44:30Bohao Chu, Qianli Wang, Hendrik Damm, Hui Wang, Ula Muhabbek, Elisabeth Livingstone, Christoph M. Friedrich, Norbert FuhrAbstract
How can system-generated responses be efficiently verified, especially in the high-stakes biomedical domain? To address this challenge, we introduce eTracer, a plug-and-play framework that enables traceable text generation by grounding claims against contextual evidence. Through post-hoc grounding, each response claim is aligned with contextual evidence that either supports or contradicts it. Building on claim-level grounding results, eTracer not only enables users to precisely trace responses back to their contextual source but also quantifies response faithfulness, thereby enabling the verifiability and trustworthiness of generated responses. Experiments show that our claim-level grounding approach alleviates the limitations of conventional grounding methods in aligning generated statements with contextual sentence-level evidence, resulting in substantial improvements in overall grounding quality and user verification efficiency. The code and data are available at this https URL.
Abstract (translated)
如何高效验证系统生成的响应，尤其是在高风险的生命科学领域？为应对这一挑战，我们引入了eTracer，这是一个即插即用框架，通过将声明与上下文证据进行对齐来实现可追溯的文本生成。通过事后对齐方式，每个响应声明都会与其支持或反驳它的上下文证据相匹配。基于声明级别的对齐结果，eTracer不仅使用户能够准确地追踪响应到其上下文来源，而且还量化了响应的真实性，从而实现了生成响应的验证性和可信度。实验表明，我们提出的声明级别对齐方法缓解了传统对齐方法在将生成语句与上下文句子级证据相匹配方面的局限性，这极大地提高了整体对齐质量和用户验证效率。代码和数据可在以下链接获取：[此URL]（请将此处的"this https URL"替换为实际提供的有效链接）。
URL
https://arxiv.org/abs/2601.03669
PDF
-
FlowPlan-G2P: A Structured Generation Framework for Transforming Scientific Papers into Patent Descriptions
Abstract
Over 3.5 million patents are filed annually, with drafting patent descriptions requiring deep technical and legal expertise. Transforming scientific papers into patent descriptions is particularly challenging due to their differing rhetorical styles and stringent legal requirements. Unlike black-box text-to-text approaches that struggle to model structural reasoning and legal constraints, we propose FlowPlan-G2P, a novel framework that mirrors the cognitive workflow of expert drafters by reformulating this task into three stages: (1) Concept Graph Induction, extracting technical entities and relationships into a directed graph via expert-like reasoning; (2) Paragraph and Section Planning, reorganizing the graph into coherent clusters aligned with canonical patent sections; and (3) Graph-Conditioned Generation, producing legally compliant paragraphs using section-specific subgraphs and tailored prompts. Experiments demonstrate that FlowPlan-G2P significantly improves logical coherence and legal compliance over end-to-end LLM baselines. Our framework establishes a new paradigm for paper-to-patent generation and advances structured text generation for specialized domains.
Abstract (translated)
每年有超过350万项专利申请，撰写专利描述需要深厚的技术和法律专业知识。将科技论文转化为专利说明书尤其具有挑战性，这主要是因为它们在修辞风格上存在差异，并且要满足严格的法律规定。不同于那些难以建模结构化推理及法律约束的黑箱文本到文本的方法，我们提出了FlowPlan-G2P这一全新框架，它通过模仿专家撰稿人的认知工作流程，将任务分解为三个阶段来处理： 1. **概念图诱导**：利用类似专家的推理方式，从科技论文中提取技术实体和关系，并将其组织成一个有向图。 2. **段落与章节规划**：重新排列这些节点和边以形成一系列相互关联且连贯的小簇，并将它们对齐到标准专利章节之下。 3. **条件图生成**：使用特定于每个章节的子图以及定制提示，生成符合法律规定的段落。 实验表明，FlowPlan-G2P在逻辑一致性和法律合规性方面相比端到端LLM基准有了显著提高。我们所提出的框架为论文转专利文本生成设立了新的范式，并推动了专门领域内的结构化文本生成技术的发展。
URL
https://arxiv.org/abs/2601.02589
PDF
