Text_Generation
-
LLM Agents Implement an NLG System from Scratch: Building Interpretable Rule-Based RDF-to-Text Generators
2025-12-20 13:16:51Mateusz Lango, Ond\v{r}ej Du\v{s}ekAbstract
We present a novel neurosymbolic framework for RDF-to-text generation, in which the model is "trained" through collaborative interactions among multiple LLM agents rather than traditional backpropagation. The LLM agents produce rule-based Python code for a generator for the given domain, based on RDF triples only, with no in-domain human reference texts. The resulting system is fully interpretable, requires no supervised training data, and generates text nearly instantaneously using only a single CPU. Our experiments on the WebNLG and OpenDialKG data show that outputs produced by our approach reduce hallucination, with only slight fluency penalties compared to finetuned or prompted language models
Abstract (translated)
我们提出了一种新颖的神经符号框架，用于从RDF（资源描述框架）生成文本。在这个框架中，模型是通过多个大型语言模型代理之间的协作交互来“训练”的，而不是传统的反向传播方法。这些LLM代理仅基于RDF三元组生成基于规则的Python代码，而不需要特定领域的参考文本。结果系统完全可解释，无需监督学习数据，并且使用单个CPU几乎可以即时生成文本。我们在WebNLG和OpenDialKG数据集上的实验表明，与经过微调或提示的语言模型相比，我们的方法产生的输出减少了幻觉（即不准确的或虚构的信息），仅在流畅性方面略有牺牲。
URL
https://arxiv.org/abs/2512.18360
PDF
-
Region-Constraint In-Context Generation for Instructional Video Editing
2025-12-19 14:49:30Zhongwei Zhang, Fuchen Long, Wei Li, Zhaofan Qiu, Wu Liu, Ting Yao, Tao MeiAbstract
The In-context generation paradigm recently has demonstrated strong power in instructional image editing with both data efficiency and synthesis quality. Nevertheless, shaping such in-context learning for instruction-based video editing is not trivial. Without specifying editing regions, the results can suffer from the problem of inaccurate editing regions and the token interference between editing and non-editing areas during denoising. To address these, we present ReCo, a new instructional video editing paradigm that novelly delves into constraint modeling between editing and non-editing regions during in-context generation. Technically, ReCo width-wise concatenates source and target video for joint denoising. To calibrate video diffusion learning, ReCo capitalizes on two regularization terms, i.e., latent and attention regularization, conducting on one-step backward denoised latents and attention maps, respectively. The former increases the latent discrepancy of the editing region between source and target videos while reducing that of non-editing areas, emphasizing the modification on editing area and alleviating outside unexpected content generation. The latter suppresses the attention of tokens in the editing region to the tokens in counterpart of the source video, thereby mitigating their interference during novel object generation in target video. Furthermore, we propose a large-scale, high-quality video editing dataset, i.e., ReCo-Data, comprising 500K instruction-video pairs to benefit model training. Extensive experiments conducted on four major instruction-based video editing tasks demonstrate the superiority of our proposal.
Abstract (translated)
最近，基于上下文生成的范式在数据效率和合成质量方面展示了强大的图像编辑能力。然而，将这种基于上下文的学习方法应用于指令驱动的视频编辑并非易事。如果没有指定编辑区域，在去噪过程中会遇到不准确的编辑区域问题以及编辑区域与非编辑区域之间令牌干扰的问题。为了解决这些问题，我们提出了ReCo（重新约束），这是一种新的指令式视频编辑范例，它新颖地在上下文生成期间探讨了编辑区域和非编辑区域之间的约束建模。 技术上讲，ReCo 横向连接源视频和目标视频进行联合去噪处理。为了校准视频扩散学习，ReCo 利用了两种正则化项：即潜伏正则化（latent regularization）和注意力正则化（attention regularization），分别作用于一步逆去噪后的潜在特征和注意力图上。前者增加了源视频与目标视频之间编辑区域的潜在差异，同时减少了非编辑区域的潜在差异，强调了对编辑区域进行修改的重要性，并减轻了外部未预期内容生成的问题。后者抑制了编辑区域内令牌对源视频相应位置令牌的关注度，从而在目标视频中的新对象生成过程中减少了它们之间的干扰。 此外，我们还提出了一套大规模、高质量的视频编辑数据集，即ReCo-Data，包含50万条指令-视频配对的数据以支持模型训练。我们在四个主要基于指令的视频编辑任务上进行了一系列实验，证明了我们的方法具有显著优势。
URL
https://arxiv.org/abs/2512.17650
PDF
-
XLM: A Python package for non-autoregressive language models
2025-12-18 21:05:10Dhruvesh Patel, Durga Prasad Maram, Sai Sreenivas Chintha, Benjamin Rozonoyer, Andrew McCallumAbstract
In recent years, there has been a resurgence of interest in non-autoregressive text generation in the context of general language modeling. Unlike the well-established autoregressive language modeling paradigm, which has a plethora of standard training and inference libraries, implementations of non-autoregressive language modeling have largely been bespoke making it difficult to perform systematic comparisons of different methods. Moreover, each non-autoregressive language model typically requires it own data collation, loss, and prediction logic, making it challenging to reuse common components. In this work, we present the XLM python package, which is designed to make implementing small non-autoregressive language models faster with a secondary goal of providing a suite of small pre-trained models (through a companion xlm-models package) that can be used by the research community. The code is available at this https URL.
Abstract (translated)
近年来，非自回归文本生成在通用语言模型领域的兴趣重新兴起。与已经成熟的自回归语言建模范式不同，后者有大量的标准训练和推理库，而非自回归语言模型的实现大多数是定制化的，这使得对不同方法进行系统的比较变得困难。此外，每一个非自回归语言模型通常都需要自己的数据收集、损失函数和预测逻辑，从而难以重用通用组件。在本工作中，我们介绍了XLM Python包，它旨在加快小型非自回归语言模型的实现速度，并且其次要目标是提供一系列可以通过配套的xlm-models包使用的预训练小模型，以供研究社区使用。代码可以在[这个链接](https://this https URL)找到。
URL
https://arxiv.org/abs/2512.17065
PDF
-
Kling-Omni Technical Report
2025-12-18 17:08:12Kling Team, Jialu Chen, Yuanzheng Ci, Xiangyu Du, Zipeng Feng, Kun Gai, Sainan Guo, Feng Han, Jingbin He, Kang He, Xiao Hu, Xiaohua Hu, Boyuan Jiang, Fangyuan Kong, Hang Li, Jie Li, Qingyu Li, Shen Li, Xiaohan Li, Yan Li, Jiajun Liang, Borui Liao, Yiqiao Liao, Weihong Lin, Quande Liu, Xiaokun Liu, Yilun Liu, Yuliang Liu, Shun Lu, Hangyu Mao, Yunyao Mao, Haodong Ouyang, Wenyu Qin, Wanqi Shi, Xiaoyu Shi, Lianghao Su, Haozhi Sun, Peiqin Sun, Pengfei Wan, Chao Wang, Chenyu Wang, Meng Wang, Qiulin Wang, Runqi Wang, Xintao Wang, Xuebo Wang, Zekun Wang, Min Wei, Tiancheng Wen, Guohao Wu, Xiaoshi Wu, Zhenhua Wu, Da Xie, Yingtong Xiong, Yulong Xu, Sile Yang, Zikang Yang, Weicai Ye, Ziyang Yuan, Shenglong Zhang, Shuaiyu Zhang, Yuanxing Zhang, Yufan Zhang, Wenzheng Zhao, Ruiliang Zhou, Yan Zhou, Guosheng Zhu, Yongjie ZhuAbstract
We present Kling-Omni, a generalist generative framework designed to synthesize high-fidelity videos directly from multimodal visual language inputs. Adopting an end-to-end perspective, Kling-Omni bridges the functional separation among diverse video generation, editing, and intelligent reasoning tasks, integrating them into a holistic system. Unlike disjointed pipeline approaches, Kling-Omni supports a diverse range of user inputs, including text instructions, reference images, and video contexts, processing them into a unified multimodal representation to deliver cinematic-quality and highly-intelligent video content creation. To support these capabilities, we constructed a comprehensive data system that serves as the foundation for multimodal video creation. The framework is further empowered by efficient large-scale pre-training strategies and infrastructure optimizations for inference. Comprehensive evaluations reveal that Kling-Omni demonstrates exceptional capabilities in in-context generation, reasoning-based editing, and multimodal instruction following. Moving beyond a content creation tool, we believe Kling-Omni is a pivotal advancement toward multimodal world simulators capable of perceiving, reasoning, generating and interacting with the dynamic and complex worlds.
Abstract (translated)
我们介绍了Kling-Omni，这是一个专为从多模态视觉语言输入直接合成高保真视频而设计的通用生成框架。通过端到端的方法视角，Kling-Omni弥合了各种视频生成、编辑和智能推理任务之间的功能分离，并将它们整合成一个整体系统。与独立的工作流程方法不同，Kling-Omni支持多样化的用户输入形式，包括文本指令、参考图像和视频上下文信息，并将其处理为统一的多模态表示，以交付电影级质量和高度智能化的视频内容创作。为了支持这些能力，我们构建了一个全面的数据系统，作为多模态视频创作的基础。此外，该框架通过高效的大规模预训练策略和推理基础设施优化得到了进一步增强。 综合评估表明，Kling-Omni在上下文生成、基于推理的编辑以及多模态指令遵循方面表现出了卓越的能力。超越了单纯的内容创建工具角色，我们认为Kling-Omni是向具备感知、推理、生成及与动态复杂世界互动能力的多模态世界模拟器迈进的重要进步。
URL
https://arxiv.org/abs/2512.16776
PDF
-
Towards Nepali-language LLMs: Efficient GPT training with a Nepali BPE tokenizer
2025-12-16 16:53:11Adarsha Shrestha, Basanta Pokharel, Binit Shrestha, Smriti Adhikari, Dinesh GotheAbstract
Nepali, a low-resource language spoken by over 32 million people, continues to face challenges in natural language processing (NLP) due to its complex grammar, agglutinative morphology, and limited availability of high-quality corpora. Most efforts to date have centered on basic encoder architectures; they remain insufficient for Nepali-specific text generation. This study presents a GPT-2-based Nepali language model trained using several training strategies inspired by GPT-3, including optimized learning rate schedules, batch scaling, and architectural refinements. A custom 16k Byte-Pair Encoding (BPE) tokenizer was trained exclusively on Nepali text to ensure more consistent segmentation and improved input representation. The model was pretrained on a combined dataset comprising a 10.75GB cleaned NepBERTa corpus and additional web-scraped Nepali news articles. FlashAttention was integrated to reduce memory usage and stabilize training. After two epochs, the model achieved a training loss of 3.168177, a validation loss of 3.081982, and a final perplexity of 21.80, demonstrating its capability to generate coherent Nepali news-style text.
Abstract (translated)
尼泊尔语是一种资源较少的语言，全世界有超过三千万人使用。由于其复杂的语法、黏着形态学以及高质量语料库的稀缺性，自然语言处理（NLP）领域在处理尼泊尔语时仍面临诸多挑战。迄今为止，大多数努力集中在基本编码架构上；然而，这些方法对于特定于尼泊尔语的文本生成来说依然不够充分。本研究提出了一种基于GPT-2的尼泊尔语语言模型，并采用了几种受GPT-3启发的训练策略进行训练，包括优化的学习率计划、批量缩放以及架构改进。为了确保更一致的分词和改善输入表示，我们专门为尼泊尔文本训练了一个定制的16k字节对编码（BPE）词元化器。模型在包含经清理后的NepBERTa语料库（大小为10.75GB）以及额外抓取的尼泊尔新闻文章所组成的组合数据集上进行了预训练。此外，还整合了FlashAttention技术以减少内存使用量并稳定训练过程。经过两轮训练后，模型达到了3.168177的训练损失、3.081982的验证损失以及最终困惑度为21.80的表现，证明其能够生成连贯的尼泊尔新闻风格文本。
URL
https://arxiv.org/abs/2512.14585
PDF
-
Polypersona: Persona-Grounded LLM for Synthetic Survey Responses
2025-12-16 16:33:23Tejaswani Dash, Dinesh Karri, Anudeep Vurity, Gautam Datla, Tazeem Ahmad, Saima Rafi, Rohith TanguduAbstract
This paper introduces PolyPersona, a generative framework for synthesizing persona-conditioned survey responses across multiple domains. The framework instruction-tunes compact chat models using parameter-efficient LoRA adapters with 4-bit quantization under a resource-adaptive training setup. A dialogue-based data pipeline explicitly preserves persona cues, ensuring consistent behavioral alignment across generated responses. Using this pipeline, we construct a dataset of 3,568 synthetic survey responses spanning ten domains and 433 distinct personas, enabling controlled instruction tuning and systematic multi-domain evaluation. We evaluate the generated responses using a multi-metric evaluation suite that combines standard text generation metrics, including BLEU, ROUGE, and BERTScore, with survey-specific metrics designed to assess structural coherence, stylistic consistency, and sentiment this http URL results show that compact models such as TinyLlama 1.1B and Phi-2 achieve performance comparable to larger 7B to 8B baselines, with a highest BLEU score of 0.090 and ROUGE-1 of 0.429. These findings demonstrate that persona-conditioned fine-tuning enables small language models to generate reliable and coherent synthetic survey data. The proposed framework provides an efficient and reproducible approach for survey data generation, supporting scalable evaluation while facilitating bias analysis through transparent and open protocols.
Abstract (translated)
本文介绍了PolyPersona，这是一个用于跨多个领域生成条件化人物的调查回应的生成框架。该框架使用参数高效的LoRA适配器和4位量化技术，在资源自适应训练设置下对紧凑型聊天模型进行指令微调。一个基于对话的数据管道明确保留了人物线索，确保生成的回答在行为上保持一致性。利用这个数据管道，我们构建了一个包含3,568个合成调查回应的数据集，这些回应涵盖了十个领域和433种不同的角色，这使得控制性指令微调和系统性的跨域评估成为可能。 通过一个多指标评估套件对生成的响应进行了评估，该套件结合了标准文本生成度量（如BLEU、ROUGE和BERTScore）与针对调查设计的特定度量，旨在评估结构连贯性、风格一致性以及情感准确性。实验结果表明，像TinyLlama 1.1B和Phi-2这样的紧凑模型在性能上可媲美7B至8B规模的基础线模型，最高BLEU得分为0.090，ROUGE-1为0.429。 这些发现证明了人物条件化微调使小型语言模型能够生成可靠且连贯的合成调查数据。所提出的框架提供了一种高效且可重复的方法来生成调查数据，并支持大规模评估同时通过透明和开放协议促进偏差分析。
URL
https://arxiv.org/abs/2512.14562
PDF
-
Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Stud
2025-12-16 02:43:37Jhessica Silva, Diego A. B. Moreira, Gabriel O. dos Santos, Alef Ferreira, Helena Maia, Sandra Avila, Helio PedriniAbstract
In Artificial Intelligence (AI), language models have gained significant importance due to the widespread adoption of systems capable of simulating realistic conversations with humans through text generation. Because of their impact on society, developing and deploying these language models must be done responsibly, with attention to their negative impacts and possible harms. In this scenario, the number of AI Ethics Tools (AIETs) publications has recently increased. These AIETs are designed to help developers, companies, governments, and other stakeholders establish trust, transparency, and responsibility with their technologies by bringing accepted values to guide AI's design, development, and use stages. However, many AIETs lack good documentation, examples of use, and proof of their effectiveness in practice. This paper presents a methodology for evaluating AIETs in language models. Our approach involved an extensive literature survey on 213 AIETs, and after applying inclusion and exclusion criteria, we selected four AIETs: Model Cards, ALTAI, FactSheets, and Harms Modeling. For evaluation, we applied AIETs to language models developed for the Portuguese language, conducting 35 hours of interviews with their developers. The evaluation considered the developers' perspective on the AIETs' use and quality in helping to identify ethical considerations about their model. The results suggest that the applied AIETs serve as a guide for formulating general ethical considerations about language models. However, we note that they do not address unique aspects of these models, such as idiomatic expressions. Additionally, these AIETs did not help to identify potential negative impacts of models for the Portuguese language.
Abstract (translated)
在人工智能（AI）领域，语言模型由于能够通过文本生成模拟与人类的真实对话而变得尤为重要。鉴于其对社会的影响，开发和部署这些语言模型必须负责任地进行，并注意它们可能带来的负面影响和潜在危害。在这种背景下，最近有关于AI伦理工具（AIETs）的出版物数量有所增加。这些AIETs旨在帮助开发者、公司、政府和其他利益相关者通过引入公认的道德价值观来建立其技术的信任、透明度和责任，以指导AI的设计、开发和使用阶段。然而，许多AIET缺乏良好的文档记录、使用示例以及在实践中有效的证明。 本文提出了一种评估语言模型中AIET的方法。我们的方法包括对213个AIET的广泛文献调查，并通过应用包含和排除标准后选择了四个AIET：Model Cards（模型卡片）、ALTAI、FactSheets（事实表）和Harms Modeling（危害建模）。在评估过程中，我们将这些AIET应用于为葡萄牙语开发的语言模型中，并对开发者进行了总计35小时的访谈。评估考虑了开发者从他们的视角出发使用和评价AIET的情况及其质量，在帮助识别他们模型的伦理考量方面的作用。 结果显示，应用的AIET可以作为制定关于语言模型的一般伦理考量的指南。然而，我们注意到它们没有解决这些模型的独特方面，如习惯用语等。此外，这些AIET未能帮助识别葡萄牙语语言模型潜在的负面效果。
URL
https://arxiv.org/abs/2512.15791
PDF
-
CrossTrafficLLM: A Human-Centric Framework for Interpretable Traffic Intelligence via Large Language Model
2025-12-15 23:15:23Zeming Du, Qitan Shao, Hongfei Liu, Yong ZhangAbstract
While accurate traffic forecasting is vital for Intelligent Transportation Systems (ITS), effectively communicating predicted conditions via natural language for human-centric decision support remains a challenge and is often handled separately. To address this, we propose CrossTrafficLLM, a novel GenAI-driven framework that simultaneously predicts future spatiotemporal traffic states and generates corresponding natural language descriptions, specifically targeting conditional abnormal event summaries. We tackle the core challenge of aligning quantitative traffic data with qualitative textual semantics by leveraging Large Language Models (LLMs) within a unified architecture. This design allows generative textual context to improve prediction accuracy while ensuring generated reports are directly informed by the forecast. Technically, a text-guided adaptive graph convolutional network is employed to effectively merge high-level semantic information with the traffic network structure. Evaluated on the BJTT dataset, CrossTrafficLLM demonstrably surpasses state-of-the-art methods in both traffic forecasting performance and text generation quality. By unifying prediction and description generation, CrossTrafficLLM delivers a more interpretable, and actionable approach to generative traffic intelligence, offering significant advantages for modern ITS applications.
Abstract (translated)
虽然准确的交通预测对于智能交通系统（ITS）至关重要，但通过自然语言有效传达预测情况以支持以人为中心的决策仍然是一项挑战，并且通常被单独处理。为了解决这个问题，我们提出了CrossTrafficLLM，这是一种新颖的基于生成AI（GenAI）的框架，该框架能够同时预测未来的时空交通状态并生成相应的自然语言描述，特别针对条件异常事件总结。我们通过利用大型语言模型（LLMs）在统一架构中解决将定量交通数据与定性文本语义对齐的核心挑战。这种设计允许生成性的文本上下文提高预测准确性，并确保生成的报告直接基于预测结果进行信息补充。 从技术角度来看，CrossTrafficLLM采用了由文本引导的自适应图卷积网络来有效地融合高级语义信息和交通网络结构。在BJTT数据集上的评估表明，CrossTrafficLLM在交通预测性能和文本生成质量方面均超越了现有的最先进的方法。通过统一预测和描述生成过程，CrossTrafficLLM提供了一种更具解释性和可操作性的生成性交通智能方法，并为现代ITS应用带来了显著的优势。
URL
https://arxiv.org/abs/2601.06042
PDF
-
PrahokBART: A Pre-trained Sequence-to-Sequence Model for Khmer Natural Language Generation
2025-12-15 17:11:31Hour Kaing, Raj Dabre, Haiyue Song, Van-Hien Tran, Hideki Tanaka, Masao UtiyamaAbstract
This work introduces {\it PrahokBART}, a compact pre-trained sequence-to-sequence model trained from scratch for Khmer using carefully curated Khmer and English corpora. We focus on improving the pre-training corpus quality and addressing the linguistic issues of Khmer, which are ignored in existing multilingual models, by incorporating linguistic components such as word segmentation and normalization. We evaluate PrahokBART on three generative tasks: machine translation, text summarization, and headline generation, where our results demonstrate that it outperforms mBART50, a strong multilingual pre-trained model. Additionally, our analysis provides insights into the impact of each linguistic module and evaluates how effectively our model handles space during text generation, which is crucial for the naturalness of texts in Khmer.
Abstract (translated)
这项工作介绍了{\it PrahokBART}，这是一种专门为高棉语从零开始训练的紧凑型预训练序列到序列模型，并使用精心整理的高棉语和英语语料库。我们专注于提高预训练语料库的质量并解决现有多语言模型忽略的高棉语的语言问题，通过引入诸如词分割和规范化等语言组件来实现这一目标。我们在三项生成任务上评估了PrahokBART：机器翻译、文本摘要和标题生成，在这些任务中，我们的结果表明它优于mBART50，这是一个强大的多语言预训练模型。此外，我们的分析提供了对每个语言模块影响的见解，并评估了在高棉语文本生成过程中我们模型处理空间的效果，这对提高文本自然度至关重要。
URL
https://arxiv.org/abs/2512.13552
PDF
-
HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks
2025-12-14 04:28:39Yiming Zeng, Jinghan Cao, Zexin Li, Wanhao Yu, Zhankai Ye, Dawei Xiang, Ting Hua, Xin Liu, Shangqian Gao, Tingting YuAbstract
Instruction-based text editing is increasingly critical for real-world applications such as code editors (e.g., Cursor), but Large Language Models (LLMs) continue to struggle with this task. Unlike free-form generation, editing requires faithfully implementing user instructions while preserving unchanged content, as even minor unintended modifications can break functionality. Existing approaches treat editing as generic text generation, leading to two key failures: they struggle to faithfully align edits with diverse user intents, and they often over-edit unchanged regions. We propose HyperEdit to address both issues. First, we introduce hypernetwork-based dynamic adaptation that generates request-specific parameters, enabling the model to tailor its editing strategy to each instruction. Second, we develop difference-aware regularization that focuses supervision on modified spans, preventing over-editing while ensuring precise, minimal changes. HyperEdit achieves a 9%--30% relative improvement in BLEU on modified regions over state-of-the-art baselines, despite utilizing only 3B parameters.
Abstract (translated)
基于指令的文本编辑在诸如代码编辑器（例如Cursor）等实际应用中变得越来越重要，但大型语言模型（LLMs）在此任务上仍然面临挑战。与自由形式生成不同，编辑要求忠实执行用户的指示并保留未更改的内容，因为即使是细微的意外修改也可能破坏功能完整性。现有的方法将编辑视为通用文本生成处理方式，导致两个关键问题：它们难以准确地根据不同的用户意图实施编辑，并且经常在未改变的部分过度编辑。我们提出了HyperEdit来解决这些问题。 首先，我们引入了基于超网络的动态适应机制，该机制可以为每个指令生成特定请求的参数，从而使模型能够为其编辑策略定制化以满足具体需求。 其次，我们开发了一种差异感知正则化方法，这种方法专注于监督修改区域，防止过度编辑的同时确保精确且最小化的更改。 HyperEdit在已修改部分实现了显著改进，在BLEU评分上比最先进的基准高出9%到30%，而仅使用了约30亿参数。
URL
https://arxiv.org/abs/2512.12544
PDF
-
Super Suffixes: Bypassing Text Generation Alignment and Guard Models Simultaneously
2025-12-12 18:52:09Andrew Adiletta, Kathryn Adiletta, Kemal Derya, Berk SunarAbstract
The rapid deployment of Large Language Models (LLMs) has created an urgent need for enhanced security and privacy measures in Machine Learning (ML). LLMs are increasingly being used to process untrusted text inputs and even generate executable code, often while having access to sensitive system controls. To address these security concerns, several companies have introduced guard models, which are smaller, specialized models designed to protect text generation models from adversarial or malicious inputs. In this work, we advance the study of adversarial inputs by introducing Super Suffixes, suffixes capable of overriding multiple alignment objectives across various models with different tokenization schemes. We demonstrate their effectiveness, along with our joint optimization technique, by successfully bypassing the protection mechanisms of Llama Prompt Guard 2 on five different text generation models for malicious text and code generation. To the best of our knowledge, this is the first work to reveal that Llama Prompt Guard 2 can be compromised through joint optimization. Additionally, by analyzing the changing similarity of a model's internal state to specific concept directions during token sequence processing, we propose an effective and lightweight method to detect Super Suffix attacks. We show that the cosine similarity between the residual stream and certain concept directions serves as a distinctive fingerprint of model intent. Our proposed countermeasure, DeltaGuard, significantly improves the detection of malicious prompts generated through Super Suffixes. It increases the non-benign classification rate to nearly 100%, making DeltaGuard a valuable addition to the guard model stack and enhancing robustness against adversarial prompt attacks.
Abstract (translated)
大型语言模型（LLMs）的快速部署已经产生了对机器学习（ML）中增强的安全性和隐私措施的迫切需求。这些模型越来越多地被用于处理未经验证的文本输入，甚至生成可执行代码，并且经常能够访问敏感系统控制。为解决安全问题，一些公司推出了防护模型，即专门设计的小型模型，旨在保护文本生成模型免受对抗性或恶意输入的影响。在这项工作中，我们通过引入超级后缀（Super Suffixes）来推进对对抗性输入的研究，这些是能够在具有不同分词方案的各种模型上覆盖多个对齐目标的后缀。我们展示了它们的有效性以及我们的联合优化技术，并成功绕过了五种不同的文本生成模型中的Llama Prompt Guard 2的安全机制以生成恶意文本和代码。据我们所知，这是首次揭示Llama Prompt Guard 2可以通过联合优化被攻破的工作。 此外，通过分析模型在处理令牌序列时内部状态与特定概念方向之间的相似性变化，我们提出了一种有效的、轻量级的方法来检测超级后缀攻击。我们证明了残差流和某些概念方向之间的余弦相似度构成了模型意图的独特指纹。我们的防御措施DeltaGuard显著提高了对通过超级后缀生成的恶意提示进行检测的能力。它将非良性分类率提高到接近100%，使DeltaGuard成为防护模型堆栈中的宝贵补充，并增强了对抗性提示攻击的鲁棒性。
URL
https://arxiv.org/abs/2512.11783
PDF
-
Semantic Reconstruction of Adversarial Plagiarism: A Context-Aware Framework for Detecting and Restoring 'Tortured Phrases' in Scientific Literature
2025-12-11 08:53:25Agniva Maiti, Prajwal Panth, Suresh Chandra SatapathyAbstract
The integrity and reliability of scientific literature is facing a serious threat by adversarial text generation techniques, specifically from the use of automated paraphrasing tools to mask plagiarism. These tools generate "tortured phrases", statistically improbable synonyms (e.g. "counterfeit consciousness" for "artificial intelligence"), that preserve the local grammar while obscuring the original source. Most existing detection methods depend heavily on static blocklists or general-domain language models, which suffer from high false-negative rates for novel obfuscations and cannot determine the source of the plagiarized content. In this paper, we propose Semantic Reconstruction of Adversarial Plagiarism (SRAP), a framework designed not only to detect these anomalies but to mathematically recover the original terminology. We use a two-stage architecture: (1) statistical anomaly detection with a domain-specific masked language model (SciBERT) using token-level pseudo-perplexity, and (2) source-based semantic reconstruction using dense vector retrieval (FAISS) and sentence-level alignment (SBERT). Experiments on a parallel corpus of adversarial scientific text show that while zero-shot baselines fail completely (0.00 percent restoration accuracy), our retrieval-augmented approach achieves 23.67 percent restoration accuracy, significantly outperforming baseline methods. We also show that static decision boundaries are necessary for robust detection in jargon-heavy scientific text, since dynamic thresholding fails under high variance. SRAP enables forensic analysis by linking obfuscated expressions back to their most probable source documents.
Abstract (translated)
科学文献的完整性和可靠性正面临着敌对文本生成技术的严重威胁，尤其是自动改写工具被用于掩盖抄袭行为。这些工具会产生“扭曲的短语”，即统计上不太可能的同义词（例如，“伪造意识”替代“人工智能”），它们在保留局部语法的同时隐藏了原始来源。现有的大多数检测方法依赖于静态黑名单或通用领域的语言模型，这导致对新颖混淆的假阴性率高，并且无法确定抄袭内容的来源。在这篇论文中，我们提出了语义重构敌对剽窃（SRAP）框架，该框架不仅设计用于检测这些异常情况，还旨在通过数学手段恢复原始术语。 我们的方法采用两阶段架构：(1) 使用领域特定的掩码语言模型（SciBERT）进行统计异常检测，并利用标记级伪困惑度；(2) 使用密集向量检索（FAISS）和句子级别对齐（SBERT）进行基于源语义重构。在对抗性科学文本平行语料库上的实验表明，零样本基准方法完全失败（恢复准确率为0.00%），而我们的增强检索方法实现了23.67%的恢复准确性，显著优于基线方法。 此外，我们还展示了静态决策边界对于识别富含专业术语的科学文献中的剽窃行为是必需的，因为动态阈值在高方差情况下无法正常工作。SRAP通过将混淆表达链接回最可能的源文档来实现法医分析。
URL
https://arxiv.org/abs/2512.10435
PDF
-
PARAN: Persona-Augmented Review ANswering system on Food Delivery Review Dataset
2025-12-10 23:04:48Moonsoo Park, Jeongseok Yun, Bohyung KimAbstract
Personalized review response generation presents a significant challenge in domains where user information is limited, such as food delivery platforms. While large language models (LLMs) offer powerful text generation capabilities, they often produce generic responses when lacking contextual user data, reducing engagement and effectiveness. In this work, we propose a two-stage prompting framework that infers both explicit (e.g., user-stated preferences) and implicit (e.g., demographic or stylistic cues) personas directly from short review texts. These inferred persona attributes are then incorporated into the response generation prompt to produce user-tailored replies. To encourage diverse yet faithful generations, we adjust decoding temperature during inference. We evaluate our method using a real-world dataset collected from a Korean food delivery app, and assess its impact on precision, diversity, and semantic consistency. Our findings highlight the effectiveness of persona-augmented prompting in enhancing the relevance and personalization of automated responses without requiring model fine-tuning.
Abstract (translated)
个性化评论回复生成在用户信息有限的领域（如食品配送平台）中面临重大挑战。虽然大型语言模型（LLMs）提供了强大的文本生成能力，但当缺少上下文用户数据时，它们往往会产生通用且缺乏个性化的响应，从而降低参与度和有效性。为此，我们提出了一种两阶段提示框架，该框架可以直接从简短的评论文本中推断出显式的（如用户声明的偏好）和个人隐含的人格特征（如人口统计或风格线索）。然后将这些推断出来的人格属性整合到响应生成提示中，以产生个性化的回复。为了鼓励多样化且忠实的生成，在推理过程中调整解码温度。 我们使用从韩国食品配送应用程序收集的真实世界数据集评估了我们的方法，并对其在精度、多样性和语义一致性方面的效果进行了评价。研究结果表明，人格增强提示对于提高自动化响应的相关性和个性化程度非常有效，而无需对模型进行微调。
URL
https://arxiv.org/abs/2512.10148
PDF
-
Workflow is All You Need: Escaping the 'Statistical Smoothing Trap' via High-Entropy Information Foraging and Adversarial Pacing
2025-12-10 22:13:55Zhongjie JiangAbstract
Central to long-form text generation in vertical domains is the "impossible trinity" confronting current large language models (LLMs): the simultaneous achievement of low hallucination, deep logical coherence, and personalized expression. This study establishes that this bottleneck arises from existing generative paradigms succumbing to the Statistical Smoothing Trap, a phenomenon that overlooks the high-entropy information acquisition and structured cognitive processes integral to expert-level writing. To address this limitation, we propose the DeepNews Framework, an agentic workflow that explicitly models the implicit cognitive processes of seasoned financial journalists. The framework integrates three core modules: first, a dual-granularity retrieval mechanism grounded in information foraging theory, which enforces a 10:1 saturated information input ratio to mitigate hallucinatory outputs; second, schema-guided strategic planning, a process leveraging domain expert knowledge bases (narrative schemas) and Atomic Blocks to forge a robust logical skeleton; third, adversarial constraint prompting, a technique deploying tactics including Rhythm Break and Logic Fog to disrupt the probabilistic smoothness inherent in model-generated text. Experiments delineate a salient Knowledge Cliff in deep financial reporting: content truthfulness collapses when retrieved context falls below 15,000 characters, while a high-redundancy input exceeding 30,000 characters stabilizes the Hallucination-Free Rate (HFR) above 85%. In an ecological validity blind test conducted with a top-tier Chinese technology media outlet, the DeepNews system--built on a previous-generation model (DeepSeek-V3-0324)-achieved a 25% submission acceptance rate, significantly outperforming the 0% acceptance rate of zero-shot generation by a state-of-the-art (SOTA) model (GPT-5).
Abstract (translated)
在垂直领域中长篇文本生成的核心问题，是当前大型语言模型（LLMs）面临的“不可能的三难困境”：即无法同时实现低虚假信息率、深刻的逻辑连贯性和个性化表达。本研究确立了这一瓶颈源自现有生成范式陷入统计平滑陷阱的现象，该现象忽略了获取高熵信息和结构化认知过程对于专家级写作的重要性。为解决此限制，我们提出了DeepNews框架，这是一个代理工作流，明确地模拟资深财经记者的隐性认知过程。该框架集成了三个核心模块：首先是一个基于觅食理论的双粒度检索机制，它规定了一个10:1的信息饱和输入比例以减少虚假信息输出；其次是一套方案引导的战略规划流程，利用领域专家知识库（叙事模式）和原子块构建一个坚固的逻辑骨架；第三是使用包括节奏打断和逻辑迷雾等战术的对抗性约束提示技术，以破坏模型生成文本中的概率平滑性。实验揭示了深度财经报道中显著的知识断崖：当检索到的内容少于15,000字符时，内容真实性会崩溃，而超过30,000字符的高冗余输入可以将无虚假信息率（HFR）稳定在85%以上。在一个顶级中国科技媒体进行的生态有效盲测中，基于前一代模型（DeepSeek-V3-0324）构建的DeepNews系统实现了25%的提交接受率，显著优于零样本生成状态下最先进（SOTA）模型（GPT-5）的0%接受率。
URL
https://arxiv.org/abs/2512.10121
PDF
-
Generate-Then-Validate: A Novel Question Generation Approach Using Small Language Models
2025-12-10 21:59:36Yumou Wei, John Stamper, Paulo F. CarvalhoAbstract
We explore the use of small language models (SLMs) for automatic question generation as a complement to the prevalent use of their large counterparts in learning analytics research. We present a novel question generation pipeline that leverages both the text generation and the probabilistic reasoning abilities of SLMs to generate high-quality questions. Adopting a "generate-then-validate" strategy, our pipeline first performs expansive generation to create an abundance of candidate questions and refine them through selective validation based on novel probabilistic reasoning. We conducted two evaluation studies, one with seven human experts and the other with a large language model (LLM), to assess the quality of the generated questions. Most judges (humans or LLMs) agreed that the generated questions had clear answers and generally aligned well with the intended learning objectives. Our findings suggest that an SLM can effectively generate high-quality questions when guided by a well-designed pipeline that leverages its strengths.
Abstract (translated)
我们探讨了使用小型语言模型（SLM）进行自动问题生成的用途，作为在学习分析研究中广泛使用的大型模型的一种补充。我们提出了一种新颖的问题生成管道，该管道利用了SLM的文字生成能力和概率推理能力来产生高质量的问题。采用“先生成再验证”的策略，我们的管道首先通过广泛的生成过程创建了大量的候选问题，并通过基于新型概率推理的精选验证对其进行细化和优化。 为了评估所生成问题的质量，我们进行了两项评价研究：一项是与七位人类专家合作进行的研究；另一项则是借助大型语言模型（LLM）开展的研究。大多数评判者（无论是人类还是LLM）都认为生成的问题有明确的答案，并且通常符合预期的学习目标。我们的发现表明，在一个设计得当的管道指导下，SLM能够有效地生成高质量的问题，该管道充分地利用了其优势。
URL
https://arxiv.org/abs/2512.10110
PDF
-
ImageTalk: Designing a Multimodal AAC Text Generation System Driven by Image Recognition and Natural Language Generation
2025-12-10 12:57:55Boyin Yang, Puming Jiang, Per Ola KristenssonAbstract
People living with Motor Neuron Disease (plwMND) frequently encounter speech and motor impairments that necessitate a reliance on augmentative and alternative communication (AAC) systems. This paper tackles the main challenge that traditional symbol-based AAC systems offer a limited vocabulary, while text entry solutions tend to exhibit low communication rates. To help plwMND articulate their needs about the system efficiently and effectively, we iteratively design and develop a novel multimodal text generation system called ImageTalk through a tailored proxy-user-based and an end-user-based design phase. The system demonstrates pronounced keystroke savings of 95.6%, coupled with consistent performance and high user satisfaction. We distill three design guidelines for AI-assisted text generation systems design and outline four user requirement levels tailored for AAC purposes, guiding future research in this field.
Abstract (translated)
患有运动神经元疾病（plwMND）的人常常面临言语和运动障碍，这需要依赖辅助性和替代性沟通（AAC）系统。本文探讨了传统基于符号的AAC系统词汇量有限的问题，而文本输入解决方案往往表现出较低的通信效率。为了帮助plwMND用户有效地表达他们对系统的反馈，我们通过定制的代理用户基础设计阶段和最终用户基础设计阶段迭代地设计并开发了一种新型多模态文本生成系统——ImageTalk。该系统展示了显著的按键节省率（95.6%），同时保持了稳定的性能和高用户满意度。我们提炼出三项针对AI辅助文本生成系统的界面设计理念，并概述了四个针对AAC目的定制的用户需求层次，为这一领域的未来研究提供了指导方针。
URL
https://arxiv.org/abs/2512.09610
PDF
-
Knowledge-Augmented Large Language Model Agents for Explainable Financial Decision-Making
2025-12-10 09:08:33Qingyuan Zhang, Yuxi Wang, Cancan Hua, Yulin Huang, Ning LyuAbstract
This study investigates an explainable reasoning method for financial decision-making based on knowledge-enhanced large language model agents. To address the limitations of traditional financial decision methods that rely on parameterized knowledge, lack factual consistency, and miss reasoning chains, an integrated framework is proposed that combines external knowledge retrieval, semantic representation, and reasoning generation. The method first encodes financial texts and structured data to obtain semantic representations, and then retrieves task-related information from external knowledge bases using similarity computation. Internal representations and external knowledge are combined through weighted fusion, which ensures fluency while improving factual accuracy and completeness of generated content. In the reasoning stage, a multi-head attention mechanism is introduced to construct logical chains, allowing the model to present transparent causal relationships and traceability during generation. Finally, the model jointly optimizes task objectives and explanation consistency objectives, which enhances predictive performance and reasoning interpretability. Experiments on financial text processing and decision tasks show that the method outperforms baseline approaches in accuracy, text generation quality, and factual support, verifying the effectiveness of knowledge enhancement and explainable reasoning. Overall, the proposed approach overcomes the limitations of traditional models in semantic coverage and reasoning transparency, and demonstrates strong practical value in complex financial scenarios.
Abstract (translated)
这项研究探讨了一种基于知识增强的大语言模型代理的可解释推理方法在金融决策中的应用。为了克服传统金融决策方法依赖参数化知识、缺乏事实一致性以及忽略推理链等问题，提出了一种集成了外部知识检索、语义表示和推理生成于一体的框架。该方法首先将财务文本和结构化数据编码为语义表示，并通过相似度计算从外部知识库中检索任务相关的信息。内部表示与外部知识结合时采用加权融合的方式，在保证流畅性的同时，提升了生成内容的事实准确性和完整性。 在推理阶段，引入了多头注意力机制来构建逻辑链，使模型能够在生成过程中呈现透明的因果关系和可追溯性。最后，该模型通过同时优化任务目标和解释一致性目标，不仅提高了预测性能，还增强了推理的可解释性。通过对金融文本处理及决策任务的实验表明，该方法在准确性、文本生成质量和事实支持方面均优于基线方法，验证了知识增强与可解释推理的有效性。 总体而言，提出的这种方法克服了传统模型在语义覆盖和推理透明度方面的局限性，并在复杂的金融场景中展现了强大的实际应用价值。
URL
https://arxiv.org/abs/2512.09440
PDF
-
Identifying Bias in Machine-generated Text Detection
2025-12-10 03:34:12Kevin Stowe, Svetlana Afanaseva, Rodolfo Raimundo, Yitao Sun, Kailash PatilAbstract
The meteoric rise in text generation capability has been accompanied by parallel growth in interest in machine-generated text detection: the capability to identify whether a given text was generated using a model or written by a person. While detection models show strong performance, they have the capacity to cause significant negative impacts. We explore potential biases in English machine-generated text detection systems. We curate a dataset of student essays and assess 16 different detection systems for bias across four attributes: gender, race/ethnicity, English-language learner (ELL) status, and economic status. We evaluate these attributes using regression-based models to determine the significance and power of the effects, as well as performing subgroup analysis. We find that while biases are generally inconsistent across systems, there are several key issues: several models tend to classify disadvantaged groups as machine-generated, ELL essays are more likely to be classified as machine-generated, economically disadvantaged students' essays are less likely to be classified as machine-generated, and non-White ELL essays are disproportionately classified as machine-generated relative to their White counterparts. Finally, we perform human annotation and find that while humans perform generally poorly at the detection task, they show no significant biases on the studied attributes.
Abstract (translated)
文本生成能力的迅速提升伴随着对机器生成文本检测兴趣的平行增长：即识别给定文本是否由模型生成还是由人书写的这一能力。尽管检测模型表现出色，但它们有可能产生显著的负面影响。我们探讨了英语机器生成文本检测系统中存在的潜在偏见。为此，我们收集了一组学生作文，并评估了16种不同的检测系统在这四个属性上的偏差：性别、种族/族裔、作为英语语言学习者（ELL）的状态以及经济状况。我们使用基于回归的模型来确定这些影响的重要性和效力，并进行了子群体分析。研究发现，虽然不同系统之间的偏见通常并不一致，但存在几个关键问题：多个模型倾向于将弱势群体归类为机器生成；作为ELL的学生作文更容易被分类为机器生成；经济条件较差学生的作品更不容易被分类为机器生成；而非白人ELL学生的作文则比其白人同学的作品更多地被错误分类为机器生成。最后，我们进行了人工标注，并发现虽然人类在检测任务上表现一般较低，但在研究的属性方面并未显示出显著偏见。
URL
https://arxiv.org/abs/2512.09292
PDF
-
Large Language Models for Education and Research: An Empirical and User Survey-based Analysis
2025-12-08 21:35:28Md Mostafizer Rahman, Ariful Islam Shiplu, Md Faizul Ibne Amin, Yutaka Watanobe, Lu PengAbstract
Pretrained Large Language Models (LLMs) have achieved remarkable success across diverse domains, with education and research emerging as particularly impactful areas. Among current state-of-the-art LLMs, ChatGPT and DeepSeek exhibit strong capabilities in mathematics, science, medicine, literature, and programming. In this study, we present a comprehensive evaluation of these two LLMs through background technology analysis, empirical experiments, and a real-world user survey. The evaluation explores trade-offs among model accuracy, computational efficiency, and user experience in educational and research affairs. We benchmarked these LLMs performance in text generation, programming, and specialized problem-solving. Experimental results show that ChatGPT excels in general language understanding and text generation, while DeepSeek demonstrates superior performance in programming tasks due to its efficiency- focused design. Moreover, both models deliver medically accurate diagnostic outputs and effectively solve complex mathematical problems. Complementing these quantitative findings, a survey of students, educators, and researchers highlights the practical benefits and limitations of these models, offering deeper insights into their role in advancing education and research.
Abstract (translated)
预训练大型语言模型（LLMs）在各个领域取得了显著的成功，特别是在教育和研究方面尤为突出。当前最先进的LLM中，ChatGPT和DeepSeek展示了在数学、科学、医学、文学和编程等多个领域的强大能力。本研究通过技术背景分析、实证实验以及真实用户调查对这两种LLM进行了全面评估。此次评估探索了模型准确性、计算效率与用户体验之间的权衡问题，并针对教育和科研事务中的文本生成、编程及专业问题解决等方面，对这些LLM的性能进行了基准测试。 实验结果显示，在通用语言理解和文本生成方面，ChatGPT表现出色；而在编程任务中，DeepSeek因其专注于效率的设计而展现出优异的表现。此外，这两种模型都能提供准确的医学诊断结果，并能有效地解决复杂的数学问题。为了补充这些定量研究发现，我们对学生们、教育工作者和研究人员进行了调查，探讨了这些模型的实际益处与局限性，为了解它们在促进教育与科研发展方面的作用提供了更深入的见解。
URL
https://arxiv.org/abs/2512.08057
PDF
-
SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents
2025-12-08 13:17:27Michelle Wastl, Jannis Vamvas, Rico SennrichAbstract
Recognizing semantic differences across documents, especially in different languages, is crucial for text generation evaluation and multilingual content alignment. However, as a standalone task it has received little attention. We address this by introducing SwissGov-RSD, the first naturalistic, document-level, cross-lingual dataset for semantic difference recognition. It encompasses a total of 224 multi-parallel documents in English-German, English-French, and English-Italian with token-level difference annotations by human annotators. We evaluate a variety of open-source and closed source large language models as well as encoder models across different fine-tuning settings on this new benchmark. Our results show that current automatic approaches perform poorly compared to their performance on monolingual, sentence-level, and synthetic benchmarks, revealing a considerable gap for both LLMs and encoder models. We make our code and datasets publicly available.
Abstract (translated)
识别不同文档之间的语义差异，尤其是在不同的语言中，对于文本生成评估和多语言内容对齐至关重要。然而，作为一个独立的任务，它并没有得到太多的关注。我们通过引入SwissGov-RSD来解决这个问题，这是第一个旨在跨语言语义差异识别的自然化、文档级数据集，包括总共224份在英语-德语、英语-法语和英语-意大利语之间的多平行文档，并由人工标注员提供了词级别差异注释。我们在这个新的基准上评估了各种开源和闭源的大规模语言模型以及不同微调设置下的编码器模型。我们的结果表明，当前的自动方法在与单语种、句子级和合成基准相比时表现较差，显示出大规模语言模型（LLMs）和编码器模型在这方面的显著差距。我们公开发布了代码和数据集。
URL
https://arxiv.org/abs/2512.07538
PDF
