Text_Generation
-
Video-As-Prompt: Unified Semantic Control for Video Generation
2025-10-23 17:59:52Yuxuan Bian, Xin Chen, Zenan Li, Tiancheng Zhi, Shen Sang, Linjie Luo, Qiang XuAbstract
Unified, generalizable semantic control in video generation remains a critical open challenge. Existing methods either introduce artifacts by enforcing inappropriate pixel-wise priors from structure-based controls, or rely on non-generalizable, condition-specific finetuning or task-specific architectures. We introduce Video-As-Prompt (VAP), a new paradigm that reframes this problem as in-context generation. VAP leverages a reference video as a direct semantic prompt, guiding a frozen Video Diffusion Transformer (DiT) via a plug-and-play Mixture-of-Transformers (MoT) expert. This architecture prevents catastrophic forgetting and is guided by a temporally biased position embedding that eliminates spurious mapping priors for robust context retrieval. To power this approach and catalyze future research, we built VAP-Data, the largest dataset for semantic-controlled video generation with over 100K paired videos across 100 semantic conditions. As a single unified model, VAP sets a new state-of-the-art for open-source methods, achieving a 38.7% user preference rate that rivals leading condition-specific commercial models. VAP's strong zero-shot generalization and support for various downstream applications mark a significant advance toward general-purpose, controllable video generation.
Abstract (translated)
URL
https://arxiv.org/abs/2510.20888
PDF
-
A Use-Case Specific Dataset for Measuring Dimensions of Responsible Performance in LLM-generated Text
2025-10-23 17:50:55Alicia Sagae, Chia-Jung Lee, Sandeep Avula, Brandon Dang, Vanessa MurdockAbstract
Current methods for evaluating large language models (LLMs) typically focus on high-level tasks such as text generation, without targeting a particular AI application. This approach is not sufficient for evaluating LLMs for Responsible AI dimensions like fairness, since protected attributes that are highly relevant in one application may be less relevant in another. In this work, we construct a dataset that is driven by a real-world application (generate a plain-text product description, given a list of product features), parameterized by fairness attributes intersected with gendered adjectives and product categories, yielding a rich set of labeled prompts. We show how to use the data to identify quality, veracity, safety, and fairness gaps in LLMs, contributing a proposal for LLM evaluation paired with a concrete resource for the research community.
Abstract (translated)
目前评估大型语言模型（LLMs）的方法通常集中在如文本生成等高层次任务上，而没有针对特定的人工智能应用程序。这种方法不足以对像公平性这样的负责任AI维度进行评价，因为在某一应用中非常相关的受保护属性，在其他应用中可能不太相关。在本项工作中，我们构建了一个由真实世界应用场景驱动的数据集（根据产品特征列表生成纯文本的产品描述），该数据集通过与性别形容词和产品类别相交的公平性属性参数化，产生了一系列丰富的带有标签的提示。我们展示了如何使用这些数据来识别LLMs的质量、真实性、安全性和公平性的差距，并为研究社区提出了一个结合具体资源的评估方法提案。
URL
https://arxiv.org/abs/2510.20782
PDF
-
Context-level Language Modeling by Learning Predictive Context Embeddings
2025-10-23 07:09:45Beiya Dai, Yuliang Liu, Daozheng Xue, Qipeng Guo, Kai Chen, Xinbing WangAbstract
Next-token prediction (NTP) is the cornerstone of modern large language models (LLMs) pretraining, driving their unprecedented capabilities in text generation, reasoning, and instruction following. However, the token-level prediction limits the model's capacity to capture higher-level semantic structures and long-range contextual relationships. To overcome this limitation, we introduce \textbf{ContextLM}, a framework that augments standard pretraining with an inherent \textbf{next-context prediction} objective. This mechanism trains the model to learn predictive representations of multi-token contexts, leveraging error signals derived from future token chunks. Crucially, ContextLM achieves this enhancement while remaining fully compatible with the standard autoregressive, token-by-token evaluation paradigm (e.g., perplexity). Extensive experiments on the GPT2 and Pythia model families, scaled up to $1.5$B parameters, show that ContextLM delivers consistent improvements in both perplexity and downstream task performance. Our analysis indicates that next-context prediction provides a scalable and efficient pathway to stronger language modeling, yielding better long-range coherence and more effective attention allocation with minimal computational overhead.
Abstract (translated)
下一代标记预测（NTP）是现代大型语言模型（LLM）预训练的基础，推动了这些模型在文本生成、推理和指令遵循方面的前所未有的能力。然而，基于标记级别的预测限制了模型捕捉更高层次的语义结构以及长距离上下文关系的能力。为了解决这一局限性，我们引入了**ContextLM**框架，该框架通过添加内在的**下一个上下文预测**目标来增强标准预训练过程。这种机制使模型能够学习多令牌上下文的预测表示，并利用来自未来标记块的误差信号。至关重要的是，ContextLM能够在完全兼容标准自回归、逐令牌评估范式（如困惑度）的同时实现这一改进。在GPT2和Pythia模型系列上的广泛实验表明，ContextLM可以在参数规模达到15亿的情况下，在困惑度和下游任务性能上提供一致的改进。我们的分析表明，上下文预测为更强的语言建模提供了可扩展且高效的途径，能够带来更好的长距离连贯性和更有效的注意力分配，并且几乎不需要额外计算资源。
URL
https://arxiv.org/abs/2510.20280
PDF
-
A Multimodal, Multitask System for Generating E Commerce Text Listings from Images
2025-10-22 11:50:49Nayan Kumar SinghAbstract
Manually generating catchy descriptions and names is labor intensive and a slow process for retailers. Although generative AI provides an automation solution in form of Vision to Language Models (VLM), the current VLMs are prone to factual "hallucinations". Siloed, single task models are not only inefficient but also fail to capture interdependent relationships between features. To address these challenges, we propose an end to end, multi task system that generates factually grounded textual listings from a single image. The contributions of this study are two proposals for the model architecture. First, application of multi task learning approach for fine tuning a vision encoder where a single vision backbone is jointly trained on attribute prediction such as color, hemline and neck style and price regression. Second, introduction of a hierarchical generation process where the model's own predicted attributes are embedded in a prompt and fed to the text decoder to improve factual consistency. The experiments demonstrate the superiority of this architecture. The multi tasking approach outperforms both the independent price regression, with a 3.6% better R2 Value and attribute classification, with a 6.6% improvement F1 score. Critically, the hierarchical generation process proves highly effective, slashing the factual hallucination rate from 12.7% to 7.1%, a 44.5% relative reduction, compared to a non hierarchical ablation. The hierarchical approach also reduces the latency of the autoregressive text generation process by a factor of 3.5 when compared to direct vision to language model of similar size. One minor caveat is that the model does perform 3.5% worse than direct vision-to-language model on ROUGE-L score.
Abstract (translated)
URL
https://arxiv.org/abs/2510.21835
PDF
-
Re-evaluating Minimum Bayes Risk Decoding for Automatic Speech Recognition
2025-10-22 11:06:20Yuu JinnaiAbstract
Recent work has shown that sample-based Minimum Bayes Risk (MBR) decoding outperforms beam search in text-to-text generation tasks, such as machine translation, text summarization, and image captioning. On the other hand, beam search is the current practice for speech-to-text tasks such as automatic speech recognition (ASR) and Speech Translation (ST). Given that MBR decoding is effective in text-to-text generation tasks, it is reasonable to expect it to also be effective for speech-to-text tasks. In this paper, we evaluate MBR decoding for ASR and ST tasks on English and Japanese using Whisper and its derivative models. We observe that the accuracy of MBR decoding outperforms that of beam search in most of the experimental settings we have evaluated. The results show that MBR decoding is a promising method for offline ASR and ST tasks that require high accuracy. The code is available at this https URL
Abstract (translated)
最近的研究表明，在文本到文本生成任务（如机器翻译、文本摘要和图像描述）中，基于样本的最小贝叶斯风险(Minimum Bayes Risk, MBR)解码优于束搜索。然而，在语音转文本的任务（例如自动语音识别(ASR) 和语音翻译(ST)）中，当前实践中普遍使用束搜索方法。鉴于MBR 解码在文本到文本生成任务中的有效性，可以合理地期望它同样适用于语音转文本的任务。在这篇论文中，我们评估了MBR解码在英语和日语的自动语音识别（ASR）以及语音翻译(ST)任务上的表现，使用的是Whisper及其衍生模型。我们观察到，在大多数实验设置下，MBR 解码的准确性优于束搜索方法。研究结果表明，对于需要高准确性的离线 ASR 和 ST 任务来说，MBR 解码是一种有前景的方法。相关代码可以在[这里](https://URL)获取。
URL
https://arxiv.org/abs/2510.19471
PDF
-
Unified Reinforcement and Imitation Learning for Vision-Language Models
2025-10-22 07:12:14Byung-Kwan Lee, Ryo Hachiuma, Yong Man Ro, Yu-Chiang Frank Wang, Yueh-Hua WuAbstract
Vision-Language Models (VLMs) have achieved remarkable progress, yet their large scale often renders them impractical for resource-constrained environments. This paper introduces Unified Reinforcement and Imitation Learning (RIL), a novel and efficient training algorithm designed to create powerful, lightweight VLMs. RIL distinctively combines the strengths of reinforcement learning with adversarial imitation learning. This enables smaller student VLMs not only to mimic the sophisticated text generation of large teacher models but also to systematically improve their generative capabilities through reinforcement signals. Key to our imitation framework is an LLM-based discriminator that adeptly distinguishes between student and teacher outputs, complemented by guidance from multiple large teacher VLMs to ensure diverse learning. This unified learning strategy, leveraging both reinforcement and imitation, empowers student models to achieve significant performance gains, making them competitive with leading closed-source VLMs. Extensive experiments on diverse vision-language benchmarks demonstrate that RIL significantly narrows the performance gap with state-of-the-art open- and closed-source VLMs and, in several instances, surpasses them.
Abstract (translated)
视觉语言模型（VLMs）已经取得了显著的进步，但其庞大的规模往往使其在资源受限的环境中难以应用。本文介绍了统一强化与模仿学习（RIL），这是一种新颖且高效的训练算法，旨在创建强大而轻量级的VLM。RIL独特地结合了强化学习和对抗性模仿学习的优势。这使得较小的学生VLM不仅能够模仿大型教师模型复杂的文本生成能力，还能通过强化信号系统性地提高其生成能力。 我们所提出的模仿框架的关键是一个基于大规模语言模型（LLM）的判别器，它能巧妙地区分学生模型与教师模型的输出，并辅以多个大型教师VLM的指导，确保多样化的学习。这种统一的学习策略，同时利用了强化和模仿的优势，使学生模型能够实现显著的性能提升，使其在与领先闭源VLM的竞争中具备竞争力。 广泛的实验证明，在多种视觉语言基准上，RIL大大缩小了与最先进的开源和闭源VLM之间的性能差距，并且在某些情况下超越了它们。
URL
https://arxiv.org/abs/2510.19307
PDF
-
PoSh: Using Scene Graphs To Guide LLMs-as-a-Judge For Detailed Image Descriptions
2025-10-21 20:30:20Amith Ananthram, Elias Stengel-Eskin, Lorena A. Bradford, Julia Demarest, Adam Purvis, Keith Krut, Robert Stein, Rina Elster Pantalony, Mohit Bansal, Kathleen McKeownAbstract
While vision-language models (VLMs) have advanced into detailed image description, evaluation remains a challenge. Standard metrics (e.g. CIDEr, SPICE) were designed for short texts and tuned to recognize errors that are now uncommon, such as object misidentification. In contrast, long texts require sensitivity to attribute and relation attachments and scores that localize errors to particular text spans. In this work, we introduce PoSh, a metric for detailed image description that uses scene graphs as structured rubrics to guide LLMs-as-a-Judge, producing aggregate scores grounded in fine-grained errors (e.g. mistakes in compositional understanding). PoSh is replicable, interpretable and a better proxy for human raters than existing metrics (including GPT4o-as-a-Judge). To validate PoSh, we introduce a challenging new dataset, DOCENT. This novel benchmark contains artwork, paired with expert-written references, and model-generated descriptions, augmented with granular and coarse judgments of their quality from art history students. Thus, DOCENT enables evaluating both detailed image description metrics and detailed image description itself in a challenging new domain. We show that PoSh achieves stronger correlations (+0.05 Spearman $\rho$) with the human judgments in DOCENT than the best open-weight alternatives, is robust to image type (using CapArena, an existing dataset of web imagery) and is a capable reward function, outperforming standard supervised fine-tuning. Then, using PoSh, we characterize the performance of open and closed models in describing the paintings, sketches and statues in DOCENT and find that foundation models struggle to achieve full, error-free coverage of images with rich scene dynamics, establishing a demanding new task to gauge VLM progress. Through both PoSh and DOCENT, we hope to enable advances in important areas such as assistive text generation.
Abstract (translated)
尽管视觉-语言模型（VLMs）在详细图像描述方面取得了进展，但评估仍然面临挑战。标准度量（如CIDEr、SPICE）是为短文本设计的，并调整以识别现在少见的错误，例如对象误识。相比之下，长文本需要对属性和关系附加的敏感性以及将误差定位到特定文本片段的分数。 在这项工作中，我们引入了PoSh，这是一种用于详细图像描述的度量标准，使用场景图作为结构化的评分准则来指导LLMs（大型语言模型）作为裁判进行操作，生成基于细粒度错误（例如组合理解中的错误）的聚合得分。PoSh具有可重复性、解释性和比现有指标（包括GPT4o-as-a-Judge）更好的人类评分代理能力。 为了验证PoSh的有效性，我们引入了一个新的挑战数据集DOCENT。这个新颖的基准测试包含了艺术品及其专家撰写的参考描述，并附有模型生成的描述和艺术史学生对其质量进行细化与粗略判断的数据。因此，DOCENT能够在一个具有挑战性的新领域评估详细图像描述度量标准以及详尽的图像描述本身。 我们展示了PoSh在DOCENT的人类评分中实现了比最佳开放权重替代方案更强的相关性（+0.05 Spearman $\rho$），并且对图像类型具有鲁棒性（使用CapArena，一个现有的网络图像数据集）。此外，它是一个有效的奖励函数，在标准监督微调方面表现出色。然后，利用PoSh分析了在描述DOCENT中的绘画、素描和雕像时开放模型与封闭模型的表现，并发现基础模型难以实现包含丰富场景动态的图像的完全无误覆盖，从而建立了一个评估VLM进展的新任务。 通过PoSh和DOCENT这两项工作，我们希望能够在辅助文本生成等重要领域取得进展。
URL
https://arxiv.org/abs/2510.19060
PDF
-
GPTFace: Generative Pre-training of Facial-Linguistic Transformer by Span Masking and Weakly Correlated Text-image Data
2025-10-21 06:55:44Yudong Li, Hao Li, Xianxu Hou, Linlin ShenAbstract
Compared to the prosperity of pre-training models in natural image understanding, the research on large-scale pre-training models for facial knowledge learning is still limited. Current approaches mainly rely on manually assembled and annotated face datasets for training, but labeling such datasets is labor-intensive and the trained models have limited scalability beyond the training data. To address these limitations, we present a generative pre-training model for facial knowledge learning that leverages large-scale web-built data for training. We use texts and images containing human faces crawled from the internet and conduct pre-training on self-supervised tasks, including masked image/language modeling (MILM) and image-text matching (ITM). During the generation stage, we further utilize the image-text matching loss to pull the generation distribution towards the control signal for controllable image/text generation. Experimental results demonstrate that our model achieves comparable performance to state-of-the-art pre-training models for various facial downstream tasks, such as attribution classification and expression recognition. Furthermore, our approach is also applicable to a wide range of face editing tasks, including face attribute editing, expression manipulation, mask removal, and photo inpainting.
Abstract (translated)
与自然图像理解中预训练模型的繁荣相比，针对面部知识学习的大规模预训练研究仍处于初级阶段。目前的方法主要依赖于手动收集和标注的人脸数据集进行训练，但这种标注过程费时费力，并且所训练出的模型在超出训练数据范围后可扩展性有限。为了解决这些问题，我们提出了一种基于大规模网络构建的数据来训练的生成式预训练模型，用于面部知识学习。我们从互联网上抓取包含人脸的文本和图像，并进行包括掩码图像/语言建模（MILM）和图像-文本匹配（ITM）在内的自监督任务的预训练。在生成阶段，进一步利用图像-文本匹配损失将生成分布拉向控制信号，以实现可控的图像/文本生成。 实验结果表明，在各种面部下游任务中，如属性分类和表情识别方面，我们的模型实现了与当前最先进的预训练模型相当的表现。此外，该方法还适用于广泛的面部编辑任务，包括人脸属性编辑、表情操作、口罩去除以及照片修补等。
URL
https://arxiv.org/abs/2510.18345
PDF
-
Planned Diffusion
2025-10-20 20:27:48Daniel Israel, Tian Jin, Ellie Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, Michael CarbinAbstract
A central challenge in large language model inference is the trade-off between generation speed and output quality. Autoregressive models produce high-quality text but generate tokens sequentially. Diffusion models can generate tokens in parallel but often need many iterations to match the same quality. We propose planned diffusion, a hybrid method that combines the strengths of both paradigms. Planned diffusion works in two stages: first, the model creates a short autoregressive plan that breaks the output into smaller, independent spans. Second, the model generates these spans simultaneously using diffusion. This approach expands the speed-quality Pareto frontier and provides a practical path to faster, high-quality text generation. On AlpacaEval, a suite of 805 instruction-following prompts, planned diffusion achieves Pareto-optimal trade-off between quality and latency, achieving 1.27x to 1.81x speedup over autoregressive generation with only 0.87\% to 5.4\% drop in win rate, respectively. Our sensitivity analysis shows that the planning mechanism of planned diffusion is minimal and reliable, and simple runtime knobs exist to provide flexible control of the quality-latency trade-off.
Abstract (translated)
大型语言模型推理中一个核心挑战是在生成速度和输出质量之间的权衡。自回归模型能够产生高质量的文本，但其生成令牌的过程是顺序进行的；而扩散模型可以并行生成令牌，但却常常需要多次迭代才能达到相同的质量水平。我们提出了一种名为“计划式扩散”的混合方法，该方法结合了这两种范式的优点。 计划式扩散的工作流程分为两个阶段：首先，模型创建一个简短的自回归计划，将输出分解为较小且独立的小片段；其次，在这一规划的基础上，模型同时生成这些小片段。这种方法扩展了速度与质量之间的帕累托前沿，并提供了一条实现更快、高质量文本生成的实际路径。 在包含805个指令跟随提示的AlpacaEval测试集上，计划式扩散实现了质量和延迟之间的帕累托最优权衡，在自回归生成的基础上分别取得了1.27倍到1.81倍的速度提升，同时仅以质量胜率减少0.87%至5.4%为代价。我们的敏感性分析表明，计划式扩散的规划机制是微小且可靠的，并存在一些简单的运行时控制参数来灵活地调控质量和延迟之间的权衡。 通过这种方法，大型语言模型能够更高效地生成高质量文本，在确保输出质量的同时显著提高生成速度。
URL
https://arxiv.org/abs/2510.18087
PDF
-
HouseTour: A Virtual Real Estate Agent
2025-10-20 19:47:35Ata \c{C}elen, Marc Pollefeys, Daniel Barath, Iro ArmeniAbstract
We introduce HouseTour, a method for spatially-aware 3D camera trajectory and natural language summary generation from a collection of images depicting an existing 3D space. Unlike existing vision-language models (VLMs), which struggle with geometric reasoning, our approach generates smooth video trajectories via a diffusion process constrained by known camera poses and integrates this information into the VLM for 3D-grounded descriptions. We synthesize the final video using 3D Gaussian splatting to render novel views along the trajectory. To support this task, we present the HouseTour dataset, which includes over 1,200 house-tour videos with camera poses, 3D reconstructions, and real estate descriptions. Experiments demonstrate that incorporating 3D camera trajectories into the text generation process improves performance over methods handling each task independently. We evaluate both individual and end-to-end performance, introducing a new joint metric. Our work enables automated, professional-quality video creation for real estate and touristic applications without requiring specialized expertise or equipment.
Abstract (translated)
我们介绍了HouseTour方法，这是一种从一系列描绘现有三维空间的图像中生成具有空间意识的3D相机轨迹和自然语言摘要的方法。与现有的视觉-语言模型（VLMs）不同，这些模型在几何推理方面存在困难，我们的方法通过一个受已知相机姿态约束的扩散过程来生成平滑的视频轨迹，并将此信息整合到VLM中以产生基于三维空间的描述。我们使用3D高斯点喷射技术合成最终视频，在整个轨迹上渲染新的视角视图。为了支持这项任务，我们提出了HouseTour数据集，该数据集包括超过1,200个房屋导览视频、相机姿态、三维重建和房地产描述。实验表明，将3D摄像机轨迹纳入文本生成过程中可以改善独立处理每个任务的方法的性能。我们评估了单独的任务性能以及端到端的整体性能，并引入了一个新的联合评价指标。我们的工作使得无需专门的技术知识或设备便能够自动创建出高质量的专业级视频，适用于房地产和旅游应用领域。
URL
https://arxiv.org/abs/2510.18054
PDF
-
AcademicEval: Live Long-Context LLM Benchmark
2025-10-20 16:42:30Haozhen Zhang, Tao Feng, Pengrui Han, Jiaxuan YouAbstract
Large Language Models (LLMs) have recently achieved remarkable performance in long-context understanding. However, current long-context LLM benchmarks are limited by rigid context length, labor-intensive annotation, and the pressing challenge of label leakage issues during LLM training. Therefore, we propose \textsc{AcademicEval}, a live benchmark for evaluating LLMs over long-context generation tasks. \textsc{AcademicEval} adopts papers on arXiv to introduce several academic writing tasks with long-context inputs, \textit{i.e.}, \textsc{Title}, \textsc{Abstract}, \textsc{Introduction}, and \textsc{Related Work}, which cover a wide range of abstraction levels and require no manual labeling. Moreover, \textsc{AcademicEval} integrates high-quality and expert-curated few-shot demonstrations from a collected co-author graph to enable flexible context length. Especially, \textsc{AcademicEval} features an efficient live evaluation, ensuring no label leakage. We conduct a holistic evaluation on \textsc{AcademicEval}, and the results illustrate that LLMs perform poorly on tasks with hierarchical abstraction levels and tend to struggle with long few-shot demonstrations, highlighting the challenge of our benchmark. Through experimental analysis, we also reveal some insights for enhancing LLMs' long-context modeling capabilities. Code is available at this https URL
Abstract (translated)
最近，大型语言模型（LLMs）在处理长上下文理解方面取得了显著的性能提升。然而，现有的长上下文评估基准面临固定上下文长度、标注工作量大以及训练过程中标签泄露等问题。为此，我们提出了一个名为\textsc{AcademicEval} 的实时基准测试工具，用于评测LLM在长上下文生成任务中的表现。 \textsc{AcademicEval} 使用arXiv上的论文引入了几项涉及长上下文输入的学术写作任务，即标题、摘要、引言和相关工作部分。这些任务涵盖了广泛的抽象层级，并且无需人工标注。此外，\textsc{AcademicEval} 结合了一种高质量的、由专家策划的少量样本演示方法，该方法基于收集到的共作者图来实现灵活的上下文长度。特别地，\textsc{AcademicEval} 设计了高效的实时评估机制，确保没有标签泄露问题。 我们对\textsc{AcademicEval} 进行了全面评测，并且结果表明LLM在具有层次抽象水平的任务上表现不佳，而且难以处理长的少量样本演示，这突显出了该基准测试的独特挑战。通过实验分析，我们也揭示了一些增强LLM长上下文建模能力的见解。 代码可在以下链接获取：[提供具体的URL地址]
URL
https://arxiv.org/abs/2510.17725
PDF
-
Qomhra: A Bilingual Irish-English Large Language Model
2025-10-20 15:27:53Joseph McInerneyAbstract
This paper introduces Qomhrá, a bilingual Irish-English large language model (LLM), developed under low-resource constraints presenting a complete pipeline spanning bilingual continued pre-training, instruction tuning, and alignment from human preferences. Newly accessible Irish corpora and English text are mixed and curated to improve Irish performance while preserving English ability. 6 closed-weight LLMs are judged for their Irish text generation by a native speaker, a learner and other LLMs. Google's Gemini-2.5-Pro is ranked the highest and is subsequently used to synthesise instruction tuning and human preference datasets. Two datasets are contributed leveraging Gemini-2.5-Pro: a 30K Irish-English parallel instruction tuning dataset and a 1K human preference dataset, generating accepted and rejected responses that show near perfect alignment with a native Irish speaker. Qomhrá is comprehensively evaluated across benchmarks testing translation, gender understanding, topic identification and world knowledge with gains of up to 29% in Irish and 44% in English. Qomhrá also undergoes instruction tuning and demonstrates clear progress in instruction following, crucial for chatbot functionality.
Abstract (translated)
这篇论文介绍了Qomhrá，这是一个在资源有限的情况下开发的爱尔兰语-英语双语大型语言模型（LLM）。该模型涵盖了从双语持续预训练、指令微调到人类偏好的对齐在内的完整流程。通过混合和整理新的可获取的爱尔兰语语料库和英语文本，以提高爱尔兰语的表现同时保持其在英语方面的能力。六种封闭权重的LLM由一位母语使用者、学习者以及其他LLM对其生成的爱尔兰语文本进行了评估。Google的Gemini-2.5-Pro排名最高，并被用于合成指令微调和人类偏好数据集。该研究贡献了两个基于Gemini-2.5-Pro的数据集：一个30K条目的爱尔兰语-英语并行指令微调数据集，以及一个人类偏好数量为1K的偏好数据集，这两个数据集中包含了母语使用者接受或拒绝的回答，这些回答几乎完美地与一位爱尔兰母语者的要求一致。Qomhrá在跨基准测试中表现出色，在翻译、性别理解、话题识别和世界知识方面分别提高了高达29%（爱尔兰语）和44%（英语）。此外，通过指令微调，Qomhrá还展示了明显遵循指令的进步，这对于聊天机器人的功能至关重要。
URL
https://arxiv.org/abs/2510.17652
PDF
-
Language Confusion Gate: Language-Aware Decoding Through Model Self-Distillation
2025-10-20 14:02:37Collin Zhang, Fei Huang, Chenhan Yuan, Junyang LinAbstract
Large language models (LLMs) often experience language confusion, which is the unintended mixing of languages during text generation. Current solutions to this problem either necessitate model retraining or cannot differentiate between harmful confusion and acceptable code-switching. This paper introduces the Language Confusion Gate (LCG), a lightweight, plug-in solution that filters tokens during decoding without altering the base LLM. The LCG is trained using norm-adjusted self-distillation to predict appropriate language families and apply masking only when needed. Our method is based on the findings that language confusion is infrequent, correct-language tokens are usually among the top predictions, and output token embedding norms are larger for high-resource languages, which biases sampling. When evaluated across various models, including Qwen3, GPT-OSS, Gemma3, Llama3.1, LCG decreases language confusion significantly, often by an order of magnitude, without negatively impacting task performance. Code is available at this https URL.
Abstract (translated)
大型语言模型（LLMs）常常会遇到语言混淆的问题，即在文本生成过程中意外地混杂使用不同语言。目前解决这个问题的方法要么需要重新训练模型，要么无法区分有害的混淆和可接受的语言转换（code-switching）。本文介绍了一种名为“语言混淆门”（Language Confusion Gate, LCG）的技术，这是一种轻量级、插件式的解决方案，在解码过程中过滤掉不需要的标记，而无需修改基础LLM。LCG通过调整范数的自我蒸馏训练方法来预测合适的语言家族，并仅在必要时应用屏蔽操作。 本文的方法基于以下发现：语言混淆是相对罕见的现象；正确语言中的词汇通常出现在模型输出的最前面位置；资源丰富的语言（如英语）的输出标记嵌入规范较大，这会偏向于采样这些语言。通过跨多个模型进行评估，包括Qwen3、GPT-OSS、Gemma3和Llama3.1等，LCG显著减少了语言混淆现象，有时甚至可以减少一个数量级，并且不会对任务性能产生负面影响。 代码可以在提供的链接地址获取。
URL
https://arxiv.org/abs/2510.17555
PDF
-
DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning
2025-10-20 12:41:44Yongxin He, Shan Zhang, Yixuan Cao, Lei Ma, Ping LuoAbstract
Detecting AI-involved text is essential for combating misinformation, plagiarism, and academic misconduct. However, AI text generation includes diverse collaborative processes (AI-written text edited by humans, human-written text edited by AI, and AI-generated text refined by other AI), where various or even new LLMs could be involved. Texts generated through these varied processes exhibit complex characteristics, presenting significant challenges for detection. Current methods model these processes rather crudely, primarily employing binary classification (purely human vs. AI-involved) or multi-classification (treating human-AI collaboration as a new class). We observe that representations of texts generated through different processes exhibit inherent clustering relationships. Therefore, we propose DETree, a novel approach that models the relationships among different processes as a Hierarchical Affinity Tree structure, and introduces a specialized loss function that aligns text representations with this tree. To facilitate this learning, we developed RealBench, a comprehensive benchmark dataset that automatically incorporates a wide spectrum of hybrid texts produced through various human-AI collaboration processes. Our method improves performance in hybrid text detection tasks and significantly enhances robustness and generalization in out-of-distribution scenarios, particularly in few-shot learning conditions, further demonstrating the promise of training-based approaches in OOD settings. Our code and dataset are available at this https URL.
Abstract (translated)
检测包含AI参与的文本对于打击错误信息、抄袭和学术不端行为至关重要。然而，AI生成的文本包括各种协作过程（如由人类编辑过的AI写作、经过AI编辑的人类写作以及被其他AI进一步精炼的AI生成文本），这些过程中可能涉及不同的甚至是新的大型语言模型(LLMs)。通过这些多样化的过程产生的文本具有复杂的特性，为检测带来了重大挑战。当前的方法较为粗糙地建模这些过程，主要采用二元分类（纯粹由人类编写 vs. 包含AI参与）或多类别分类（将人机协作视为一个新类别）。我们观察到，不同过程中生成的文本在表示上存在固有的聚类关系。因此，我们提出了DETree这一创新方法，该方法通过层次亲和树结构来建模各种过程之间的关系，并引入了一种特殊的损失函数以使文本表示与这种树相一致。为了支持这种方法的学习，我们开发了RealBench，这是一个全面的基准数据集，能够自动整合通过多种人机协作过程产生的广泛混合类型文本。我们的方法在混合文本检测任务中提高了性能，并显著增强了面对分布外（OOD）场景时的鲁棒性和泛化能力，特别是在少样本学习条件下。这进一步展示了基于训练的方法在OOD设置中的前景。我们的代码和数据集可以在以下链接获取：[此URL]。 注意：上述链接需要替换为实际可用的链接地址。
URL
https://arxiv.org/abs/2510.17489
PDF
-
Check Yourself Before You Wreck Yourself: Selectively Quitting Improves LLM Agent Safety
2025-10-18 13:22:19Vamshi Krishna Bonagiri, Ponnurangam Kumaragurum, Khanh Nguyen, Benjamin PlautAbstract
As Large Language Model (LLM) agents increasingly operate in complex environments with real-world consequences, their safety becomes critical. While uncertainty quantification is well-studied for single-turn tasks, multi-turn agentic scenarios with real-world tool access present unique challenges where uncertainties and ambiguities compound, leading to severe or catastrophic risks beyond traditional text generation failures. We propose using "quitting" as a simple yet effective behavioral mechanism for LLM agents to recognize and withdraw from situations where they lack confidence. Leveraging the ToolEmu framework, we conduct a systematic evaluation of quitting behavior across 12 state-of-the-art LLMs. Our results demonstrate a highly favorable safety-helpfulness trade-off: agents prompted to quit with explicit instructions improve safety by an average of +0.39 on a 0-3 scale across all models (+0.64 for proprietary models), while maintaining a negligible average decrease of -0.03 in helpfulness. Our analysis demonstrates that simply adding explicit quit instructions proves to be a highly effective safety mechanism that can immediately be deployed in existing agent systems, and establishes quitting as an effective first-line defense mechanism for autonomous agents in high-stakes applications.
Abstract (translated)
随着大型语言模型（LLM）代理越来越多地在具有现实世界后果的复杂环境中运行，其安全性变得至关重要。虽然不确定性量化对于单轮任务已有深入研究，但多轮代理场景中与现实世界工具互动的情况带来了独特的挑战：不确定性和模糊性会累积起来，导致超出传统文本生成失败的风险，可能会引发严重或灾难性的风险。我们提出使用“退出”作为简单而有效的行为机制，使LLM代理在缺乏信心的情况下能够识别并从当前情境中撤出。 利用ToolEmu框架，我们在12种最先进的LLM上对退出行为进行了系统的评估。我们的结果显示，在安全性和有用性之间存在一个高度有利的权衡：通过明确指令提示代理退出，在0到3的评分尺度上，所有模型的安全性能平均提高+0.39（专有模型为+0.64），同时保持了微不足道的帮助性能平均下降-0.03。我们的分析表明，简单地添加明确的退出指令被证明是一种高度有效的安全机制，可以立即部署在现有的代理系统中，并且确定“退出”作为自主代理在高风险应用中的有效一线防御机制。
URL
https://arxiv.org/abs/2510.16492
PDF
-
GraphMind: Interactive Novelty Assessment System for Accelerating Scientific Discovery
2025-10-17 14:49:07Italo Luis da Silva, Hanqi Yan, Lin Gui, Yulan HeAbstract
Large Language Models (LLMs) show strong reasoning and text generation capabilities, prompting their use in scientific literature analysis, including novelty assessment. While evaluating novelty of scientific papers is crucial for peer review, it requires extensive knowledge of related work, something not all reviewers have. While recent work on LLM-assisted scientific literature analysis supports literature comparison, existing approaches offer limited transparency and lack mechanisms for result traceability via an information retrieval module. To address this gap, we introduce $\textbf{GraphMind}$, an easy-to-use interactive web tool designed to assist users in evaluating the novelty of scientific papers or drafted ideas. Specially, $\textbf{GraphMind}$ enables users to capture the main structure of a scientific paper, explore related ideas through various perspectives, and assess novelty via providing verifiable contextual insights. $\textbf{GraphMind}$ enables users to annotate key elements of a paper, explore related papers through various relationships, and assess novelty with contextual insight. This tool integrates external APIs such as arXiv and Semantic Scholar with LLMs to support annotation, extraction, retrieval and classification of papers. This combination provides users with a rich, structured view of a scientific idea's core contributions and its connections to existing work. $\textbf{GraphMind}$ is available at this https URL and a demonstration video at this https URL. The source code is available at this https URL.
Abstract (translated)
大型语言模型（LLMs）展示了强大的推理和文本生成能力，因此被用于科学文献分析，包括新颖性评估。尽管在同行评审中评价科学论文的新颖性至关重要，但这项工作需要广泛了解相关领域的工作，而这并不是所有审稿人都具备的。 虽然最近关于利用LLM辅助进行科学文献分析的研究支持了文献比较的功能，但是现有方法提供的透明度有限，并且没有通过信息检索模块实现结果可追溯性的机制。为解决这一缺口，我们引入了一种易于使用的交互式网络工具$\textbf{GraphMind}$，旨在帮助用户评估科学论文或草稿想法的新颖性。 具体来说，$\textbf{GraphMind}$使用户能够捕捉一篇科学论文的主要结构，从不同角度探索相关概念，并通过提供可验证的上下文洞察来评估新颖性。该工具允许用户标注论文的关键元素、通过各种关系探索相关的其他文献，并利用情境洞察进行新颖性的评估。 这种结合了arXiv和Semantic Scholar等外部API与LLM的技术为用户提供了一个丰富且结构化的视角，展示科学想法的核心贡献及其与现有工作的联系。 $\textbf{GraphMind}$可在[这个网址](https://example.com/graphmind)获取，并提供[演示视频链接](https://example.com/video)，源代码可访问[这里](https://example.com/sourcecode)。
URL
https://arxiv.org/abs/2510.15706
PDF
-
Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection
2025-10-17 14:28:57Joshua Wolfe Brook, Ilia MarkovAbstract
This research introduces a novel approach to textual and multimodal Hate Speech Detection (HSD), using Large Language Models (LLMs) as dynamic knowledge bases to generate background context and incorporate it into the input of HSD classifiers. Two context generation strategies are examined: one focused on named entities and the other on full-text prompting. Four methods of incorporating context into the classifier input are compared: text concatenation, embedding concatenation, a hierarchical transformer-based fusion, and LLM-driven text enhancement. Experiments are conducted on the textual Latent Hatred dataset of implicit hate speech and applied in a multimodal setting on the MAMI dataset of misogynous memes. Results suggest that both the contextual information and the method by which it is incorporated are key, with gains of up to 3 and 6 F1 points on textual and multimodal setups respectively, from a zero-context baseline to the highest-performing system, based on embedding concatenation.
Abstract (translated)
这项研究提出了一种新颖的方法来检测文本和多模态仇恨言论（HSD），该方法利用大型语言模型（LLMs）作为动态知识库，生成背景信息并将其整合到HSD分类器的输入中。研究考察了两种上下文生成策略：一种侧重于命名实体，另一种侧重于全文提示。比较了四种将上下文融入分类器输入的方法：文本拼接、嵌入拼接、基于分层变换器融合和由LLM驱动的文本增强方法。实验在包含隐含仇恨言论的Latent Hatred数据集上进行，并在包含厌女表情包的MAMI多模态数据集中应用。结果表明，上下文信息及其整合方式至关重要，在从零背景到最高性能系统的基准上，文本和多模态设置分别提高了高达3分和6分的F1分数（基于嵌入拼接）。
URL
https://arxiv.org/abs/2510.15685
PDF
-
Accelerating Mobile Language Model Generation via Hybrid Context and Hardware Coordination
2025-10-17 04:59:43Zhiyang Chen, Daliang Xu, Haiyang Shen, Mengwei Xu, Shangguang Wang, Yun MaAbstract
Enhancing on-device large language models (LLMs) with contextual information from local data enables personalized and task-aware generation, powering use cases such as intelligent assistants and UI agents. While recent developments in neural processors have substantially improved the efficiency of prefill on mobile devices, the token-by-token generation process still suffers from high latency and limited hardware utilization due to its inherently memory-bound characteristics. This work presents CoordGen, a mobile inference framework that integrates speculative decoding with dynamic hardware scheduling to accelerate context-aware text generation on mobile devices. The framework introduces three synergistic components: (1) adaptive execution scheduling, which dynamically balances compute graphs between prefill and decoding phases; (2) context-aligned drafting, which improves speculative efficiency through lightweight online calibration to current tasks; and (3) hardware-efficient draft extension, which reuses and expands intermediate sequences to improve processing parallelism and reduce verification cost. Experiments on multiple smartphones and representative workloads show consistent improvements of up to 3.8x in generation speed and 4.7x in energy efficiency compared with existing mobile inference solutions. Component-level analysis further validates the contribution of each optimization.
Abstract (translated)
通过在设备上的大型语言模型（LLM）中加入来自本地数据的上下文信息，可以实现个性化和任务感知的生成功能，支持诸如智能助手和UI代理等应用场景。尽管最近在神经处理器方面的进展大大提高了移动设备上预填充过程的效率，但由于生成过程中固有的内存瓶颈特性，逐令牌生成仍然面临着高延迟和有限硬件利用率的问题。 这项工作提出了一种名为CoordGen的移动推理框架，该框架通过将推测性解码与动态硬件调度相结合，加速了在移动设备上下文感知文本生成的过程。框架引入了三个相互协同的组件： 1. **自适应执行调度**：这种机制能够根据当前任务需求，在预填充和解码阶段之间动态平衡计算图。 2. **上下文对齐草稿**：通过轻量级在线校准，提升推测效率，使得生成过程更加贴近实际任务需求。 3. **硬件高效的草稿扩展**：该组件复用并扩展中间序列以提高处理的并行性，并减少验证成本。 在多个智能手机和典型工作负载上的实验表明，相比现有的移动推理解决方案，CoordGen框架能够在文本生成速度上实现最高达3.8倍的提升，在能源效率方面则可达到4.7倍的增长。组件级别的分析进一步证实了每一项优化对于整体性能改进的具体贡献。
URL
https://arxiv.org/abs/2510.15312
PDF
-
MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics
2025-10-16 17:55:14Yuxing Lu, Xukai Zhao, J. Ben Tamo, Micky C. Nnamdi, Rui Peng, Shuang Zeng, Xingyu Hu, Jinzhuo Wang, May D. WangAbstract
Large Language Models (LLMs) have demonstrated remarkable capabilities on general text; however, their proficiency in specialized scientific domains that require deep, interconnected knowledge remains largely uncharacterized. Metabolomics presents unique challenges with its complex biochemical pathways, heterogeneous identifier systems, and fragmented databases. To systematically evaluate LLM capabilities in this domain, we introduce MetaBench, the first benchmark for metabolomics assessment. Curated from authoritative public resources, MetaBench evaluates five capabilities essential for metabolomics research: knowledge, understanding, grounding, reasoning, and research. Our evaluation of 25 open- and closed-source LLMs reveals distinct performance patterns across metabolomics tasks: while models perform well on text generation tasks, cross-database identifier grounding remains challenging even with retrieval augmentation. Model performance also decreases on long-tail metabolites with sparse annotations. With MetaBench, we provide essential infrastructure for developing and evaluating metabolomics AI systems, enabling systematic progress toward reliable computational tools for metabolomics research.
Abstract (translated)
大型语言模型（LLMs）在通用文本方面展示了卓越的能力；然而，它们在需要深厚、相互关联知识的专门化科学领域中的专业能力仍然缺乏系统性的表征。代谢组学因其复杂的生化途径、异质性标识符系统和碎片化的数据库而提出了独特的挑战。为了系统地评估LLM在该领域的表现，我们引入了MetaBench——第一个用于代谢组学评估的基准测试工具。MetaBench从权威的公共资源中精心挑选内容，以评估代谢组学研究所需的五种关键能力：知识、理解、实体关联、推理和科研支持。对25个开源及闭源LLM进行的评估显示，在代谢组学任务上表现出不同的性能模式：尽管模型在文本生成任务上表现良好，但在跨数据库标识符关联的任务中仍然面临挑战，即使增加了检索增强也无法显著改善这一状况。此外，对于那些注释稀疏的长尾代谢物，模型的表现也会下降。通过MetaBench，我们为开发和评估代谢组学AI系统提供了重要的基础设施，从而推动了可靠计算工具在代谢组学研究中的系统性进步。
URL
https://arxiv.org/abs/2510.14944
PDF
-
Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models
2025-10-16 16:24:36Guinan Su, Yanwu Yang, Li Shen, Lu Yin, Shiwei Liu, Jonas GeipingAbstract
Mixture-of-Experts (MoE) models achieve efficient scaling through sparse expert activation, but often suffer from suboptimal routing decisions due to distribution shifts in deployment. While existing test-time adaptation methods could potentially address these issues, they primarily focus on dense models and require access to external data, limiting their practical applicability to MoE architectures. However, we find that, instead of relying on reference data, we can optimize MoE expert selection on-the-fly based only on input context. As such, we propose \textit{a data-free, online test-time framework} that continuously adapts MoE routing decisions during text generation without external supervision or data. Our method cycles between two phases: During the prefill stage, and later in regular intervals, we optimize the routing decisions of the model using self-supervision based on the already generated sequence. Then, we generate text as normal, maintaining the modified router until the next adaption. We implement this through lightweight additive vectors that only update router logits in selected layers, maintaining computational efficiency while preventing over-adaptation. The experimental results show consistent performance gains on challenging reasoning tasks while maintaining robustness to context shifts. For example, our method achieves a 5.5\% improvement on HumanEval with OLMoE. Furthermore, owing to its plug-and-play property, our method naturally complements existing test-time scaling techniques, e.g., achieving 6\% average gains when incorporated with self-consistency on DeepSeek-V2-Lite.
Abstract (translated)
混合专家（Mixture-of-Experts，MoE）模型通过稀疏的专家激活实现了高效的扩展能力，但在部署过程中由于分布偏移往往会导致次优的路由决策。尽管现有的测试时间适应方法可以解决这些问题，但它们主要针对密集型模型，并且需要外部数据访问权限，这限制了其在MoE架构中的实际应用性。然而，我们发现，可以通过仅基于输入上下文来优化MoE专家选择，而无需依赖参考数据。因此，我们提出了一种“无数据、在线测试时间框架”，该框架可以在文本生成过程中持续调整MoE的路由决策，且不需外部监督或数据。 我们的方法在两个阶段之间循环进行：在预填充阶段以及之后的定期间隔内，模型使用自监督来优化其基于已经生成序列的路由决策。然后，我们像正常一样生成文本，并保持修改后的路由器直到下一次适应为止。通过轻量级加性向量实现这一过程，这些向量仅更新选定层中的路由器日志几率（logits），从而在防止过度调整的同时维持计算效率。 实验结果显示，在具有挑战性的推理任务上性能得到了持续的提升，并且能够保持对上下文变化的强大鲁棒性。例如，在使用OLMoE模型时，我们的方法实现了5.5%的人类评估改进。此外，由于其即插即用特性，该方法自然地补充了现有的测试时间扩展技术，如在与深度搜索V2 Lite的自我一致性结合使用时可以获得平均6%的性能提升。
URL
https://arxiv.org/abs/2510.14853
PDF
