Abstract
Author response (rebuttal) writing is a critical stage of scientific peer review that demands substantial author effort. Recent work frames this task as automatic text generation, underusing author expertise and intent. In practice, authors possess domain expertise, author-only information, revision and response strategies--concrete forms of author expertise and intent--to address reviewer concerns, and seek NLP assistance that integrates these signals to support effective response writing in peer review. We reformulate author response generation as an author-in-the-loop task and introduce REspGen, a generation framework that integrates explicit author input, multi-attribute control, and evaluation-guided refinement, together with REspEval, a comprehensive evaluation suite with 20+ metrics covering input utilization, controllability, response quality, and discourse. To support this formulation, we construct Re$^3$Align, the first large-scale dataset of aligned review--response--revision triplets, where revisions provide signals of author expertise and intent. Experiments with state-of-the-art LLMs show the benefits of author input and evaluation-guided refinement, the impact of input design on response quality, and trade-offs between controllability and quality. We make our dataset, generation and evaluation tools publicly available.
Abstract (translated)
作者回复（反驳）撰写是科学同行评审中的一个关键阶段，这一过程需要作者投入大量精力。近期的研究将这项任务视为自动文本生成的任务，这在很大程度上忽视了作者的专业知识和意图。实际上，在实践中，作者掌握着领域专业知识、仅作者所知的信息以及应对审稿人关注的修订与回复策略——这些都是构成作者专业知识和意图的具体形式，并寻求自然语言处理（NLP）的支持以整合这些信号来支持有效的同行评审回复撰写。我们重新定义了作者响应生成为一种“作者在回路”任务，引入了REspGen框架，该框架集成了明确的作者输入、多属性控制以及指导性评估改进的功能，并且还提出了REspEval，一个包含20多项以上指标的全面评估套件，涵盖了输入利用、可控制性、响应质量及对话体裁。为了支持这一构想，我们构建了Re$^3$Align，这是首个大规模对齐的审稿-回复-修订三元组数据集，在此数据集中，修订提供了有关作者专业知识和意图的信号。 实验显示，最先进的大型语言模型在引入明确的作者输入以及评估指导改进后展现出显著的优势；探讨了输入设计对于响应质量的影响，并揭示出可控制性与质量之间的权衡。我们将该数据集、生成工具及评价工具向公众开放。
URL
https://arxiv.org/abs/2602.11173
