Abstract
Large language models perform well in short text generation but still struggle with long text generation, particularly under complex constraints. Such tasks involve multiple tightly coupled objectives, including global structural consistency, local semantic coherence, and constraint feasibility, forming a challenging constrained optimization problem. Existing approaches mainly rely on static planning or offline supervision, limiting effective coordination between global and local objectives during generation. To address these challenges, we propose HiFlow, a hierarchical feedback-driven optimization framework for constrained long text generation. HiFlow formulates generation as a two-level optimization process, consisting of a planning layer for global structure and constraint modeling, and a generation layer for conditioned text generation. By incorporating constraint-aware plan screening and closed-loop feedback at both levels, HiFlow enables joint optimization of planning quality and generation behavior, progressively guiding the model toward high-quality, constraint-satisfying outputs. Experiments on multiple backbones confirm HiFlow's effectiveness over baseline methods.
Abstract (translated)
大型语言模型在短文本生成方面表现出色，但在长文本生成方面仍然面临挑战，尤其是在存在复杂约束的情况下。这类任务涉及多个紧密耦合的目标，包括全局结构一致性、局部语义连贯性和约束可行性，形成了一个具有挑战性的受限优化问题。现有的方法主要依赖于静态规划或离线监督，这限制了在生成过程中全局和局部目标的有效协调。为了解决这些挑战，我们提出了HiFlow，这是一种分层反馈驱动的优化框架，用于受约束的长文本生成。HiFlow将生成过程表述为一个两层次的优化过程，包括用于全局结构和约束建模的规划层以及用于条件文本生成的生成层。通过在两个层面中加入考虑约束的计划筛选与闭环反馈机制，HiFlow能够同时优化规划质量和生成行为，逐步引导模型产生高质量且符合约束要求的输出。实验证明了HiFlow相对于基线方法的有效性。
URL
https://arxiv.org/abs/2603.04996
