Abstract
In online advertising, advertising text plays a critical role in attracting user engagement and driving advertiser value. Existing industrial systems typically follow a two-stage paradigm, where candidate texts are first generated and subsequently aligned with online performance metrics such as click-through rate(CTR). This separation often leads to misaligned optimization objectives and low funnel efficiency, limiting global optimality. To address these limitations, we propose RELATE, a reinforcement learning-based end-to-end framework that unifies generation and objective alignment within a single model. Instead of decoupling text generation from downstream metric alignment, RELATE integrates performance and compliance objectives directly into the generation process via policy learning. To better capture ultimate advertiser value beyond click-level signals, We incorporate conversion-oriented metrics into the objective and jointly model them with compliance constraints as multi-dimensional rewards, enabling the model to generate high-quality ad texts that improve conversion performance under policy constraints. Extensive experiments on large-scale industrial datasets demonstrate that RELATE consistently outperforms baselines. Furthermore, online deployment on a production advertising platform yields statistically significant improvements in click-through conversion rate(CTCVR) under strict policy constraints, validating the robustness and real-world effectiveness of the proposed framework.
Abstract (translated)
在在线广告领域，广告文案对于吸引用户互动和增加广告主价值起着关键作用。现有的工业系统通常遵循两阶段模式：首先是生成候选文本，然后将这些文本与点击率（CTR）等线上性能指标进行对齐。这种分离往往导致优化目标不一致及低效率的漏斗效应，从而限制了全局最优性。为了解决这些问题，我们提出了一种基于强化学习的端到端框架——RELATE，它通过单一模型统一生成和目标对齐过程。与将文本生成从下游指标对齐中分离不同，RELATE 通过策略学习直接将性能和合规性目标融入生成过程中。 为了更好地捕捉超出点击级信号的最终广告价值，我们引入了面向转化率的目标，并将其与合规约束共同建模为多维奖励，使模型能够在政策限制下生成高质量且能提升转化表现的广告文案。大规模工业数据集上的广泛实验表明，RELATE 一直优于基线方法。此外，在一个生产广告平台上进行的在线部署在严格的政策条件下显著提高了点击后转化率（CTCVR），验证了所提框架的稳健性和实际效果。
URL
https://arxiv.org/abs/2602.11780
