Abstract
Existing controllable text generation systems rely on annotated attributes, which greatly limits their capabilities and applications. In this work, we make the first successful attempt to use VAEs to achieve controllable text generation without supervision. We do so by decomposing the latent space of the VAE into two parts: one incorporates structural constraints to capture dominant global variations implicitly present in the data, e.g., sentiment or topic; the other is unstructured and is used for the reconstruction of the source sentences. With the enforced structural constraint, the underlying global variations will be discovered and disentangled during the training of the VAE. The structural constraint also provides a natural recipe for mitigating posterior collapse for the structured part, which cannot be fully resolved by the existing techniques. On the task of text style transfer, our unsupervised approach achieves significantly better performance than previous supervised approaches. By showcasing generation with finer-grained control including Cards-Against-Humanity-style topic transitions within a sentence, we demonstrate that our model can perform controlled text generation in a more flexible way than existing methods.
Abstract (translated)
现有的可控文本生成系统依赖于注释属性，这极大地限制了它们的功能和应用。在这项工作中，我们首次成功地尝试使用vaes来实现无监督的可控文本生成。我们通过将vae的潜在空间分解为两部分来实现这一点：一部分包含结构约束，以捕获数据中隐含的主要全局变化，例如情感或主题；另一部分是非结构化的，用于源语句的重构。在强制结构约束下，在虚拟企业的培训过程中，将发现并解开潜在的全局变化。结构约束还提供了一种减轻结构部件后塌陷的自然方法，现有技术无法完全解决后塌陷问题。在文本样式转换任务中，我们的无监督方法比以前的有监督方法获得了显著的更好的性能。通过在一个句子中展示带有更细粒度控制的生成，包括针对人性风格主题转换的卡片，我们证明了我们的模型可以以比现有方法更灵活的方式执行受控文本生成。
URL
https://arxiv.org/abs/1905.11975
