Abstract
While neural language models have recently demonstrated impressive performance in unconditional text generation, controllable generation and manipulation of text remain challenging. Latent variable generative models provide a natural approach for control, but their application to text has proven more difficult than to images. Models such as variational autoencoders may suffer from posterior collapse or learning an irregular latent geometry. We propose to instead employ adversarial autoencoders (AAEs) and add local perturbations by randomly replacing/removing words from input sentences during training. Within the prior enforced by the adversary, structured perturbations in the data space begin to carve and organize the latent space. Theoretically, we prove that perturbations encourage similar sentences to map to similar latent representations. Experimentally, we investigate the trade-off between text-generation and autoencoder-reconstruction capabilities. Our straightforward approach significantly improves over regular AAEs as well as other autoencoders, and enables altering the tense/sentiment of sentences through simple addition of a fixed vector offset to their latent representation.
Abstract (translated)
虽然神经语言模型最近在无条件文本生成方面表现出了令人印象深刻的性能，但文本的可控生成和操作仍然具有挑战性。潜在变量生成模型为控制提供了一种自然的方法，但事实证明，它们在文本中的应用比图像更困难。像变分自动编码器这样的模型可能会出现后塌陷或学习不规则的潜在几何结构。我们建议在培训期间使用对抗性自动编码器（aaes）并通过随机替换/删除输入句子中的单词来增加局部干扰。在敌方实施的先例中，数据空间中的结构化扰动开始分割和组织潜在空间。理论上，我们证明了扰动会促使相似的句子映射到相似的潜在表示。实验上，我们研究了文本生成和自动编码器重建能力之间的权衡。我们的直截了当的方法比常规的aaes和其他自动编码器有了显著的改进，并且能够通过简单地添加一个固定向量来偏移句子的潜在表示来改变句子的时态/情感。
URL
https://arxiv.org/abs/1905.12777
