Abstract
We propose a topic-guided variational autoencoder (TGVAE) model for text generation. Distinct from existing variational autoencoder (VAE) based approaches, which assume a simple Gaussian prior for the latent code, our model specifies the prior as a Gaussian mixture model (GMM) parametrized by a neural topic module. Each mixture component corresponds to a latent topic, which provides guidance to generate sentences under the topic. The neural topic module and the VAE-based neural sequence module in our model are learned jointly. In particular, a sequence of invertible Householder transformations is applied to endow the approximate posterior of the latent code with high flexibility during model inference. Experimental results show that our TGVAE outperforms alternative approaches on both unconditional and conditional text generation, which can generate semantically-meaningful sentences with various topics.
Abstract (translated)
我们提出一个主题引导的变分自动编码器（TGVAE）模型用于文本生成。与现有的基于变分自动编码器（VAE）的方法不同，该方法假定潜在代码具有简单的高斯先验值，我们的模型将先验值指定为由神经主题模块参数化的高斯混合模型（GMM）。每个混合成分对应一个潜在的主题，它提供了在主题下生成句子的指导。将神经主题模块和基于VAE的神经序列模块结合起来学习。特别地，在模型推理过程中，采用了一系列可逆的户主变换，使得潜在代码的近似后验具有很高的灵活性。实验结果表明，我们的tgvae在无条件和条件文本生成方面都优于其他方法，它可以生成具有不同主题的语义意义的句子。
URL
https://arxiv.org/abs/1903.07137
