Abstract
In this work we address supervised learning via lifted network formulations. Lifted networks are interesting because they allow training on massively parallel hardware and assign energy models to discriminatively trained neural networks. We demonstrate that training methods for lifted networks proposed in the literature have significant limitations, and therefore we propose to use a contrastive loss to train lifted networks. We show that this contrastive training approximates back-propagation in theory and in practice, and that it is superior to the regular training objective for lifted networks.
Abstract (translated)
在这项工作中,我们通过提升的网络公式处理监督学习。提升网络之所以有趣,是因为它们允许在大规模并行硬件上进行训练,并将能量模型分配给有区别的训练神经网络。我们证明文献中提出的提升网络的训练方法有很大的局限性,因此我们建议使用对比损失来训练提升网络。结果表明,这种对比训练在理论和实践上都近似于反向传播,优于常规的提升网络训练目标。
URL
https://arxiv.org/abs/1905.02507