Abstract
Recent developments using End-to-End Deep Learning models have been shown to have near or better performance than state of the art Recurrent Neural Networks (RNNs) on Automatic Speech Recognition tasks. These models tend to be lighter weight and require less training time than traditional RNN-based approaches. However, these models take frequentist approach to weight training. In theory, network weights are drawn from a latent, intractable probability distribution. We introduce BayesSpeech for end-to-end Automatic Speech Recognition. BayesSpeech is a Bayesian Transformer Network where these intractable posteriors are learned through variational inference and the local reparameterization trick without recurrence. We show how the introduction of variance in the weights leads to faster training time and near state-of-the-art performance on LibriSpeech-960.
Abstract (translated)
最近使用端到端深度学习模型的发展表明,它们在自动语音识别任务中的表现接近于或优于最先进的循环神经网络(RNNs)。这些模型通常具有较轻的重量,所需的训练时间比传统的RNN-based方法要少。然而,这些模型在权重训练方面采用了概率统计的观点。理论上,网络权重可以从隐藏的、不可变的概率分布中抽样。我们介绍了贝叶斯语音合成器,以用于端到端自动语音 Recognition。贝叶斯语音合成器是一个贝叶斯Transformer网络,这些隐藏的 posteriors 通过异步变量推断和局部参数化技巧来学习,而无需循环重复。我们展示了如何在权重中引入差异会导致更快的训练时间和接近于最先进的性能在libriSpeech-960上。
URL
https://arxiv.org/abs/2301.11276
