Abstract
Real-time conversational AI agents face challenges in performing Natural Language Understanding (NLU) in dynamic, outdoor environments like automated drive-thru systems. These settings require NLU models to handle background noise, diverse accents, and multi-intent queries while operating under strict latency and memory constraints on edge devices. Additionally, robustness to errors from upstream Automatic Speech Recognition (ASR) is crucial, as ASR outputs in these environments are often noisy. We introduce Babylon, a transformer-based architecture that tackles NLU as an intent translation task, converting natural language inputs into sequences of regular language units ('transcodes') that encode both intents and slot information. This formulation allows Babylon to manage multi-intent scenarios in a single dialogue turn. Furthermore, Babylon incorporates an LSTM-based token pooling mechanism to preprocess phoneme sequences, reducing input length and optimizing for low-latency, low-memory edge deployment. This also helps mitigate inaccuracies in ASR outputs, enhancing system robustness. While this work focuses on drive-thru ordering, Babylon's design extends to similar noise-prone scenarios, for e.g. ticketing kiosks. Our experiments show that Babylon achieves significantly better accuracy-latency-memory footprint trade-offs over typically employed NMT models like Flan-T5 and BART, demonstrating its effectiveness for real-time NLU in edge deployment settings.
Abstract (translated)
实时对话人工智能代理在动态的户外环境中执行自然语言理解（NLU）时面临挑战，比如自动化的得来速系统。这些设置要求NLU模型处理背景噪音、多样的口音以及多意图查询，同时还要受到严格的延迟和内存限制的影响。此外，在上游自动语音识别（ASR）错误下的鲁棒性也至关重要，因为在这种环境中的ASR输出通常包含噪声。我们引入了Babylon，这是一种基于变换器的架构，它将NLU视为一种意图翻译任务，可以将自然语言输入转换为一系列常规语言单元（“转码”），这些单元编码了意图和槽位信息。这种表述方式使得Babylon能够在单一对话轮次中管理多意图场景。此外，Babylon还采用了基于LSTM的标记池化机制来预处理音素序列，减少输入长度并优化低延迟、低内存边缘设备部署。这也有助于减轻ASR输出中的不准确之处，提高系统的鲁棒性。尽管这项工作专注于得来速点餐场景，但Babylon的设计同样适用于类似噪声干扰的场景，例如售票亭等。实验表明，与通常使用的NMT模型如Flan-T5和BART相比，Babylon在准确性、延迟性和内存占用方面实现了显著更好的权衡，证明了其在边缘部署环境下实时NLU的有效性。
URL
https://arxiv.org/abs/2411.15372
