Abstract
Nonlinearities are crucial for capturing complex input-output relationships especially in deep neural networks. However, nonlinear functions often incur various hardware and compute overheads. Meanwhile, stochastic computing (SC) has emerged as a promising approach to tackle this challenge by trading output precision for hardware simplicity. To this end, this paper proposes a first-of-its-kind stochastic multivariate universal-radix finite-state machine (SMURF) that harnesses SC for hardware-simplistic multivariate nonlinear function generation at high accuracy. We present the finite-state machine (FSM) architecture for SMURF, as well as analytical derivations of sampling gate coefficients for accurately approximating generic nonlinear functions. Experiments demonstrate the superiority of SMURF, requiring only 16.07% area and 14.45% power consumption of Taylor-series approximation, and merely 2.22% area of look-up table (LUT) schemes.
Abstract (translated)
非线性在深度神经网络中捕捉复杂的输入-输出关系非常重要。然而，非线性函数通常会引入各种硬件和计算开销。与此同时，随机计算（SC）作为一种解决这个挑战的有效方法，通过将输出精度换取硬件的简单性，浮现出来。为此，本文提出了一个独一无二的随机多维统一 radical-30 有限状态机（SMURF），利用 SC 在高精度的硬件简约多维非线性函数生成中。我们呈现了 SMURF 的有限状态机（FSM）架构，以及用于准确近似的通用非线性函数的采样门系数分析。实验证明，SMURF 的优越性，只需要 16.07% 的面积和 14.45% 的功耗，以及仅仅 2.22% 的查找表（LUT）方案的面积。
URL
https://arxiv.org/abs/2405.02356
