Abstract
Current deep learning primitives dealing with temporal dynamics suffer from a fundamental dichotomy: they are either discrete and unstable (LSTMs) \citep{pascanu_difficulty_2013}, leading to exploding or vanishing gradients; or they are continuous and dissipative (Neural ODEs) \citep{dupont_augmented_2019}, which destroy information over time to ensure stability. We propose the \textbf{Causal Hamiltonian Learning Unit} (pronounced: \textit{clue}), a novel Physics-grounded computational learning primitive. By enforcing a Relativistic Hamiltonian structure and utilizing symplectic integration, a CHLU strictly conserves phase-space volume, as an attempt to solve the memory-stability trade-off. We show that the CHLU is designed for infinite-horizon stability, as well as controllable noise filtering. We then demonstrate a CHLU's generative ability using the MNIST dataset as a proof-of-principle.
Abstract (translated)
当前用于处理时间动态的深度学习基本构建块面临着一个根本性的二难困境：它们要么是离散且不稳定的（如LSTM）\citep{pascanu_difficulty_2013}，导致梯度爆炸或消失；要么是连续但耗散的（如神经ODE）\citep{dupont_augmented_2019}，这会导致信息随时间流逝而被破坏以确保稳定性。我们提出了**因果哈密顿学习单元**（发音为：clue），这是一个新型基于物理原理的计算学习基本构建块。通过强制执行相对论哈密顿结构并利用辛积分法，CHLU 严格地保持相空间体积不变，试图解决内存和稳定性的权衡问题。我们展示了 CHLU 设计用于无限时间范围内的稳定性以及可控噪声过滤能力。然后，我们将使用 MNIST 数据集展示 CHLU 的生成能力作为证明概念的实例。
URL
https://arxiv.org/abs/2603.01768
