Abstract
We present a novel variational framework for performing inference in (neural) stochastic differential equations (SDEs) driven by Markov-approximate fractional Brownian motion (fBM). SDEs offer a versatile tool for modeling real-world continuous-time dynamic systems with inherent noise and randomness. Combining SDEs with the powerful inference capabilities of variational methods, enables the learning of representative function distributions through stochastic gradient descent. However, conventional SDEs typically assume the underlying noise to follow a Brownian motion (BM), which hinders their ability to capture long-term dependencies. In contrast, fractional Brownian motion (fBM) extends BM to encompass non-Markovian dynamics, but existing methods for inferring fBM parameters are either computationally demanding or statistically inefficient. In this paper, building upon the Markov approximation of fBM, we derive the evidence lower bound essential for efficient variational inference of posterior path measures, drawing from the well-established field of stochastic analysis. Additionally, we provide a closed-form expression to determine optimal approximation coefficients. Furthermore, we propose the use of neural networks to learn the drift, diffusion and control terms within our variational posterior, leading to the variational training of neural-SDEs. In this framework, we also optimize the Hurst index, governing the nature of our fractional noise. Beyond validation on synthetic data, we contribute a novel architecture for variational latent video prediction,-an approach that, to the best of our knowledge, enables the first variational neural-SDE application to video perception.
Abstract (translated)
我们提出了一个新的用于(神经)随机微分方程(SDE)驱动的马尔可夫近似分数布朗运动(fBM)进行推理的框架。SDE提供了一个灵活的工具来建模具有固有噪声和随机性的真实世界连续时间动态系统。将SDE与变分方法的力量相结合,可以通过随机梯度下降学习具有代表性的函数分布。然而,传统的SDE通常假设底层噪声遵循布朗运动(BM),这阻碍了它们捕捉长期依赖关系的能力。相比之下,分数布朗运动(fBM)扩展了BM,涵盖了非马尔可夫过程,但现有的对fBM参数进行推断的方法要么计算复杂度高,要么统计效率低。在本文中,我们在fBM的马尔可夫近似的基础上,推导出用于有效进行后验路径测量的证据下界,并从随机分析领域得到 established 的结论。此外,我们还提供了确定最优逼近系数的 closed-form 表达式。另外,我们提出使用神经网络来学习我们随后的变分后验中的漂移、扩散和控制项,从而实现神经-SDE的变分训练。在这个框架中,我们还优化了Hurst指数,控制了我们的分数噪声的本质。除了在合成数据上的验证外,我们还为变分随机场预测提供了一种新的架构,这种方法,据我们所知,是第一个变分神经-SDE应用视频感知。
URL
https://arxiv.org/abs/2310.12975
