Abstract
We propose a novel machine learning method based on differentiable vortex particles to infer and predict fluid dynamics from a single video. The key design of our system is a particle-based latent space to encapsulate the hidden, Lagrangian vortical evolution underpinning the observable, Eulerian flow phenomena. We devise a novel differentiable vortex particle system in conjunction with their learnable, vortex-to-velocity dynamics mapping to effectively capture and represent the complex flow features in a reduced space. We further design an end-to-end training pipeline to directly learn and synthesize simulators from data, that can reliably deliver future video rollouts based on limited observation. The value of our method is twofold: first, our learned simulator enables the inference of hidden physics quantities (e.g. velocity field) purely from visual observation, to be used for motion analysis; secondly, it also supports future prediction, constructing the input video's sequel along with its future dynamics evolution. We demonstrate our method's efficacy by comparing quantitatively and qualitatively with a range of existing methods on both synthetic and real-world videos, displaying improved data correspondence, visual plausibility, and physical integrity.
Abstract (translated)
我们提出了一种基于可区分涡旋颗粒的新机器学习方法,以从单个视频中推断和预测流体动力学。我们的系统的关键设计是基于颗粒的潜伏空间,以包裹隐藏、拉格朗日流变现象的基础。我们设计了一种新的可区分涡旋颗粒系统,并与他们的可学习涡流-速度动态映射一起开发,以有效地捕捉和代表在减少的空间中的复杂流变特征。我们还设计了一个完整的端到端训练管道,以直接学习并合成模拟器,可靠地基于有限的观察提供未来的视频生成。我们的方法的价值有两个:首先,我们的学习模拟器使纯粹从视觉观察推断隐藏的物理量(如速度场)成为可能,可用于运动分析;其次,它还支持未来预测,与输入视频的未来动力学演化一起构建它的后续。我们通过比较数量和质量上与各种现有方法在合成和实际视频上的表现,展示了我们方法的有效性,并展示了改进的数据对应、视觉可信度和物理完整性。
URL
https://arxiv.org/abs/2301.11494
