Abstract
Neural networks are a powerful tool for learning patterns from data. However, they do not respect known scientific laws, nor can they reveal novel scientific insights due to their black-box nature. In contrast, scientific reasoning distills biological or physical principles from observations and controlled experiments, and quantitatively interprets them with process-based models made of mathematical equations. Yet, process-based models rely on numerous free parameters that must be set in an ad-hoc manner, and thus often fit observations poorly in cross-scale predictions. While prior work has embedded process-based models in conventional neural networks, discovering interpretable relationships between parameters in process-based models and input features is still a grand challenge for scientific discovery. We thus propose Scientifically-Interpretable Reasoning Network (ScIReN), a fully-transparent framework that combines interpretable neural and process-based reasoning. An interpretable encoder predicts scientifically-meaningful latent parameters, which are then passed through a differentiable process-based decoder to predict labeled output variables. ScIReN also uses a novel hard-sigmoid constraint layer to restrict latent parameters to meaningful ranges defined by scientific prior knowledge, further enhancing its interpretability. While the embedded process-based model enforces established scientific knowledge, the encoder reveals new scientific mechanisms and relationships hidden in conventional black-box models. We apply ScIReN on two tasks: simulating the flow of organic carbon through soils, and modeling ecosystem respiration from plants. In both tasks, ScIReN outperforms black-box networks in predictive accuracy while providing substantial scientific interpretability -- it can infer latent scientific mechanisms and their relationships with input features.
Abstract (translated)
神经网络是学习数据模式的强大工具。然而，它们不遵守已知的科学定律，并且由于其黑盒性质也无法揭示新的科学见解。相比之下，科学研究从观察和控制实验中提炼出生物学或物理学原理，并用基于过程的数学方程模型对其进行定量解释。尽管如此，基于过程的模型依赖于许多需要随意设定的自由参数，在跨尺度预测中往往难以很好地拟合观测数据。先前的工作已经将这些基于过程的模型嵌入到传统的神经网络中，但发现这些模型中的参数与输入特征之间的可解释关系仍然是科学发现的一大挑战。 因此，我们提出了一个完全透明的框架——Scientifically-Interpretable Reasoning Network (ScIReN)，该框架结合了可解释的神经推理和基于过程的推理。在这个框架中，一个可解释的编码器预测出具有科学意义的潜在参数，然后这些参数通过一个微分方程构成的过程解码器传递以预测带有标签的目标变量。此外，ScIReN 使用了一种新颖的硬 sigmoid 约束层来限制潜在参数到由先验科学知识定义的意义范围内，进一步增强了其可解释性。 嵌入的基于过程模型施加了已确立的科学知识，而编码器揭示了隐藏在传统黑盒模型中的新的科学机制和关系。我们在两个任务中应用了 ScIReN：模拟有机碳通过土壤的流动以及从植物中建模生态系统呼吸作用。在这两项任务中，ScIReN 不仅比传统的黑箱网络表现出更高的预测准确性，而且还提供了大量的科学可解释性——它可以推断出潜在的科学机制及其与输入特征的关系。
URL
https://arxiv.org/abs/2506.14054
