Abstract
We propose an efficient hybrid least squares/gradient descent method to accelerate DeepONet training. Since the output of DeepONet can be viewed as linear with respect to the last layer parameters of the branch network, these parameters can be optimized using a least squares (LS) solve, and the remaining hidden layer parameters are updated by means of gradient descent form. However, building the LS system for all possible combinations of branch and trunk inputs yields a prohibitively large linear problem that is infeasible to solve directly. To address this issue, our method decomposes the large LS system into two smaller, more manageable subproblems $\unicode{x2014}$ one for the branch network and one for the trunk network $\unicode{x2014}$ and solves them separately. This method is generalized to a broader type of $L^2$ loss with a regularization term for the last layer parameters, including the case of unsupervised learning with physics-informed loss.
Abstract (translated)
我们提出了一种高效的混合最小二乘法/梯度下降方法，以加速DeepONet的训练过程。由于DeepONet的输出可以被视为与分支网络最后一层参数呈线性关系，因此这些参数可以通过最小二乘（LS）求解进行优化，而其余隐藏层参数则通过梯度下降方式进行更新。然而，为所有可能的分支和主干输入组合构建LS系统会产生一个规模过大且难以直接求解的线性问题。为了应对这一挑战，我们的方法将大型LS系统分解为两个较小、更易管理的子问题——一个是针对分支网络的问题，另一个是针对主干网络的问题，并分别解决这两个子问题。该方法还被推广到一种更为广泛的$L^2$损失类型，包括最后一层参数的正则化项，在无监督学习中涉及物理信息损失的情况也适用。
URL
https://arxiv.org/abs/2508.15394
