Abstract
Kolmogorov-Arnold Networks (KANs) have recently shown promise for solving partial differential equations (PDEs). Yet their original formulation is computationally and memory intensive, motivating the introduction of Chebyshev Type-I-based KANs (Chebyshev1KANs). Although Chebyshev1KANs have outperformed the vanilla KANs architecture, our rigorous theoretical analysis reveals that they still suffer from rank collapse, ultimately limiting their expressive capacity. To overcome these limitations, we enhance Chebyshev1KANs by integrating wavelet-activated MLPs with learnable parameters and an internal attention mechanism. We prove that this design preserves a full-rank Jacobian and is capable of approximating solutions to PDEs of arbitrary order. Furthermore, to alleviate the loss instability and imbalance introduced by the Chebyshev polynomial basis, we externally incorporate a Residual Gradient Attention (RGA) mechanism that dynamically re-weights individual loss terms according to their gradient norms and residual magnitudes. By jointly leveraging internal and external attention, we present AC-PKAN, a novel architecture that constitutes an enhancement to weakly supervised Physics-Informed Neural Networks (PINNs) and extends the expressive power of KANs. Experimental results from nine benchmark tasks across three domains show that AC-PKAN consistently outperforms or matches state-of-the-art models such as PINNsFormer, establishing it as a highly effective tool for solving complex real-world engineering problems in zero-data or data-sparse regimes. The code will be made publicly available upon acceptance.
Abstract (translated)
科洛莫戈罗夫-阿诺尔德网络（KANs）最近在解决偏微分方程（PDEs）方面显示出潜力。然而，它们的原始形式计算和内存消耗较大，因此引入了基于切比雪夫Type-I的KANs（Chebyshev1KANs）。虽然Chebyshev1KANs的表现优于传统的KAN架构，但我们的严格理论分析揭示出这些网络仍然存在秩塌陷的问题，这最终限制了它们的表达能力。为克服这一局限性，我们通过集成具有可学习参数和内部注意力机制的波浪激活多层感知机（MLP），对Chebyshev1KANs进行了增强。我们证明这种设计能够保持全秩雅可比矩阵，并且有能力近似任意阶数的偏微分方程的解。 为了缓解切比雪夫多项式基底引入的损失不稳定性和不平衡问题，我们在外部加入了残差梯度注意力（RGA）机制，该机制可以根据各个损失项的梯度范数和残差幅度动态重新加权。通过同时利用内部和外部注意机制，我们提出了AC-PKAN这一新架构，它构成了对弱监督物理信息神经网络（PINNs）的一种增强，并扩展了KANs的表达能力。 九个跨三个领域的基准任务实验结果表明，AC-PKAN在无数据或数据稀疏环境下始终优于或等同于现有最佳模型如PINNsFormer，在解决复杂的现实世界工程问题方面具有高度有效性。代码将在接受后公开发布。
URL
https://arxiv.org/abs/2505.08687
