Abstract
Kolmogorov-Arnold Networks (KANs) have garnered attention for replacing fixed activation functions with learnable univariate functions, but they exhibit practical limitations, including high computational costs and performance deficits in general classification tasks. In this paper, we propose the Modulation Joint KAN (MJKAN), a novel neural network layer designed to overcome these challenges. MJKAN integrates a FiLM (Feature-wise Linear Modulation)-like mechanism with Radial Basis Function (RBF) activations, creating a hybrid architecture that combines the non-linear expressive power of KANs with the efficiency of Multilayer Perceptrons (MLPs). We empirically validated MJKAN's performance across a diverse set of benchmarks, including function regression, image classification (MNIST, CIFAR-10/100), and natural language processing (AG News, SMS Spam). The results demonstrate that MJKAN achieves superior approximation capabilities in function regression tasks, significantly outperforming MLPs, with performance improving as the number of basis functions increases. Conversely, in image and text classification, its performance was competitive with MLPs but revealed a critical dependency on the number of basis functions. We found that a smaller basis size was crucial for better generalization, highlighting that the model's capacity must be carefully tuned to the complexity of the data to prevent overfitting. In conclusion, MJKAN offers a flexible architecture that inherits the theoretical advantages of KANs while improving computational efficiency and practical viability.
Abstract (translated)
Kolmogorov-Arnold Networks（KAN）因其用可学习的一元函数替代固定激活函数而受到关注，但它们在实际应用中存在一些限制，包括计算成本高和一般分类任务中的性能不足。本文提出了一种新的神经网络层——调制联合KAN（MJKAN），旨在克服这些挑战。MJKAN结合了类似于FiLM（按特征线性调制）的机制和径向基函数（RBF）激活，构建了一个混合架构，它融合了KAN的非线性表达能力和多层感知机（MLP）的效率。 我们通过一系列基准测试对MJKAN的表现进行了实证验证，包括函数回归、图像分类（MNIST, CIFAR-10/100）和自然语言处理（AG News, SMS Spam）。结果表明，在函数回归任务中，MJKAN展示了优越的逼近能力，并显著超越了MLP，且随着基函数数量的增加性能有所提升。然而，在图像和文本分类中，其表现与MLP相当，但显现出对基函数数量的关键依赖性。我们发现较小的基础大小对于更好的泛化至关重要，这表明模型容量需要根据数据复杂度进行仔细调整以防止过拟合。 综上所述，MJKAN提供了一种灵活的架构，继承了KAN理论上的优势，同时在计算效率和实际可行性方面进行了改进。
URL
https://arxiv.org/abs/2507.04690
