Abstract
Takagi-Sugeno-Kang (TSK) fuzzy systems are very useful machine learning models for regression problems. However, to our knowledge, there has not existed an efficient and effective training algorithm that enables them to deal with big data. Inspired by the connections between TSK fuzzy systems and neural networks, we extend three powerful neural network optimization techniques, i.e., mini-batch gradient descent, regularization, and AdaBound, to TSK fuzzy systems, and also propose a novel DropRule technique specifically for training TSK fuzzy systems. Our final algorithm, mini-batch gradient descent with regularization, DropRule and AdaBound (MBGD-RDA), can achieve fast convergence in training TSK fuzzy systems, and also superior generalization performance in testing. It can be used for training TSK fuzzy systems on datasets of any size; however, it is particularly useful for big datasets, on which currently no other efficient training algorithms exist.
Abstract (translated)
Takagi-Sugeno-Kang（TSK）模糊系统是非常有用的回归问题机器学习模型。然而，据我们所知，目前还没有一种有效的训练算法来处理大数据。基于TSK模糊系统与神经网络之间的联系，将三种强大的神经网络优化技术，即小批量梯度下降、正则化和自适应，扩展到TSK模糊系统，并提出了一种新的专门用于训练TSK模糊系统的降速规则技术。最后提出的算法是正则化、下降规则和自适应约束的小批量梯度下降算法（MBGD-RDA），可以在训练TSK模糊系统时实现快速收敛，在测试中具有良好的泛化性能。它可以用于在任何大小的数据集上训练TSK模糊系统；但是，它特别适用于大数据集，目前没有其他有效的训练算法。
URL
https://arxiv.org/abs/1903.10951
