Abstract
Machine learning techniques, such as Transformers and Long Short-Term Memory (LSTM) networks, play a crucial role in Symbolic Music Generation (SMG). Existing literature indicates a difference between LSTMs and Transformers regarding their ability to model local melodic continuity versus maintaining global structural coherence. However, their specific properties within the context of SMG have not been systematically studied. This paper addresses this gap by providing a fine-grained comparative analysis of LSTMs versus Transformers for SMG, examining local and global properties in detail using 17 musical quality metrics on the Deutschl dataset. We find that LSTM networks excel at capturing local patterns but fail to preserve long-range dependencies, while Transformers model global structure effectively but tend to produce irregular phrasing. Based on this analysis and leveraging their respective strengths, we propose a Hybrid architecture combining a Transformer Encoder with an LSTM Decoder and evaluate it against both baselines. We evaluated 1,000 generated melodies from each of the three architectures on the Deutschl dataset. The results show that the hybrid method achieves better local and global continuity and coherence compared to the baselines. Our work highlights the key characteristics of these models and demonstrates how their properties can be leveraged to design superior models. We also supported the experiments with ablation studies and human perceptual evaluations, which statistically support the findings and provide robust validation for this work.
Abstract (translated)
在符号音乐生成（SMG）领域，Transformer和长短期记忆（LSTM）网络等机器学习技术发挥着关键作用。现有文献表明，LSTM与Transformer在建模局部旋律连贯性与保持全局结构一致性方面存在差异，但二者在SMG场景下的具体特性尚未得到系统研究。本文通过细粒度对比分析填补这一空白，使用17项音乐质量指标在Deutschl数据集上详细检验了LSTM与Transformer的局部与全局特性。研究发现：LSTM网络擅长捕捉局部模式却难以保持长程依赖，而Transformer能有效建模全局结构但易产生不规则的乐句划分。基于此分析并融合二者优势，我们提出了结合Transformer编码器与LSTM解码器的混合架构，并与两种基线模型进行对比评估。在Deutschl数据集上对三种架构生成的1000段旋律进行评估后，结果显示混合方法在局部与全局的连贯性和一致性上均优于基线模型。本研究揭示了这些模型的关键特性，并展示了如何利用其属性设计更优模型。此外，通过消融实验和人类感知评估对实验进行了辅助验证，统计结果支持研究发现，为本工作提供了稳健的实证依据。
URL
https://arxiv.org/abs/2603.21282
