Abstract
Musicians and nonmusicians alike use rhythmic sound gestures, such as tapping and beatboxing, to express drum patterns. While these gestures effectively communicate musical ideas, realizing these ideas as fully-produced drum recordings can be time-consuming, potentially disrupting many creative workflows. To bridge this gap, we present TRIA (The Rhythm In Anything), a masked transformer model for mapping rhythmic sound gestures to high-fidelity drum recordings. Given an audio prompt of the desired rhythmic pattern and a second prompt to represent drumkit timbre, TRIA produces audio of a drumkit playing the desired rhythm (with appropriate elaborations) in the desired timbre. Subjective and objective evaluations show that a TRIA model trained on less than 10 hours of publicly-available drum data can generate high-quality, faithful realizations of sound gestures across a wide range of timbres in a zero-shot manner.
Abstract (translated)
无论是音乐人还是非音乐人都会使用诸如敲击和节奏口技等有节奏的声音手势来表达鼓点模式。虽然这些手势能够有效地传达音乐构思，但将这些想法转化为完整的鼓录音可能需要耗费大量时间，从而扰乱许多创意工作流程。为了解决这个问题，我们提出了TRIA（The Rhythm In Anything），这是一种掩码变压器模型，用于将有节奏的声音手势映射到高保真的鼓录音上。给定一个所需节拍模式的音频提示和另一个表示鼓组音色的提示，TRIA可以生成所需的节奏在指定音色下演奏的鼓组音频（并进行适当的扩展）。主观和客观评估表明，使用不到10小时公开可用的鼓数据训练出的TRIA模型可以在零样本条件下，在广泛的音色范围内产生高质量且忠实的声音手势实现。
URL
https://arxiv.org/abs/2509.15625
