Abstract
Breakthroughs in text-to-music generation models are transforming the creative landscape, equipping musicians with innovative tools for composition and experimentation like never before. However, controlling the generation process to achieve a specific desired outcome remains a significant challenge. Even a minor change in the text prompt, combined with the same random seed, can drastically alter the generated piece. In this paper, we explore the application of existing text-to-music diffusion models for instrument editing. Specifically, for an existing audio track, we aim to leverage a pretrained text-to-music diffusion model to edit the instrument while preserving the underlying content. Based on the insight that the model first focuses on the overall structure or content of the audio, then adds instrument information, and finally refines the quality, we show that selecting a well-chosen intermediate timestep, identified through an instrument classifier, yields a balance between preserving the original piece's content and achieving the desired timbre. Our method does not require additional training of the text-to-music diffusion model, nor does it compromise the generation process's speed.
Abstract (translated)
文本到音乐生成模型的突破正在重塑创意领域，为作曲家提供了前所未有的创新工具。然而，控制生成过程以实现特定目标仍然是一个重大挑战。即使是细微的文本提示变化，在相同的随机种子下也会大幅改变生成的作品。本文探讨了现有文本到音乐扩散模型在乐器编辑中的应用。具体而言，对于现有的音频轨道，我们旨在利用预训练的文本到音乐扩散模型来编辑乐器，同时保留底层内容。 根据模型首先关注音频的整体结构或内容，然后添加乐器信息，最后细化质量这一洞察，我们展示了通过使用一个乐器分类器选择适当的时间步长，可以实现保留原始作品内容与达到期望音色之间的平衡。我们的方法不需要额外训练文本到音乐的扩散模型，并且不会影响生成过程的速度。
URL
https://arxiv.org/abs/2506.15530
