Abstract
This paper introduces LLM2Fx-Tools, a multimodal tool-calling framework that generates executable sequences of audio effects (Fx-chain) for music post-production. LLM2Fx-Tools uses a large language model (LLM) to understand audio inputs, select audio effects types, determine their order, and estimate parameters, guided by chain-of-thought (CoT) planning. We also present LP-Fx, a new instruction-following dataset with structured CoT annotations and tool calls for audio effects modules. Experiments show that LLM2Fx-Tools can infer an Fx-chain and its parameters from pairs of unprocessed and processed audio, enabled by autoregressive sequence modeling, tool calling, and CoT reasoning. We further validate the system in a style transfer setting, where audio effects information is transferred from a reference source and applied to new content. Finally, LLM-as-a-judge evaluation demonstrates that our approach generates appropriate CoT reasoning and responses for music production queries. To our knowledge, this is the first work to apply LLM-based tool calling to audio effects modules, enabling interpretable and controllable music production.
Abstract (translated)
本文介绍了LLM2Fx-Tools，这是一个多模态工具调用框架，用于生成音乐后期制作中的可执行音频效果序列（Fx链）。LLM2Fx-Tools 使用大型语言模型 (LLM) 来理解音频输入、选择音频效果类型、确定其顺序并估算参数，通过链条思维 (CoT) 规划来进行引导。我们还介绍了一个新的指令跟随数据集LP-Fx，该数据集带有结构化的CoT注释和音频效果模块的工具调用。实验表明，LLM2Fx-Tools 可以从未经处理和已处理的音频对中推断出 Fx链及其参数，这得益于自回归序列建模、工具调用和 CoT 推理。在风格转移设置下，我们进一步验证了该系统能够将参考源中的音频效果信息转移到新内容上。最后，作为裁判的LLM评估表明，我们的方法可以为音乐制作查询生成合适的CoT推理和响应。据我们所知，这是首次将基于LLM的工具调用应用于音频效果模块的工作，从而使音乐制作变得可解释且可控。
URL
https://arxiv.org/abs/2512.01559
