Abstract
Existing audio language models typically rely on task-specific fine-tuning to accomplish particular audio tasks. In contrast, humans are able to generalize to new audio tasks with only a few examples or simple instructions. GPT-3 has shown that scaling next-token prediction pretraining enables strong generalization capabilities in text, and we believe this paradigm is equally applicable to the audio domain. By scaling MiMo-Audio's pretraining data to over one hundred million of hours, we observe the emergence of few-shot learning capabilities across a diverse set of audio tasks. We develop a systematic evaluation of these capabilities and find that MiMo-Audio-7B-Base achieves SOTA performance on both speech intelligence and audio understanding benchmarks among open-source models. Beyond standard metrics, MiMo-Audio-7B-Base generalizes to tasks absent from its training data, such as voice conversion, style transfer, and speech editing. MiMo-Audio-7B-Base also demonstrates powerful speech continuation capabilities, capable of generating highly realistic talk shows, recitations, livestreaming and debates. At the post-training stage, we curate a diverse instruction-tuning corpus and introduce thinking mechanisms into both audio understanding and generation. MiMo-Audio-7B-Instruct achieves open-source SOTA on audio understanding benchmarks (MMSU, MMAU, MMAR, MMAU-Pro), spoken dialogue benchmarks (Big Bench Audio, MultiChallenge Audio) and instruct-TTS evaluations, approaching or surpassing closed-source models. Model checkpoints and full evaluation suite are available at this https URL.
Abstract (translated)
现有的音频语言模型通常依赖于特定任务的微调来完成具体的音频任务。相比之下，人类能够在仅有少量示例或简单指令的情况下掌握新的音频任务。GPT-3 显示出扩大下一个标记预测预训练可以增强文本中的强大泛化能力，我们认为这种范式同样适用于音频领域。通过将 MiMo-Audio 的预训练数据扩展到超过一亿小时的规模，我们观察到了跨多种音频任务出现少量样本学习能力的现象。我们系统地评估了这些能力，并发现 MiMo-Audio-7B-Base 在开源模型中于语音智能和音频理解基准测试上达到了最先进的性能（SOTA）。除了标准度量之外，MiMo-Audio-7B-Base 还可以泛化到其训练数据中不存在的任务，例如声音转换、风格转移以及语音编辑。此外，MiMo-Audio-7B-Base 展现了强大的语音持续生成能力，能够产生高度逼真的脱口秀节目、朗诵、直播和辩论内容。 在预训练之后的阶段，我们整理了一个多样化的指令微调语料库，并将思考机制引入到音频理解和生成中。MiMo-Audio-7B-Instruct 在音频理解基准测试（MMSU, MMAU, MMAR, MMAU-Pro）、语音对话基准测试（Big Bench Audio, MultiChallenge Audio）以及指令TTS评估方面达到了开源模型的最高性能，甚至接近或超过了封闭源代码模型的表现。该模型的检查点和完整评测套件可以在提供的链接中找到：[此处插入链接]。
URL
https://arxiv.org/abs/2512.23808
