Abstract
We introduce Fish Audio S2, an open-sourced text-to-speech system featuring multi-speaker, multi-turn generation, and, most importantly, instruction-following control via natural-language descriptions. To scale training, we develop a multi-stage training recipe together with a staged data pipeline covering video captioning and speech captioning, voice-quality assessment, and reward modeling. To push the frontier of open-source TTS, we release our model weights, fine-tuning code, and an SGLang-based inference engine. The inference engine is production-ready for streaming, achieving an RTF of 0.195 and a time-to-first-audio below 100 this http URL code and weights are available on GitHub (this https URL) and Hugging Face (this https URL). We highly encourage readers to visit this https URL to try custom voices.
Abstract (translated)
我们推出了Fish Audio S2，这是一个开源的文本到语音系统，具有多说话人、多轮对话生成功能，并且最重要的是支持通过自然语言描述进行指令控制。为了扩大训练规模，我们开发了一套多阶段训练流程以及包括视频字幕、语音质量评估和奖励建模在内的分阶段数据管道。为了推动开源TTS的前沿发展，我们发布了我们的模型权重、微调代码及基于SGLang的推理引擎。该推理引擎已经为流媒体做好了生产准备，实现了0.195的实时因子（RTF）以及不到100毫秒的第一音频响应时间。相关代码和权重可以在GitHub (此链接) 和Hugging Face (此链接) 上获取。我们强烈建议读者访问此链接以尝试自定义语音。 请注意，原文中的“this http URL”是用于占位符的文本，实际使用时应替换为具体的URL链接地址。
URL
https://arxiv.org/abs/2603.08823
