Abstract
Evolutionary Algorithms and Generative Deep Learning have been two of the most powerful tools for sound generation tasks. However, they have limitations: Evolutionary Algorithms require complicated designs, posing challenges in control and achieving realistic sound generation. Generative Deep Learning models often copy from the dataset and lack creativity. In this paper, we propose LVNS-RAVE, a method to combine Evolutionary Algorithms and Generative Deep Learning to produce realistic and novel sounds. We use the RAVE model as the sound generator and the VGGish model as a novelty evaluator in the Latent Vector Novelty Search (LVNS) algorithm. The reported experiments show that the method can successfully generate diversified, novel audio samples under different mutation setups using different pre-trained RAVE models. The characteristics of the generation process can be easily controlled with the mutation parameters. The proposed algorithm can be a creative tool for sound artists and musicians.
Abstract (translated)
进化算法和生成式深度学习是音效生成任务中最强大的工具之一。然而，它们也有局限性：进化算法需要复杂的架构，在控制和实现真实音效生成方面存在挑战。生成式深度学习模型通常从数据集中复制，缺乏创造性。在本文中，我们提出了LVNS-RAVE方法，将进化算法和生成式深度学习相结合，以产生真实和新的音效。我们使用RAVE模型作为音效生成器，VGGish模型作为新颖性评估器在Latent Vector Novelty Search（LVNS）算法中。报道的实验结果表明，该方法在不同突变设置下，可以成功生成具有多样性的新颖音频样本。通过控制突变参数，可以轻松控制生成过程的特点。所提出的算法可以为音乐家和音响师提供一种创新工具。
URL
https://arxiv.org/abs/2404.14063
