Abstract
In video game design, audio (both environmental background music and object sound effects) play a critical role. Sounds are typically pre-created assets designed for specific locations or objects in a game. However, user-generated content is becoming increasingly popular in modern games (e.g. building custom environments or crafting unique objects). Since the possibilities are virtually limitless, it is impossible for game creators to pre-create audio for user-generated content. We explore the use of generative artificial intelligence to create music and sound effects on-the-fly based on user-generated content. We investigate two avenues for audio generation: 1) text-to-audio: using a text description of user-generated content as input to the audio generator, and 2) image-to-audio: using a rendering of the created environment or object as input to an image-to-text generator, then piping the resulting text description into the audio generator. In this paper we discuss ethical implications of using generative artificial intelligence for user-generated content and highlight two prototype games where audio is generated for user-created environments and objects.
Abstract (translated)
在游戏设计中，音频（包括环境背景音乐和物体声音效果）扮演着关键角色。声音通常是针对游戏中的特定位置或物体预先创建的资产。然而，随着现代游戏用户生成内容的越来越受欢迎（例如，创建自定义环境或制作独特物品），游戏创作者无法为用户生成内容创建预先音频。我们探讨了使用生成人工智能在用户生成内容上实时创建音乐和声音效果的方法。我们调查了两种音频生成途径：1）文本转音频：使用用户生成内容的文本描述作为输入，音频生成器将生成相应的音频；2）图像转音频：使用创建的环境或对象的渲染作为输入，然后将生成的文本描述输入到音频生成器中。在本文中，我们讨论了使用生成人工智能为用户生成内容所带来的道德后果，并重点推荐了两个为用户生成内容生成音频的游戏原型。
URL
https://arxiv.org/abs/2404.17018
