Abstract
Artificial General Intelligence (AGI) requires comprehensive understanding and generation capabilities for a variety of tasks spanning different modalities and functionalities. Integrative AI is one important direction to approach AGI, through combining multiple models to tackle complex multimodal tasks. However, there is a lack of a flexible and composable platform to facilitate efficient and effective model composition and coordination. In this paper, we propose the i-Code Studio, a configurable and composable framework for Integrative AI. The i-Code Studio orchestrates multiple pre-trained models in a finetuning-free fashion to conduct complex multimodal tasks. Instead of simple model composition, the i-Code Studio provides an integrative, flexible, and composable setting for developers to quickly and easily compose cutting-edge services and technologies tailored to their specific requirements. The i-Code Studio achieves impressive results on a variety of zero-shot multimodal tasks, such as video-to-text retrieval, speech-to-speech translation, and visual question answering. We also demonstrate how to quickly build a multimodal agent based on the i-Code Studio that can communicate and personalize for users.
Abstract (translated)
人工智能(AGI)需要对多种任务进行 comprehensive 理解和生成能力,涵盖了不同模式和功能的多方面任务。综合人工智能是接近AGI的一个重要方向,通过结合多个模型来解决复杂的多模式任务。然而,缺乏一个灵活、可组合的平台来促进高效、有效的模型组合和协调。在本文中,我们提出了 i-Code Studio,这是一个可配置和可组合的框架,用于综合人工智能。i-Code Studio 指挥多个预训练模型以无调整的方式执行复杂的多模式任务。 Instead of 简单的模型组合,i-Code Studio 提供了一个综合、灵活和可组合的环境,以开发人员快速、轻松地为特定需求构建最先进的服务和技术。i-Code Studio 在多种零次响应多模式任务中取得了令人印象深刻的结果,例如视频到文本检索、语音到语音翻译和视觉问答。我们还演示了如何使用 i-Code Studio 快速构建一个基于i-Code Studio的多模式代理,它能够对用户进行通信和个性化。
URL
https://arxiv.org/abs/2305.13738