Abstract
We present the first study on how Multimodal LLMs' (MLLMs) reasoning ability shall be elicited to evaluate the aesthetics of artworks. To facilitate this investigation, we construct MM-StyleBench, a novel high-quality dataset for benchmarking artistic stylization. We then develop a principled method for human preference modeling and perform a systematic correlation analysis between MLLMs' responses and human preference. Our experiments reveal an inherent hallucination issue of MLLMs in art evaluation, associated with response subjectivity. ArtCoT is proposed, demonstrating that art-specific task decomposition and the use of concrete language boost MLLMs' reasoning ability for aesthetics. Our findings offer valuable insights into MLLMs for art and can benefit a wide range of downstream applications, such as style transfer and artistic image generation. Code available at this https URL.
Abstract (translated)
我们介绍了首个关于如何激发多模态大型语言模型（MLLMs）的推理能力以评估艺术品美学的研究。为了支持这一研究，我们构建了MM-StyleBench，这是一个用于评测艺术风格化的高质量新数据集。随后，我们开发了一种基于原理的人类偏好建模方法，并进行了系统性的相关性分析，探讨MLLMs的回答与人类偏好的关联。我们的实验揭示了MLLMs在艺术品评价中存在一种固有的幻觉问题，这与回答的主观性有关。为了应对这一挑战，提出了ArtCoT框架，该框架证明了艺术特定任务分解以及使用具体语言可以增强MLLMs在美学方面的推理能力。我们的研究结果为MLLMs在艺术领域的应用提供了宝贵的见解，并且能够广泛应用于下游的各种应用场景，如风格转换和艺术品图像生成。代码可在[此链接](https://this-url.com)获取。
URL
https://arxiv.org/abs/2501.09012
