Abstract
Multi-modal large language models that have image output are emerging. Many image generation benchmarks focus on aesthetics instead of fine-grained generation capabilities. In PixelArena, we propose using semantic segmentation tasks to objectively examine their fine-grained generative intelligence with pixel precision. We find the latest Gemini 3 Pro Image has emergent image generation capabilities that generate semantic masks with high fidelity under zero-shot settings, showcasing visual intelligence unseen before and true generalization in new image generation tasks. We further investigate its results, compare them qualitatively and quantitatively with those of other models, and present failure cases. The findings not only signal exciting progress in the field but also provide insights into future research related to multimodality, reasoning, interpretability and benchmarking.
Abstract (translated)
多模态大型语言模型（能够生成图像）正在兴起。许多图像生成基准测试主要关注美学效果，而忽略了对精细生成能力的考察。在PixelArena中，我们提出使用语义分割任务来客观地评估这些模型的细粒度生成智能，并以像素级别的精度进行衡量。我们发现最新的Gemini 3 Pro Image模型具备了零样本设置下的图像生成能力，在这种情况下，它能够产生高保真的语义掩码，展示出前所未有的视觉智能和在新图像生成任务中的真实泛化能力。 进一步地，我们对其结果进行了深入研究，并从定性和定量两个角度与其他模型的结果进行比较。此外，还展示了该模型的失败案例。这些发现不仅标志着领域内的令人兴奋的进步，而且为与多模态、推理、可解释性以及基准测试相关的未来研究提供了宝贵的见解。
URL
https://arxiv.org/abs/2512.16303
