Abstract
Accurately generating images across the Tree of Life is difficult: there are over 10M distinct species on Earth, many of which differ only by subtle visual traits. Despite the remarkable progress in text-to-image synthesis, existing models often fail to capture the fine-grained visual cues that define species identity, even when their outputs appear photo-realistic. To this end, we propose TaxaAdapter, a simple and lightweight approach that incorporates Vision Taxonomy Models (VTMs) such as BioCLIP to guide fine-grained species generation. Our method injects VTM embeddings into a frozen text-to-image diffusion model, improving species-level fidelity while preserving flexible text control over attributes such as pose, style, and background. Extensive experiments demonstrate that TaxaAdapter consistently improves morphology fidelity and species-identity accuracy over strong baselines, with a cleaner architecture and training recipe. To better evaluate these improvements, we also introduce a multimodal Large Language Model-based metric that summarizes trait-level descriptions from generated and real images, providing a more interpretable measure of morphological consistency. Beyond this, we observe that TaxaAdapter exhibits strong generalization capabilities, enabling species synthesis in challenging regimes such as few-shot species with only a handful of training images and even species unseen during training. Overall, our results highlight that VTMs are a key ingredient for scalable, fine-grained species generation.
Abstract (translated)
在生命之树上精确生成图像具有挑战性：地球上有超过1000万个不同物种，其中许多仅凭细微的视觉特征区分。尽管文本到图像合成技术已取得显著进展，但现有模型往往难以捕捉定义物种身份的细粒度视觉线索，即便其输出结果看似逼真。为此，我们提出了TaxaAdapter——一种简单轻量的方法，通过引入BioCLIP等视觉分类模型（VTMs）来指导细粒度物种生成。我们的方法将VTM嵌入注入冻结的文本到图像扩散模型，在保持对姿态、风格和背景等属性的灵活文本控制的同时，提升了物种级别的保真度。大量实验表明，TaxaAdapter在形态保真度和物种身份准确性上持续优于强基线模型，并具备更简洁的架构与训练方案。为更好地评估这些改进，我们还引入了一种基于多模态大语言模型的评估指标，该指标能汇总生成图像与真实图像的性状级描述，提供更具可解释性的形态一致性度量。此外，我们观察到TaxaAdapter展现出强大的泛化能力，可在小样本物种（仅需少量训练图像）乃至训练中未见的物种等具有挑战性的场景中实现物种合成。总体而言，我们的研究结果表明，视觉分类模型是实现可扩展、细粒度物种生成的关键要素。
URL
https://arxiv.org/abs/2603.26128
