Abstract
Virtual Try-ON (VTON) is a practical and widely-applied task, for which most of existing works focus on clothes. This paper presents OmniTry, a unified framework that extends VTON beyond garment to encompass any wearable objects, e.g., jewelries and accessories, with mask-free setting for more practical application. When extending to various types of objects, data curation is challenging for obtaining paired images, i.e., the object image and the corresponding try-on result. To tackle this problem, we propose a two-staged pipeline: For the first stage, we leverage large-scale unpaired images, i.e., portraits with any wearable items, to train the model for mask-free localization. Specifically, we repurpose the inpainting model to automatically draw objects in suitable positions given an empty mask. For the second stage, the model is further fine-tuned with paired images to transfer the consistency of object appearance. We observed that the model after the first stage shows quick convergence even with few paired samples. OmniTry is evaluated on a comprehensive benchmark consisting of 12 common classes of wearable objects, with both in-shop and in-the-wild images. Experimental results suggest that OmniTry shows better performance on both object localization and ID-preservation compared with existing methods. The code, model weights, and evaluation benchmark of OmniTry will be made publicly available at this https URL.
Abstract (translated)
虚拟试穿（Virtual Try-ON，VTON）是一项实用且广泛应用的任务，现有大多数研究主要集中在衣物上。本文介绍了OmniTry框架，这是一个统一的框架，将VTON从服饰扩展到任何可穿戴物品，如珠宝和配饰，并采用无掩码设置以实现更广泛的应用场景。在向各种类型的物体扩展时，数据整理变得具有挑战性，因为难以获得成对图像（即物品图像及其对应的试穿结果）。为了解决这个问题，我们提出了一个两阶段的流水线：第一阶段利用大规模未配对的数据集进行模型训练，这些数据包括任何可穿戴项目的肖像图片。具体来说，我们将修复模型重新用于在给定空掩码的情况下自动绘制合适位置的对象。第二阶段中，该模型进一步使用成对图像进行微调以转移物体外观的一致性。我们观察到，在第一阶段之后，即使只用少量的配对样本也能快速收敛。 OmniTry在一个全面的基准测试上进行了评估，此基准包含12种常见的可穿戴物品类别，并涵盖商店内和现实生活中的图片。实验结果表明，与现有的方法相比，OmniTry在物体定位和身份保存方面表现更好。OmniTry的代码、模型权重以及评估基准将在[此处](https://this-url.com)公开发布。
URL
https://arxiv.org/abs/2508.13632
