Abstract
Per-garment virtual try-on methods collect garment-specific datasets and train networks tailored to each garment to achieve superior results. However, these approaches often struggle with loose-fitting garments due to two key limitations: (1) They rely on human body semantic maps to align garments with the body, but these maps become unreliable when body contours are obscured by loose-fitting garments, resulting in degraded outcomes; (2) They train garment synthesis networks on a per-frame basis without utilizing temporal information, leading to noticeable jittering artifacts. To address these challenges, we propose a two-stage approach for robust semantic map estimation. First, we extract a garment-invariant representation from the raw input image. This representation is then passed through an auxiliary network to estimate the semantic map. This enhances the robustness of semantic map estimation under loose-fitting garments during garment-specific dataset generation. Furthermore, we introduce a recurrent garment synthesis framework that incorporates temporal dependencies to improve frame-to-frame coherence while maintaining real-time performance. We conducted qualitative and quantitative evaluations to demonstrate that our method outperforms existing approaches in both image quality and temporal coherence. Ablation studies further validate the effectiveness of the garment-invariant representation and the recurrent synthesis framework.
Abstract (translated)
基于每件衣物的虚拟试穿方法会收集特定于某件衣物的数据集，并训练针对每种衣物定制的网络以获得更佳效果。然而，这些方法在处理宽松衣物时往往面临两个关键限制：（1）它们依赖于人体语义地图来将衣物与身体对齐，但当人体轮廓被宽松衣物遮挡时，这种地图变得不可靠，导致结果质量下降；（2）它们基于每帧训练服装合成网络，而不利用时间信息，这会导致明显的抖动伪影。为了解决这些挑战，我们提出了一种两阶段的方法来估计稳健的语义图。 首先，从原始输入图像中提取一种与衣物无关的表示。这种表示然后通过一个辅助网络来估算语义地图。这种方法增强了在生成特定于某件服装的数据集时，在宽松服饰下语义地图估算的鲁棒性。此外，我们引入了一种递归服装合成框架，该框架结合了时间依赖关系以改善帧与帧之间的连贯性，并保持实时性能。 我们进行了定性和定量评估来证明我们的方法在图像质量和时间一致性方面优于现有的方法。消融研究进一步验证了衣物无关表示和递归合成框架的有效性。
URL
https://arxiv.org/abs/2506.12348
