Abstract
Lip segmentation plays a crucial role in various domains, such as lip synchronization, lipreading, and diagnostics. However, the effectiveness of supervised lip segmentation is constrained by the availability of lip contour in the training phase. A further challenge with lip segmentation is its reliance on image quality , lighting, and skin tone, leading to inaccuracies in the detected boundaries. To address these challenges, we propose a sequential lip segmentation method that integrates attention UNet and multidimensional input. We unravel the micro-patterns in facial images using local binary patterns to build multidimensional inputs. Subsequently, the multidimensional inputs are fed into sequential attention UNets, where the lip contour is reconstructed. We introduce a mask generation method that uses a few anatomical landmarks and estimates the complete lip contour to improve segmentation accuracy. This mask has been utilized in the training phase for lip segmentation. To evaluate the proposed method, we use facial images to segment the upper lips and subsequently assess lip-related facial anomalies in subjects with fetal alcohol syndrome (FAS). Using the proposed lip segmentation method, we achieved a mean dice score of 84.75%, and a mean pixel accuracy of 99.77% in upper lip segmentation. To further evaluate the method, we implemented classifiers to identify those with FAS. Using a generative adversarial network (GAN), we reached an accuracy of 98.55% in identifying FAS in one of the study populations. This method could be used to improve lip segmentation accuracy, especially around Cupid's bow, and shed light on distinct lip-related characteristics of FAS.
Abstract (translated)
嘴唇分割在唇同步、唇读和诊断等多个领域中扮演着重要角色。然而，监督式嘴唇分割的有效性受限于训练阶段中可用的嘴唇轮廓。此外，嘴唇分割还依赖于图像质量、光照条件以及肤色等因素，导致检测边界不准确的问题。为了解决这些挑战，我们提出了一种结合注意力UNet和多维输入的序列化嘴唇分割方法。 我们的方法首先利用局部二值模式技术揭示面部图像中的微小图案特征，并以此构建多维输入数据集。然后将这些多维输入数据送入序列化的注意力UNet中，重建嘴唇轮廓。我们还引入了一种生成掩码的方法，该方法使用少量解剖学标志点估计完整的嘴唇轮廓，以提高分割精度。此掩码在训练阶段被用于优化嘴唇的精确分割。 为了评估所提出的分割方法的有效性，我们在一系列面部图像上对其进行了测试，特别关注对胎儿酒精综合症（FAS）患者进行上唇分割，并进一步分析了与FAS相关的面部异常特征。通过使用我们的方法，在上唇分割任务中我们取得了平均Dice分数为84.75%，以及平均像素准确率为99.77%的成绩。 为了更全面地评估该方法，我们在一个包含FAS患者的数据集上实现了分类器，使用生成对抗网络（GAN）对数据进行了训练。结果表明，在区分FAS患者和对照组时，我们的模型达到了98.55%的准确率。 这一方法不仅能够提升嘴唇分割特别是围绕人中部分区域的精度，而且还能帮助识别与胎儿酒精综合症相关的独特唇部特征，从而为相关研究提供新的视角和技术手段。
URL
https://arxiv.org/abs/2505.05531
