Abstract
Self-Supervised Learning (SSL) frameworks became the standard for learning robust class representations by benefiting from large unlabeled datasets. For Speaker Verification (SV), most SSL systems rely on contrastive-based loss functions. We explore different ways to improve the performance of these techniques by revisiting the NT-Xent contrastive loss. Our main contribution is the definition of the NT-Xent-AM loss and the study of the importance of Additive Margin (AM) in SimCLR and MoCo SSL methods to further separate positive from negative pairs. Despite class collisions, we show that AM enhances the compactness of same-speaker embeddings and reduces the number of false negatives and false positives on SV. Additionally, we demonstrate the effectiveness of the symmetric contrastive loss, which provides more supervision for the SSL task. Implementing these two modifications to SimCLR improves performance and results in 7.85% EER on VoxCeleb1-O, outperforming other equivalent methods.
Abstract (translated)
自监督学习（SSL）框架通过利用大量未标注数据的优势，成为学习稳健类别表示的标准。对于说话人验证（SV），大多数SSL系统依赖于对比式损失函数。我们探讨了通过回顾NT-Xent对比损失来提高这些技术性能的不同方法。我们的主要贡献是定义NT-Xent-AM损失，并研究了在SimCLR和MoCo SSL方法中添加Additive Margin（AM）对进一步区分正负对的重要性。尽管存在类别碰撞，我们证明了AM能增强相同说话者嵌入的紧凑性，并减少SV上的假负和假正数量。此外，我们还证明了对称对比损失的有效性，为SSL任务提供了更多的监督。对SimCLR进行这两种修改后的性能优于其他等效方法，提高了7.85%的均方误差（EER）在VoxCeleb1-O数据集上，超过了其他方法。
URL
https://arxiv.org/abs/2404.14913
