Abstract
In order to gain fresh insights about the information processing characteristics of different audio classification models, we propose transferability analysis. Given a minimal, sufficient signal for a classification on a model $f$, transferability analysis asks whether other models accept this minimal signal as having the same classification as it did on $f$. We define what it means for a sufficient signal to be transferable and perform a large study over $3$ different classification tasks: music genre, emotion recognition and deepfake detection. We find that transferability rates vary depending on the task, with sufficient signals for music genre being transferable $\approx26\%$ of the time. The other tasks reveal much higher variance in transferability and reveal that some models, in particular on deepfake detection, have different transferability behavior. We call these models `flat-earther' models. We investigate deepfake audio in more depth, and show that transferability analysis also allows to us to discover information theoretic differences between the models which are not captured by the more familiar metrics of accuracy and precision.
Abstract (translated)
为了深入了解不同音频分类模型的信息处理特性，我们提出了可迁移性分析。该方法针对某一模型$f$所需的最小充分分类信号，探究其他模型是否也能接受该最小信号并得出与$f$相同的分类结果。我们界定了充分信号可迁移的含义，并在音乐流派分类、情感识别与深度伪造检测三项任务中开展了大规模研究。研究发现：可迁移率因任务而异，音乐流派分类的充分信号仅约26%可迁移；其余任务的可迁移性波动显著，尤其在深度伪造检测中，部分模型呈现出独特的可迁移行为，我们将其称为"地平论模型"。通过深入分析深度伪造音频，我们还发现可迁移性分析能揭示模型间存在的信息论差异，这些差异无法被常规的准确率与精确度指标所捕捉。
URL
https://arxiv.org/abs/2604.02937
