Abstract
Explainability techniques are crucial in gaining insights into the reasons behind the predictions of deep learning models, which have not yet been applied to chemical language models. We propose an explainable AI technique that attributes the importance of individual atoms towards the predictions made by these models. Our method backpropagates the relevance information towards the chemical input string and visualizes the importance of individual atoms. We focus on self-attention Transformers operating on molecular string representations and leverage a pretrained encoder for finetuning. We showcase the method by predicting and visualizing solubility in water and organic solvents. We achieve competitive model performance while obtaining interpretable predictions, which we use to inspect the pretrained model.
Abstract (translated)
解释性技术在获取深度学习模型预测背后的原因非常重要，而这些模型还没有应用于化学语言模型。我们提出了一种可解释的人工智能技术，将个体原子的重要性与其对这些模型预测的准确性赋予关联。我们的算法将相关性信息向化学输入字符串反向传播，并可视化个体原子的重要性。我们专注于在分子字符串表示中进行自我注意力Transformer的运行，并利用预训练编码器进行微调。我们展示该方法的方法是通过预测和可视化在水中和有机溶剂中的溶解度来实现的。我们实现了竞争模型性能，同时获得了可解释的预测，这些预测我们用于检查预训练模型。
URL
https://arxiv.org/abs/2305.16192
