Abstract
This study presents a multi-stage approach to mental health classification by leveraging traditional machine learning algorithms, deep learning architectures, and transformer-based models. A novel data set was curated and utilized to evaluate the performance of various methods, starting with conventional classifiers and advancing through neural networks. To broaden the architectural scope, recurrent neural networks (RNNs) such as LSTM and GRU were also evaluated to explore their effectiveness in modeling sequential patterns in the data. Subsequently, transformer models such as BERT were fine-tuned to assess the impact of contextual embeddings in this domain. Beyond these baseline evaluations, the core contribution of this study lies in a novel training strategy involving a dual-model architecture composed of a teacher and a student network. Unlike standard distillation techniques, this method does not rely on soft label transfer; instead, it facilitates information flow through both the teacher model's output and its latent representations by modifying the loss function. The experimental results highlight the effectiveness of each modeling stage and demonstrate that the proposed loss function and teacher-student interaction significantly enhance the model's learning capacity in mental health prediction tasks.
Abstract (translated)
这项研究提出了一种多阶段方法，用于通过利用传统机器学习算法、深度学习架构和基于变换器的模型来分类心理健康。研究人员精心策划并使用了一个新颖的数据集，以评估各种方法的性能，从传统的分类器开始，并逐步过渡到神经网络。为了拓宽架构范围，还评估了诸如LSTM（长短期记忆）和GRU（门控循环单元）等递归神经网络，以探索它们在建模数据中序列模式方面的有效性。 随后，对BERT等变换模型进行了微调，以评估上下文嵌入在此领域的影响力。除了这些基准测试外，这项研究的核心贡献在于一种新颖的训练策略，该策略涉及由教师和学生网络组成的双模型架构。与标准蒸馏技术不同，这种方法不依赖于软标签转移；相反，通过修改损失函数来促进信息流通过教师模型的输出及其潜在表示。 实验结果突显了每个建模阶段的有效性，并展示了所提出的损失函数及师生互动显著增强了心理健康预测任务中模型的学习能力。
URL
https://arxiv.org/abs/2504.07245
