Abstract
Synthetic data offers a promising solution for mitigating data scarcity and demographic bias in mental health analysis, yet existing approaches largely rely on pretrained large language models (LLMs), which may suffer from limited output diversity and propagate biases inherited from their training data. In this work, we propose a pretraining-free diffusion-based approach for synthetic text generation that frames bias mitigation as a style transfer problem. Using the CARMA Arabic mental health corpus, which exhibits a substantial gender imbalance, we focus on male-to-female style transfer to augment underrepresented female-authored content. We construct five datasets capturing varying linguistic and semantic aspects of gender expression in Arabic and train separate diffusion models for each setting. Quantitative evaluations demonstrate consistently high semantic fidelity between source and generated text, alongside meaningful surface-level stylistic divergence, while qualitative analysis confirms linguistically plausible gender transformations. Our results show that diffusion-based style transfer can generate high-entropy, semantically faithful synthetic data without reliance on pretrained LLMs, providing an effective and flexible framework for mitigating gender bias in sensitive, low-resource mental health domains.
Abstract (translated)
合成数据为缓解心理健康分析中的数据稀缺和人口偏见提供了有希望的解决方案，然而现有的方法大多依赖于预训练的大规模语言模型（LLM），这可能会导致输出多样性有限，并传播其训练数据中固有的偏见。在本研究中，我们提出了一种无需预训练的基于扩散的方法来生成合成文本，将偏见缓解视为一种风格转换问题。使用CARMA阿拉伯语心理健康语料库，在该语料库中性别失衡显著，我们将重点放在从男性向女性的风格转移上，以增强代表性不足的女性作者内容。我们构建了五个数据集，捕捉阿拉伯语表达中的不同语言和语义方面的性别差异，并为每种情况训练单独的扩散模型。定量评估表明，在源文本与生成文本之间保持一致的语义忠实度的同时，还实现了有意义的表面风格差异，而定性分析则确认了合乎逻辑的语言上的性别转换。我们的结果表明，基于扩散的风格转移可以生成高熵、语义忠实的合成数据，无需依赖预训练的大规模语言模型，在敏感且资源匮乏的心理健康领域提供了一种有效且灵活的框架以缓解性别偏见。
URL
https://arxiv.org/abs/2601.14124
