Abstract
We generated 25000 conversations labeled with Big Five Personality traits using prompt programming at GPT-3. Then we train Big Five classification models with these data and evaluate them with 2500 data from generated dialogues and real conversational datasets labeled in Big Five by human annotators. The results indicated that this approach is promising for creating effective training data. We then compare the performance by different training approaches and models. Our results suggest that using Adapter-Transformers and transfer learning from pre-trained RoBERTa sentiment analysis model will perform best with the generated data. Our best model obtained an accuracy of 0.71 in generated data and 0.65 in real datasets. Finally, we discuss this approach's potential limitations and confidence metric.
Abstract (translated)
我们通过在GPT-3中prompt编程生成了25000次带有Big Five人格特征的对话,并将这些数据用于训练Big Five分类模型,同时使用人类标注的生成的对话和真实对话数据集来评估模型的性能。结果表明,这种方法对于生成有效的训练数据具有前景。然后,我们比较了不同的训练方法和模型的性能。我们的结果表明,使用Adapter-Transformers和从预训练的RoBERTa情感分析模型中学习的迁移学习将表现最佳。我们的最佳模型在生成的数据上获得了0.71的准确性,而在真实数据上获得了0.65的准确性。最后,我们讨论了这种方法的潜在限制和置信度度量。
URL
https://arxiv.org/abs/2303.12279
