Abstract
When developing text classification models for real world applications, one major challenge is the difficulty to collect sufficient data for all text classes. In this work, we address this challenge by utilizing large language models (LLMs) to generate synthetic data and using such data to improve the performance of the models without waiting for more real data to be collected and labelled. As an LLM generates different synthetic data in response to different input examples, we formulate an automated workflow, which searches for input examples that lead to more ``effective'' synthetic data for improving the model concerned. We study three search strategies with an extensive set of experiments, and use experiment results to inform an ensemble algorithm that selects a search strategy according to the characteristics of a class. Our further experiments demonstrate that this ensemble approach is more effective than each individual strategy in our automated workflow for improving classification models using LLMs.
Abstract (translated)
在为实际应用开发文本分类模型时，一个主要的挑战是难以收集到所有类别足够的数据。在这项工作中，我们通过利用大型语言模型（LLM）生成合成数据来应对这一挑战，并使用这些数据改进模型性能，而无需等待更多的真实数据被收集和标注。由于LLM根据不同的输入示例会产生不同类型的合成数据，我们制定了一个自动化工作流程，该流程寻找能够产生更有“效果”的合成数据以提升目标模型的输入示例。通过广泛的实验研究了三种搜索策略，并利用实验结果来指导一种集成算法选择适应特定类别的搜索策略。进一步的实验证明，在使用LLM改进分类模型时，这种集成方法比自动化工作流程中的任何单一策略都更有效。
URL
https://arxiv.org/abs/2508.10000
