Abstract
GitHub is the world's most popular platform for storing, sharing, and managing code. Every GitHub repository has a README file associated with it. The README files should contain project-related information as per the recommendations of GitHub to support the usage and improvement of repositories. However, GitHub repository owners sometimes neglected these recommendations. This prevents a GitHub repository from reaching its full potential. This research posits that the comprehensiveness of a GitHub repository's README file significantly influences its adoption and utilization, with a lack of detail potentially hindering its full potential for widespread engagement and impact within the research community. Large Language Models (LLMs) have shown great performance in many text-based tasks including text classification, text generation, text summarization and text translation. In this study, an approach is developed to fine-tune LLMs for automatically classifying different sections of GitHub README files. Three encoder-only LLMs are utilized, including BERT, DistilBERT and RoBERTa. These pre-trained models are then fine-tuned based on a gold-standard dataset consisting of 4226 README file sections. This approach outperforms current state-of-the-art methods and has achieved an overall F1 score of 0.98. Moreover, we have also investigated the use of Parameter-Efficient Fine-Tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) and shown an economical alternative to full fine-tuning without compromising much performance. The results demonstrate the potential of using LLMs in designing an automatic classifier for categorizing the content of GitHub README files. Consequently, this study contributes to the development of automated tools for GitHub repositories to improve their identifications and potential usages.
Abstract (translated)
GitHub 是全球最受欢迎的代码存储、共享和管理平台。每个 GitHub 仓库都关联有一个 README 文件，根据 GitHub 的推荐应包含项目相关信息以支持仓库的使用与改进。然而，一些 GitHub 仓库的所有者有时会忽视这些推荐，这阻碍了仓库发挥其全部潜力。 本研究认为，GitHub 仓库的 README 文件内容详尽度对其采用和利用具有重要影响。缺乏详细信息可能会限制其在研究社区内的广泛参与和影响力。大型语言模型（LLMs）在许多文本相关任务中表现出色，包括文本分类、生成、摘要以及翻译。在这项研究中，我们开发了一种方法来微调 LLMs 以自动对 GitHub README 文件的不同部分进行分类。所使用的三种编码器模型为 BERT、DistilBERT 和 RoBERTa。然后基于包含 4226 个 README 文件段落的黄金标准数据集对这些预训练模型进行了微调。 这种方法超越了目前最先进的方法，并在总体 F1 得分方面达到了 0.98 的成绩。此外，我们还探讨了使用参数高效微调（PEFT）技术如低秩适应（LoRA），提供了一种经济高效的全量微调替代方案，在不牺牲太多性能的情况下实现目标。结果显示了 LLMs 在设计自动分类器以对 GitHub README 文件内容进行分类的潜力。 因此，这项研究为开发自动化工具改进 GitHub 仓库的识别和潜在使用做出了贡献。
URL
https://arxiv.org/abs/2507.21899
