Abstract
AI coding agents are now submitting pull requests (PRs) to software projects, acting not just as assistants but as autonomous contributors. As these agentic contributions are rapidly increasing across real repositories, little is known about how they behave in practice and why many of them fail to be merged. In this paper, we conduct a large-scale study of 33k agent-authored PRs made by five coding agents across GitHub. (RQ1) We first quantitatively characterize merged and not-merged PRs along four broad dimensions: 1) merge outcomes across task types, 2) code changes, 3) CI build results, and 4) review dynamics. We observe that tasks related to documentation, CI, and build update achieve the highest merge success, whereas performance and bug-fix tasks perform the worst. Not-merged PRs tend to involve larger code changes, touch more files, and often do not pass the project's CI/CD pipeline validation. (RQ2) To further investigate why some agentic PRs are not merged, we qualitatively analyze 600 PRs to derive a hierarchical taxonomy of rejection patterns. This analysis complements the quantitative findings in RQ1 by uncovering rejection reasons not captured by quantitative metrics, including lack of meaningful reviewer engagement, duplicate PRs, unwanted feature implementations, and agent misalignment. Together, our findings highlight key socio-technical and human-AI collaboration factors that are critical to improving the success of future agentic workflows.
Abstract (translated)
AI 编码代理现在正在向软件项目提交拉取请求（PR），它们不仅充当助手的角色，还作为自主贡献者发挥作用。随着这些代理性贡献在实际仓库中迅速增加，人们对它们的实际行为和许多未能被合并的原因知之甚少。本文进行了一项大规模研究，涵盖了 GitHub 上五个编码代理所创建的 33,000 多个代理提交的 PR。 （RQ1）我们首先从四个广泛的维度对已合并和未合并的 PR 进行了定量描述：1）任务类型跨领域的合并结果；2）代码变更；3）CI 构建结果；4）审查动态。我们观察到，与文档、CI 和构建更新相关的任务实现了最高的合并成功率，而性能和修复漏洞的任务表现最差。未被合并的 PR 往往涉及更大的代码更改，影响更多文件，并且经常未能通过项目的 CI/CD 流水线验证。 （RQ2）为了进一步探讨为什么一些代理性 PR 无法被合并，我们对600个 PR 进行了定性的分析，以衍生出一个拒绝模式的层次分类法。此分析补充了 RQ1 中量化发现中的未捕获原因，包括缺乏有意义的审查者互动、重复提交的问题、不受欢迎的功能实现以及代理与项目目标的偏差。 总的来说，我们的研究结果强调了改进未来自主工作流成功的关键社会技术及人机协作因素。
URL
https://arxiv.org/abs/2601.15195
