Abstract
Automated content moderation for collaborative knowledge hubs like Wikipedia or Wikidata is an important yet challenging task due to multiple factors. In this paper, we construct a database of discussions happening around articles marked for deletion in several Wikis and in three languages, which we then use to evaluate a range of LMs on different tasks (from predicting the outcome of the discussion to identifying the implicit policy an individual comment might be pointing to). Our results reveal, among others, that discussions leading to deletion are easier to predict, and that, surprisingly, self-produced tags (keep, delete or redirect) don't always help guiding the classifiers, presumably because of users' hesitation or deliberation within comments.
Abstract (translated)
针对维基百科或Wikidata等协作知识中心的自动化内容审核是一项重要但具有挑战性的任务，原因众多。本文中，我们构建了一个数据库，涵盖了多个维基站点上标记为待删除文章的相关讨论，并涉及三种语言的数据。然后，我们使用这些数据来评估一系列大型语言模型（LMs）在不同任务上的表现（从预测讨论的结果到识别个人评论可能指向的隐含政策）。我们的研究结果表明，例如，导致内容被删除的讨论更容易被预测；此外，令人意外的是，用户自动生成的标签（保留、删除或重定向）并不总是有助于引导分类器做出判断，这可能是由于用户在评论中表现出犹豫不决或深思熟虑所致。
URL
https://arxiv.org/abs/2503.10294
