Abstract
In Artificial Intelligence (AI), language models have gained significant importance due to the widespread adoption of systems capable of simulating realistic conversations with humans through text generation. Because of their impact on society, developing and deploying these language models must be done responsibly, with attention to their negative impacts and possible harms. In this scenario, the number of AI Ethics Tools (AIETs) publications has recently increased. These AIETs are designed to help developers, companies, governments, and other stakeholders establish trust, transparency, and responsibility with their technologies by bringing accepted values to guide AI's design, development, and use stages. However, many AIETs lack good documentation, examples of use, and proof of their effectiveness in practice. This paper presents a methodology for evaluating AIETs in language models. Our approach involved an extensive literature survey on 213 AIETs, and after applying inclusion and exclusion criteria, we selected four AIETs: Model Cards, ALTAI, FactSheets, and Harms Modeling. For evaluation, we applied AIETs to language models developed for the Portuguese language, conducting 35 hours of interviews with their developers. The evaluation considered the developers' perspective on the AIETs' use and quality in helping to identify ethical considerations about their model. The results suggest that the applied AIETs serve as a guide for formulating general ethical considerations about language models. However, we note that they do not address unique aspects of these models, such as idiomatic expressions. Additionally, these AIETs did not help to identify potential negative impacts of models for the Portuguese language.
Abstract (translated)
在人工智能（AI）领域，语言模型由于能够通过文本生成模拟与人类的真实对话而变得尤为重要。鉴于其对社会的影响，开发和部署这些语言模型必须负责任地进行，并注意它们可能带来的负面影响和潜在危害。在这种背景下，最近有关于AI伦理工具（AIETs）的出版物数量有所增加。这些AIETs旨在帮助开发者、公司、政府和其他利益相关者通过引入公认的道德价值观来建立其技术的信任、透明度和责任，以指导AI的设计、开发和使用阶段。然而，许多AIET缺乏良好的文档记录、使用示例以及在实践中有效的证明。 本文提出了一种评估语言模型中AIET的方法。我们的方法包括对213个AIET的广泛文献调查，并通过应用包含和排除标准后选择了四个AIET：Model Cards（模型卡片）、ALTAI、FactSheets（事实表）和Harms Modeling（危害建模）。在评估过程中，我们将这些AIET应用于为葡萄牙语开发的语言模型中，并对开发者进行了总计35小时的访谈。评估考虑了开发者从他们的视角出发使用和评价AIET的情况及其质量，在帮助识别他们模型的伦理考量方面的作用。 结果显示，应用的AIET可以作为制定关于语言模型的一般伦理考量的指南。然而，我们注意到它们没有解决这些模型的独特方面，如习惯用语等。此外，这些AIET未能帮助识别葡萄牙语语言模型潜在的负面效果。
URL
https://arxiv.org/abs/2512.15791
