Abstract
Recent work on decentralized computational trust models for open Multi Agent Systems has resulted in the development of CA, a biologically inspired model which focuses on the trustee's perspective. This new model addresses a serious unresolved problem in existing trust and reputation models, namely the inability to handle constantly changing behaviors and agents' continuous entry and exit from the system. In previous work, we compared CA to FIRE, a well-known trust and reputation model, and found that CA is superior when the trustor population changes, whereas FIRE is more resilient to the trustee population changes. Thus, in this paper, we investigate how the trustors can detect the presence of several dynamic factors in their environment and then decide which trust model to employ in order to maximize utility. We frame this problem as a machine learning problem in a partially observable environment, where the presence of several dynamic factors is not known to the trustor and we describe how an adaptable trustor can rely on a few measurable features so as to assess the current state of the environment and then use Deep Q Learning (DQN), in a single-agent Reinforcement Learning setting, to learn how to adapt to a changing environment. We ran a series of simulation experiments to compare the performance of the adaptable trustor with the performance of trustors using only one model (FIRE or CA) and we show that an adaptable agent is indeed capable of learning when to use each model and, thus, perform consistently in dynamic environments.
Abstract (translated)
近年来，在去中心化计算信任模型为开放多智能系统的研究中，已经发展出了CA，一种以生物启发的模型，重点关注受托人的视角。这种新模型解决现有信任和声誉模型的一个严重问题，即无法处理不断变化的行为和代理商对系统的持续进入和退出。在之前的工作里，我们比较了CA与FIRE，一个著名的信任和声誉模型，发现在信任者人口变化时，CA更优越，而FIRE对受托人人口变化更加鲁棒。因此，在本文中，我们研究了信任者如何检测其环境中的多个动态因素，然后决定要使用哪种信任模型来最大化效用。我们将这个问题描述为在部分可观测环境中运行的机器学习问题，其中几个动态因素的存在对于信任者来说是不知道的，然后我们描述了一个可适应的信任者如何利用一些可观测的特征来评估当前环境状态，然后在一个单一智能体的强化学习中使用深度Q学习（DQN）来学习如何适应变化的环境。我们进行了一系列仿真实验来比较可适应信任者与仅使用一种模型（FIRE或CA）的信托者的性能，结果表明，可适应的代理确实可以在动态环境中学习何时使用每种模型，从而在环境中表现一致。
URL
https://arxiv.org/abs/2404.18296
