key: cord-0215489-o4hnuskh
authors: Terrucha, Ines; Domingos, Elias Fern'andez; Santos, Francisco C.; Simoens, Pieter; Lenaerts, Tom
title: The art of compensation: how hybrid teams solve collective risk dilemmas
date: 2022-05-13
journal: nan
DOI: nan
sha: aa228fae6d2555620b2ff84f12504eacb7eb5b06
doc_id: 215489
cord_uid: o4hnuskh

It is widely known how the human ability to cooperate has influenced the thriving of our species. However, as we move towards a hybrid human-machine future, it is still unclear how the introduction of AI agents in our social interactions will affect this cooperative capacity. Within the context of the one-shot collective risk dilemma, where enough members of a group must cooperate in order to avoid a collective disaster, we study the evolutionary dynamics of cooperation in a hybrid population made of both adaptive and fixed-behavior agents. Specifically, we show how the first learn to adapt their behavior to compensate for the behavior of the latter. The less the (artificially) fixed agents cooperate, the more the adaptive population is motivated to cooperate, and vice-versa, especially when the risk is higher. By pinpointing how adaptive agents avoid their share of costly cooperation if the fixed-behavior agents implement a cooperative policy, our work hints towards an unbalanced hybrid world. On one hand, this means that introducing cooperative AI agents within our society might unburden human efforts. Nevertheless, it is important to note that costless artificial cooperation might not be realistic, and more than deploying AI systems that carry the cooperative effort, we must focus on mechanisms that nudge shared cooperation among all members in the hybrid system.

As Artificial Intelligence (AI) systems are making more and more decisions autonomously, we are relinquishing decision control, for example by allowing intelligent machines to accomplish some of our goals independently or alongside us (e.g., using Google translate to enable business opportunities across different languages [14] ), within the context of hybrid human-machine socio-technical systems (e.g., sharing the road with self-driving cars [24] ). Given the extraordinary difficulties humans have demonstrated when trying to overcome global crises, such as the COVID-19 pandemic [18] or climate change [23] , the question can be raised on how AI agents may help to resolve the problems in coordinating the efforts in those and similar mixed-motive situations.

Even though many different works have advocated for the introduction of beneficial AI to promote human prosociality [13, 27, 30] , others have pointed that humans may be keen to exploiting this benevolent AI behavior in their own favor [3, 4, 17, 21] . Thus, before flooding society with AI applications with the promise that they could solve some of our most pressing issues, it is worth asking: What behavioral response can be expected in the presence of AI agent partners? How may decision-making potentially be affected? Will hybrid groups involving AI with predefined decision-processes actually achieve greater collective success?

We frame here these questions within the context of the Collective Risk Dilemma (CRD) [23] , a game that abstracts the conflict between helping the group to achieve a future goal at a personal cost, or free ride on the efforts of others and just collect the gains associated with achieving the goal. CRD is a public goods game with a delayed reward that is associated with societal problems like pandemic control through vaccination, climate negotiation to achieve CO 2 reduction and energy-grid usage by prosumers. Many experiments to assess human behavior have been performed [2, 6, 22, 23, 34] , where [11] found that replacing all human participants by AI agents has a positive impact on the success rate of human groups. Yet, they also showed that in hybrid groups of humans and AI agents this success is again reduced to the level of only human participants. Within this context, this manuscript aims to unravel in more detail the dynamics in hybrid human-agent groups, providing thus knowledge that allows one to design novel experiments to further this line of research on human-AI interactions in mixed-motive and competitive scenarios.

Using the two-action one-shot CRD as defined in [31] (see also Methods), the current study aims to show which strategic responses can be expected (i.e., fraction of cooperative strategies) in groups consisting of AI agents with predefined stochastic behaviors and whether these learned responses are sufficient for success in reaching the goals. Hence two types of participants are considered, i.e. adaptive individuals that can change their behavior over time based on the outcome of their interactions (a proxy for potential human responses) and artificial agents that have a probabilistic behavior that does not change as a result of the interactions (a proxy for average AI agent behavior). We apply a social learning approach to alter the strategy of the first type of individuals. They can switch between the possible actions in function of their success in the interactions; both when there are other adapting individuals or stochastic AI agents in groups of a given size. Such social adaptation can be achieved in different ways (e.g. Roth-Erev learning [19] , Q-learning [10] ), but here an evolutionary game theoretical approach is considered wherein strategic behaviors change in the adaptive individuals population by imitating those individuals that are performing the best [15, 16, 25, 35, 36] .

In our model, automation does not assume that every AI agent will be continuously learning while acting; rule-based systems are used in AI products and they are hard-wired in the systems as learning on the fly might be costly or even dangerous. Thinking about real-world AI applications, one should always consider that producers of AI products want to give guarantees on what the product does (also on what are its limitations, which is why we use a stochastic behavior that includes errors), and that allowing for extensive adaptation while in use may be very risky. It is important to note that in this work, we are not considering the AI designer, neither the dynamics involved behind them. We are simply probing: If we consider this space of behaviors for AI agents, what kind of human behaviors emerge given constant hybrid interactions? Even though CRD scenarios are used to model very high risk events like a pandemic or the climate change, the same kind of non-linearity could be observed within any industrial or software hybrid team, which if the project is not delivered might suffer the consequences of losing their bonuses or even their jobs. This is especially interesting to probe since most teams are already hybrid if we consider the extensive productivity softwares that are already available in the market.

We will show that the adaptive individuals in this type of hybrid teams -where humans are informed about their artificial counterparts use and limitations -respond by exploiting the benevolence of the AI agents, by avoiding to contribute with cooperative efforts when the latter are already meeting the threshold needed, as previously hinted to in [17] . On the contrary, when the AI agents added to each group are associated with a lower capacity to contribute for the collective endeavor, and if the risk is high, the adaptive population cooperation levels are boosted to compensate. What is thus observed, is that in the presence of a set of pre-specified AI behaviors in a hybrid group of "humans" that can adapt their behavior in function of their gains, one obtains compensating behavior in both directions, requiring thus additional mechanisms to align the efforts of both types of participants in the hybrid group dynamic. So, adding cooperative or pro-social agents into a group decision-making process with humans may increase the success of achieving the task, but it will not necessarily promote pro-social behavior.

In [3] it is pointed out that more experimental research is needed to really understand how human strategic decision-making changes when interacting with autonomous agents. Following on this, [21] compiles a review of more than 90 experimental studies that have made use of computerized players. Its main conclusions validate that indeed, human behavior changes when some of the other players are artificial, and furthermore, the behavior deviates to become more rational (or in other words, selfish), where humans are observed to actually try to exploit the artificial players. This last conclusion was both supported by [4] and [17] . The first finds that humans cheat more against machines than against other humans, and thus prefer to play with machines, in an experiment that tested honesty in opposition to the possibility of higher financial gains. The latter recently published an experimental study that concludes that humans are keen on exploiting benevolent AI in various different classical social dilemma games. Within the context of the CRD used for the present work, [11] groups participants in hybrid teams with AI agents. Even though 3 out of the 6 group members were AI agents that were successful in avoiding the risky outcome in previous treatments, the hybrid groups were not more successful than only human groups. Looking closer at the results, one can see that the average payoff of the humans in hybrid teams actually increases. These experimental results already hint towards the adoption of a compensatory behavior on part of the human members of the group once they are informed about the addition of somewhat collaborative agents to the group.

In contrast with aforementioned works, [13] and [30] point towards the possibility of engineering prosociality in human behaviour through the use of pro-social AI agents. In the pursuit of this idea, [27] assembles a comprehensive review on the use of robots and virtual agents to trigger pro-social behaviour. Out of 23 studies included, 52% reported positive effects in triggering such cooperative behavior. However, 22% were inconclusive and 26% reported mixed results. Moreover, while recent experimental works show that programming autonomous agents [8] that include emotion [7] or some form of communication [5] may positively impact human cooperation, it is still unclear what are the mechanisms facilitating this effect.

More directly related to our theoretical study, there are different works on the dynamics of how evolving populations adapt their behavioral profile given the introduction of agents with a fixed behavior (usually cooperative) either at the group level or at the population level [20, 26, 32, 33, 38] . With our research questions, we also aim at understanding how the introduction of agents with a fixed behavior, not necessarily cooperative, affects the evolution of cooperation.

In this manuscript we adopt the person one-shot CRD [1, 9, 10, 12, 29, 31, 31, 37] . Here, a group of individuals must each decide whether to Cooperate ( ), by contributing a fraction of their initial endowment , or to Defect ( ) and contribute nothing. If the group contains at least players, i.e., the group contributes in total ( ≤ ) to the public good, then each player may keep whatever is left of their initial endowment. Otherwise, there is a probability that all players will loose all their savings and receive a payoff of 0, hence the dilemma. Thus, the expected payoff of a and a player can be defined in functions of the number of in the group, :

where ( ) is the Heaviside unit step function, with ( ) = 0 if < 0 and ( ) = 1 otherwise.

We consider a population of adaptive agents which are randomly sampled into groups of size − to play the CRD with agents from population (whose individuals display a fixed averaged behavior). This allows us, as explained in the section below, to investigate the population dynamics of this dilemma. When engaging in group interactions, each adaptive agent can either cooperate or defect . The state of the population is then defined by the number of cooperators ∈ [0, ]. The behavior of the fixed agents is defined by their probability of cooperating in each interaction, ∈ [0, 1], thus, they implement a stochastic (or mixed) strategy. In each group we can calculate the expected payoff of Ds or Cs in function of the number of cooperators from the adaptive population, , the number of fixed agents and the payoff of a D (C)

The behavioral dynamics exhibited by the population of adaptive agents are governed by a social learning mechanism, where two randomly chosen individuals compare their fitness and imitate the one who is more successful within their social environment [15, 16, 25, 35, 36] . Their fitness is the measure of the success of their current strategy (their payoff) averaged over all different group interactions. It can be defined as a function of the aforementioned variables by taking into account the population state and the payoffs given by Eqs. 1 and 2. Following on this, the fitness equations for cooperative (C) and defective (D) strategies, can be written as:

Each agent in the population of adaptive agents may change its strategy profile at a given evolutionary step in the following way: an agent with a D (C) strategy is randomly selected from the population to adapt. With probability it will mutate into a C (D) strategy, otherwise, with probability 1 − , it will compare its fitness with another randomly selected agent (assuming the newly selected agent has a different strategy) [15, 16, 25, 28, 31, 35, 36] . In case imitation is selected, a D (C) strategy will turn into a C (D) with a probability

described by the Fermi function. This changes the state of the population of adaptive agents from to + 1. This probability becomes higher with a larger difference between the fitness of the two agents, − , or with a larger selection strength of the process, .

The transition probabilities that regulate the stochastic dynamics of population , by defining the probability of increasing (+) or decreasing (-) the number of cooperators within a population are given by:

where ( → ) is obtained by replacing with , and with in Eq. 6. From these equations, we can construct the complete Markov chain of the + 1 different states that fully describe the evolutionary process of the population . From this Markov Chain we can compute the stationary distribution ( ), the average cooperation level and the average group success of each population configuration.

To compute the stationary distribution ( ), we retrieve the eigenvector corresponding to the eigenvalue 1 of the tridiagonal transition matrix = [ ] [15, 25, 35] . The values are defined by the equations:

where the formulas that define ± ( ) can be consulted in Eqs. 7 and 8. From this it follows that the cooperation level of population (for a given set of parameters , , , and ) by averaging the fraction of cooperators in each population state, / , over the stationary distribution of states ( ) as given by:

As already mentioned, within the context of the CRD [1, 9, 10, 12, 23, 29, 31, 37] another relevant quantity to derive is the probability of success of each group in reaching the threshold necessary of cooperators to avoid the collective risk.

At the population level, we compute the fraction of groups in each population state that are successful by resorting to the multivariate hypergeometric sampling, as follows

where ( ) is the Heaviside unit step-function as in Eqs. 1 and 2 of the main text. 

4 RESULTS AND DISCUSSION 4.1 Non-cooperative AI agents require the adapting population to cooperate to be successful What level of cooperation can be expected from the population of adapting individuals? Specifically, we want to know how average cooperation (see Eq. 11) varies in relation to four parameters: the number of AI agents ( ) in each group, the probability of them cooperating ( ), the probability of risking complete loss of initial endowment ( ), and the number of cooperators needed per group to avoid that risk ( ). In Fig. 1 , the variation in average cooperation is shown in terms of the number of AI agents per group and their probability of cooperation . Here, the group size is set to = 6, and the collective risk of loss to = 0.9 if the group is unable to reach a threshold of at least cooperators. = 1 (or 3 or 5) thus means that at least 1 (or 3 or 5) participant needs to cooperate to avoid disaster. It is observed in Fig. 1 that when the AI agents are at a fully cooperative behavior, i.e. = 1, there is no motivation for human cooperation once the number of agents is equal to the threshold value, = , as they already contribute with all the effort necessary to avoid collective risk of loss. On the contrary, when AI agents that contribute with probability = 0 to the avoidance of that risk are introduced, then the − adaptive agents in the group are pressured to cooperate in order to reach by themselves. Still, if the threshold is high enough (in comparison to ), the introduction of at least somewhat cooperative agents makes the threshold more attainable, especially when we consider low costs of contribution as we do here with = 0.1 (see Fig. 1C ).

Prior work on the CRD [1, 9, 10, 12, 29, 31, 31, 37] , here shown through Fig. 2 where we consider no presence of artificial agents, has interestingly shown that as the risk of disaster when not reaching the target increases, average cooperation also increases, while full cooperation (i.e. a population consisting of only -strategists) is not achieved even for high risk situations (see panel A in Fig. 2 ). In contrast, our findings portrayed in Fig. 3 shows that one can now observe that by adding exogenous AI agents (from a population ) of increasing cooperativeness (higher ), the average cooperation level of the adapting population decreases, especially for higher risk (i.e., ). Yet, for this region of high risk, when the added agents are non-cooperative, one can observe a higher level of cooperation from the adaptive population.

This trend becomes clearly visible when increasing the number of AI agents in each interacting group, i.e. , as can be observed when comparing panels A-C in Fig. 3 : The steepness of the boundary between full and almost null is increased in relation to . This transition in the cooperation level of the adapting population is maximized for = , as is clearly illustrated in Fig. 1 . So when adding AI agents, the average investment in cooperation by the adapting population may decrease for both low risk and highrisk situations where the AI agents are mostly cooperative, but it has to be maximised when the AI agents are not that much into cooperation.

Overall, the findings in Fig. 1 are consistent with recent experimental work of [17] where algorithm exploitation is proven to be the main driver of the lack of cooperation with AI. Humans were found to act selfishly, by leaving the AI agents less well-off although not out of a competitive wish to end up better off than the machine. With our work we do indeed find that whenever the threshold is met completely by = fully cooperative AI agents, the behavior selected for is less likely be cooperative. However, when the threshold is high enough, the introduction of benevolent AI agents selects for cooperative behavior so that the risk of losing all the endowment is avoided.

When studying a CRD, the average group success of that population in avoiding the collective risk, (see Eq. 13), is even more essential, and of course deeply connected to the cooperation level observed in such a population. Fig. 4 visualizes for − adaptive individuals and AI agents sampled from population that engage in a CRD with = 6, = 3 and varying . Two regions are identified as responsible for high frequencies of group success: First, in Fig. 4A -C, we show that for high risk and low probability of cooperation ( ) by the AI agents introduced in each group, the average success is high (with a frequency > 0.9), which is consistent with the findings related to the cooperation level previously studied. Indeed, Fig. 4 appears as result of the superposition between the cooperation level of the adaptive population shown in Fig. 3 and vertical lines that would represent the boundary where high cooperative effort from the fixed agents is sufficient to achieve group success.

Within the boundaries marked by low , the boost in the average success rate is due to the evolved behavior in the population alone, since within this boundary the contribution of the AI agents to reach the threshold is minimum. Note also that the range for which cooperation is preferred is larger than what one would expect in a scenario where there are no AI agents (see Fig. 2B ). Second, for high values of , the increase in average group success is especially justified by the introduction of fully cooperative AI agents ( = 1). This effect is notably observable for values of closer to , as one can see from Fig. 4C . With the increment of , the average group success also becomes increasingly more dependent on , especially in groups where − < and ≥ , since the number of humans per group cannot change in any way the group outcome and the agents are already responsible for most of the possible cooperative effort.

The addition of AI agents to each group corresponds to a change in the game environment itself. Which is why the adaptive population is able to exploit such game transformation, benefiting from having information about the cooperation level of the added AI agents.

Here we show how exactly the addition of AI agents with associated probability to cooperate ( ) affects the stationary distribution of the population dynamics engaging in these hybrid group interactions. By introducing AI agents within a group, the group size is transformed into − , reducing thus the number of adapting individuals that can be introduced in each group, and -depending on whether the AI agents are cooperative with = 1 (or defective with = 0) -one also changes the number of those individuals needed to avoid the risk as → − (or → ). This effect can be observed by looking at Figs. 5A. and B. In Fig. 5A , the values − and were fixed to be able to observe these trends. Indeed, when adding AI agents that contribute nothing to reach threshold (with = 0) one can observe that it is the same as adding no AI agents, since − > and there is still the possibility for the cooperative members of the population to reach the threshold themselves. Furthermore, we find that for bigger values of − , cooperation becomes harder as the stationary distribution shifts to less cooperative states (similarly to the observed effect of increasing [31] ). When the added AI agents are cooperative, the transformation → − is also valid for changing the threshold → − . As it is shown in Fig. 5B ., the addition of a cooperative AI agents corresponds to both limiting the available places within the group and to lowering the threshold by that number of agents, who are contributing to the cooperative efforts themselves. , averaged over every population state. In A. = 1, in B. = 2 and in C. = = 3 AI agents are added in each group. D. = = 3 artificial members are added in each group, which corresponds to the threshold of cooperators needed to avoid the collective risk. In C., two boundaries can be clearly identified to achieve group success: the combination of a low AI agent cooperativeness and high risk of collective loss portray the upper left corner semi circle, and the presence of enough cooperative agents to achieve cooperators per group on their own draw a vertical line on the right of the graph. Overall, we can see that the introduction of AI agents increases the area of × where the threshold is met at least 90% (painted in yellow) of the times. For this figure, = 100, = 0.01, = 2, = 1, = 0.1.

However, when adding AI agents with a fixed non-deterministic behavior, i.e. for 0 < < 1, the resulting stationary distribution cannot be simply accounted for by a transformation in or in , as is evinced for example in Fig. 5A . Both the number of AI agents in each group and their cooperation level contribute to shape the resulting stationary distribution of the adaptive population. To support these conclusions one may look at Fig. 5C ., where we show the resulting stationary distributions for the same , and = , where the latter should account for the cooperative effort employed by the addition of agents to the group. We find that even though the is fixed at 1, different curves are obtained.

Overall, Fig. 5 exacerbates the relevance of this study by pointing towards the complexity of adapting a population simply by fixing the behavior of some group members present in each interaction.

Indeed, even though we are simply transforming the game environment, the resulting dynamics of the adaptive population are not trivial.

In this work we investigated what behavior is selected by social learning in the context of the one-shot CRD, when interactions occur in hybrid groups made of adapting agents (a proxy for human decision-making) and AI agents with fixed probabilistic behaviors (a proxy for average AI behavior). This model is used as a thought experiment to reason about the behavior one could expect in hybrid groups of humans and AI agents. It focuses on mixed-motive situations where there is a conflict between individual and common interests, as well as a risk that of not getting any benefits when not achieving the common goal, which affects the group as a whole. The model allowed us to understand under what conditions the introduction of AI agents in interacting groups actually contributes to an increase in their cooperation and their success rate in reaching a common goal. We have disentangled how the changes in success rate are related to the effort produced by the AI agents: whenever the latter are perceived as highly cooperative, the behavior of the adaptive population evolves to exploit the AI agents' benevolence. This effect was also shown by [17] . However, when the AI agents are low contributors, the adaptive population shifts to compensate those low contributions, as success can only be achieved when the goal is reached. Yet, this is only true for higher risk levels. In general, our model appears to indicate that the success rate of hybrid human-agents teams may be higher for a larger variety of settings than for human-only groups, or even for AI agents alone if they are not able to make the full contribution needed (lower ). Thus, our research suggests that there is potential benefit of using AI to nudge cooperation in human groups, which will need to be verified experimentally. Nevertheless, our findings also point towards an unbalanced future for human and AI teams. By working alongside cooperative AI, humans will eventually adapt to relax their own cooperative efforts. Hence, we must either identify AI policies that avoid this scenario and still promote cooperation to avoid collective risks or promote other modes of interaction in-between hybrid teams. 

Evolutionary Dynamics of Strategic Behavior in a Collective-Risk Dilemma

Voluntary provision of threshold public goods with continuous contributions: experimental evidence

Artificial Intelligence and Behavioral Economics

Honesty in the digital age

Cooperating with machines

On the provision of public goods with probabilistic and ambiguous thresholds

Shaping cooperation between humans and agents with emotion expressions and framing

Human cooperation when acting through autonomous machines

Timing uncertainty in collective risk dilemmas encourages group reciprocation and polarization

Modeling behavioral experiments on uncertainty and cooperation with population-based reinforcement learning

Which risk scenarios can drive the emergence of costly cooperation?

HCI as a means to prosociality in the economy

Human-Robots And Google Translate: A Case Study Of Translation Accuracy In Translating French-Indonesian Culinary Texts

Computation and simulation of evolutionary game dynamics in finite populations

Evolutionary games and population dynamics

Algorithm exploitation: Humans are keen to exploit benevolent AI

Driving the COVID-19 Response from the Center: Institutional Mechanisms to Ensure Whole-of-Government Coordination (English)

Learning dynamics in social dilemmas

Resilient cooperators stabilize long-run cooperation in the finitely repeated prisoner's dilemma

The Behavioral Economics of Artificial Intelligence: Lessons from Experiments with Computer Players. CESIfo Working Paper 7926

Cooperative interaction of rich and poor can be catalyzed by intermediate climate targets

The collective-risk social dilemma and the prevention of simulated dangerous climate change

Sharing the road with autonomous vehicles: Perceived safety and regulatory preferences

Evolutionary Dynamics: Exploring the Equations of Life

Pathological altruism

Towards prosocial design: A scoping review of the use of robots and virtual agents to trigger prosocial behaviour

Evolutionary dynamics of collective action in N-person stag hunt dilemmas

Climate change governance, cooperation and self-organization

Engineering prosociality with autonomous agents

Risk of collective failure provides an escape from the tragedy of the commons

Evolution of collective fairness in hybrid populations of humans and agents

Locally noisy autonomous agents improve global human coordination in network experiments

Inequality, communication, and the avoidance of disastrous climate change in a public goods game

Exploration dynamics in evolutionary games

Pairwise comparison and selection temperature in evolutionary game dynamics

A bottomup institutional approach to cooperative governance of risky commons

Climate policies under wealth inequality