Part V. Agentbased models vs ODE models
V2. A rather general model for games played in wellmixed populations
1. Goal
The goal of this chapter is to extend the wellmixed population model we developed in chapter II3 (nxnimitateifbetternoise) by adding two features that will make our program significantly more general:
 The possibility to use expected payoffs. In all the models we developed in Part II, agents obtain their payoffs by playing with one agent, chosen at random. In this chapter we will allow agents to use expected payoffs. The expected payoff of strategy is also the average payoff that an strategist would obtain if she played with the whole population.
 The possibility to model other decision rules besides the imitate if better rule.
2. Motivation
Once we have implemented different decision rules and the option to use expected payoffs, we will be able to model many different agentbased evolutionary dynamics. In the next chapter, we will develop the mean dynamic of each of these agentbased evolutionary dynamics, and we will see that they correspond to ODE models that have been studied in the literature.
3. Description of the model
We depart from the program we implemented in chapter II3 (nxnimitateifbetternoise).^{[1]} You can find the full description of this model in chapter II3. Our extension will include the following three additional parameters:
 payofftouse. This parameter specifies the type of payoff that agents use in each tick. It will be implemented with a chooser, with two possible values:
 “playwithonerdagent“: agents play with another agent chosen at random.
 “usestrategyexpectedpayoff“: agents use their strategy’s expected payoff.
 decisionrule. This parameter determines the decision rule that agents follow to update their strategies, just like in the models we developed in chapter III4 and chapter IV4. Note that all decision rules use the agents’ payoffs, and these are computed following the method prescribed by parameter payofftouse. Parameter decisionrule will be implemented with a chooser, with seven possible values:
 “imitateifbetter”. This is the imitate if better rule already implemented in the current model.
 “imitativepairwisedifference”. This is the imitative pairwisedifference rule we saw in chapter I2. Under this rule, the revising agent looks at another individual at random and considers imitating her strategy only if her payoff is greater than the reviser’s payoff; in that case, he switches with probability proportional to the payoff difference. In order to turn the payoff difference into a meaningful probability, we divide the payoff difference by the maximum possible payoff minus the minimum possible payoff.
 “imitativelinearattraction”. Under this rule, the revising agent selects another individual at random and imitates her strategy with probability equal to the difference between the observed individual’s payoff and the minimum possible payoff, divided by the maximum possible payoff minus the minimum possible payoff. (The revising agent’s payoff is ignored.)
 “imitativelineardissatisfaction”. Under this rule, the revising agent selects another agent at random and imitates her strategy with probability equal to the difference between the maximum possible payoff and his own payoff, divided by the maximum possible payoff minus the minimum possible payoff. (The observed agent’s payoff is ignored.)
The four decision rules above are imitative, i.e. they all start by selecting one agent and consider imitating her strategy. In contrast, the following decision rules are direct (Sandholm, 2010a).^{[2]} Under direct decision rules, revising agents choose candidate strategies directly without looking at any agent, so a strategy’s popularity does not directly influence the probability with which it is considered. In the following direct decision rules, if payofftouse = “playwithonerdagent“, each strategy is tested against one random individual, potentially different in each test.
 “directbest”. The revising agent selects the strategy with the greatest payoff. Ties are resolved uniformly at random.
 “directpairwisedifference”. The revising agent randomly selects another strategy, and considers adopting it only if its payoff is greater; in that case, he switches with probability proportional to the payoff difference. In order to turn the difference in payoffs into a meaningful probability, we divide the payoff difference by the maximum possible payoff minus the minimum possible payoff.
 “directpositiveproportionalm”. The revising agent selects one strategy at random, with probabilities proportional to payoffs raised to parameter m, and adopts it. Parameter m controls the intensity of selection (see below). We do not allow for negative payoffs when using this rule.
 m. This is the parameter that controls the intensity of selection in decision rule directpositiveproportionalm.
Everything else stays as described in chapter II3.
4. Extension I. Implementation of different ways of computing payoffs
In this first extension, we just want to implement the option to use expected payoffs.
4.1. Skeleton of the code
The main procedure we have to modify is to updatepayoff, which is where payoffs are updated. Figure 1 below shows the skeleton of the new implementation of procedure to updatepayoff.
4.2. Interface design
We depart from the model we created in chapter II3 (nxnimitateifbetternoise), so if you want to preserve it, now is a good time to duplicate it. The current interface looks as shown in figure II31. The only change we have to make to the interface is to add parameter payofftouse. We should add it as a chooser with possible values “playwithonerdagent” and “usestrategyexpectedpayoff“. The new interface should look as figure 2 below.
4.3. Code
4.3.1. Implementation of to updatepayoff
The way payoffs are computed is determined by parameter payofftouse:
 If payofftouse = “playwithonerdagent“, agents play with another agent chosen at random.
 If payofftouse = “usestrategyexpectedpayoff“, agents use their strategy’s expected payoff.
A nice and modular way of implementing procedure to updatepayoff is to create two new procedures, i.e., to playwithonerdagent and to usestrategyexpectedpayoff and call one or the other depending on the value of payofftouse (see skeleton in figure 1). Thus, the new code of to updatepayoff could look as follows:
to updatepayoff ifelse payofftouse = "playwithonerdagent" [playwithonerdagent] [usestrategyexpectedpayoff] end
Given that we have chosen the names of the two new procedures to match the possible values of parameter payofftouse, we can also use run (a primitive that can take a string containing the name of a command as an input, and it runs the command) as follows:
to updatepayoff run payofftouse end
Our following step is to implement the two new procedures.
4.3.2. Implementation of to playwithonerdagent
Note that in our baseline model nxnimitateifbetternoise, agents obtained a payoff by playing with another agent chosen at random. Thus, for the implementation of to playwithonerdagent, we can just use the code that we previously had in procedure to updatepayoff:
to playwithonerdagent let mate oneof other players set payoff item ([strategy] of mate) (item strategy payoffmatrix) end
4.3.3. Implementation of to usestrategyexpectedpayoff
This new procedure should assign the appropriate expected payoff to the agent who runs it. Note that we do not want every agent to compute the expected payoff of their strategy. This would be inefficient because there are usually many more agents than strategies. Instead, we should compute the expected payoff of each strategy just once in each tick, and have these payoffs ready for agents to read.
We will implement a procedure to compute expected payoffs in the next section, but for now, let us assume that these expected payoffs are available to agents in a global variable called strategyexpectedpayoffs. This variable will contain a list with the expected payoff of each strategy, in order. Assuming that, the code for procedure to usestrategyexpectedpayoff is particularly simple:
to usestrategyexpectedpayoff set payoff item strategy strategyexpectedpayoffs end
4.3.4. Implementation of to updatestrategyexpectedpayoffs
Our goal now is to code the procedure in charge of computing the expected payoff for each strategy and of storing them in a global variable named strategyexpectedpayoffs. Let us call this new procedure to updatestrategyexpectedpayoffs, and work through the logic together.
It is often easier to start with a concrete example and then try to generalize. Let us start by thinking how to compute the expected payoff of strategy 0. To do this, we need the payoffs that strategy 0 can obtain (e.g., [0 1 1] in figure 2) and we need the frequencies of each strategy in the population (e.g. [0.2 0.3 0.5]). Then, we would only have to multiply the two lists element by element ( [ 0*0.2 1*0.3 1*0.5 ] ) and sum all the elements of the resulting list (0*0.2 + 1*0.3 + 1*0.5).
Our current code already computes the frequencies of each strategy in procedure to updategraph, so we already have the code for that:
let strategyfrequencies map [ n > count players with [strategy = n] / nofplayers ] strategynumbers
Assuming we have the strategy frequencies stored in strategyfrequencies, to compute the expected payoff of strategy 0 in our example we would only have to do:
sum (map * [ 0 1 1 ] strategyfrequencies)
Now is the time to generalize. To compute the expected payoff of each of the strategies, we just have to apply the code snippet above to the list of payoffs of each strategy. Note that the payoff matrix contains precisely the lists of payoffs for each strategy, so we can use map as follows:
set strategyexpectedpayoffs map [ listofpayoffs >
sum (map * listofpayoffs strategyfrequencies)
] payoffmatrix
As mentioned before, since we want agents to be able to read the strategy expected payoffs, we should define variable strategyexpectedpayoffs as global. Also, there is no point in computing the strategies frequencies more than once in each tick, so we should also define strategyfrequencies as a global variable, and update it only in one place. And finally, now we will be using the strategy numbers at two places (procedures to updatestrategyexpectedpayoffs and to updategraph), so we could also define strategynumbers as a global variable:
globals [ payoffmatrix nofstrategies nofplayers strategynumbers ;; <= new global variable strategyfrequencies ;; <= new global variable strategyexpectedpayoffs ;; <= new global variable ]
With all this, the code for procedure to updatestrategyexpectedpayoffs would be:
to updatestrategyexpectedpayoffs set strategyfrequencies map [ n > count players with [strategy = n] / nofplayers ] strategynumbers set strategyexpectedpayoffs map [ listofpayoffs > sum (map * listofpayoffs strategyfrequencies) ] payoffmatrix end
Since we are updating strategyfrequencies in this procedure, and the variable is now global, we can remove that computation from procedure to updategraph. Similarly, since variable strategynumbers is now global and its value does not change over the course of a simulation run, we should move its computation from to updategraph to a setup procedure, such as to setuppayoffs:
to updategraph ; let strategynumbers (range nofstrategies) ; let strategyfrequencies map [ n > ; count players with [strategy = n] / nofplayers ; ] strategynumbers
setcurrentplot "Strategy Distribution" let bar 1 foreach strategynumbers [ n > setcurrentplotpen (word n) plotxy ticks bar set bar (bar  (item n strategyfrequencies)) ] setplotyrange 0 1 end
to setuppayoffs set payoffmatrix readfromstring payoffs set nofstrategies length payoffmatrix set strategynumbers range nofstrategies ;; <= new line end
Now, we should make sure that we call procedure to updatestrategyexpectedpayoffs (where strategyfrequencies is updated) before running procedure to updategraph, where these frequencies are used. We do that at procedures to setup and to go. We also take this opportunity to improve the efficiency of the model by including the primitive nodisplay in procedure to setup, right after clearall.
to setup clearall nodisplay ;; <= new line to improve efficiency setuppayoffs setupplayers setupgraph resetticks updatestrategyexpectedpayoffs ;; <= new line updategraph end
to go ask players [updatepayoff] ask players [ if (randomfloat 1 < probrevision) [ updatestrategyafterrevision ] ] ask players [updatestrategy] tick updatestrategyexpectedpayoffs ;; <= new line updategraph end
Finally, note that, since procedure to updatestrategyexpectedpayoffs is called from to go, we know that variable strategyexpectedpayoffs is updated in every tick, as required.
There is a subtle final issue here. We have to make sure that, given a certain population state (i.e. once agents have updated their strategies), variable strategyexpectedpayoffs is updated before agents update their payoffs in procedure to updatepayoff. Note that this is so in our current code, even at the first tick (since procedure to updatestrategyexpectedpayoffs is called from to setup too). In other words, whenever agents update their payoffs, variable strategyexpectedpayoffs contains the updated expected payoffs, i.e., the ones corresponding to the current population state.
Table 1 below shows a sketch of the order in which the main procedures are executed in a simulation run. The lines highlighted in yellow indicate when the population state changes. Note that, in between highlighted lines, procedure to updatestrategyexpectedpayoffs is executed before agents update their payoffs in procedure to updatepayoff.
↓  setup  ↓  ……..  ;; <= various setup procedures 
↓  ↓  updatestrategyexpectedpayoffs  
↓  ↓  updategraph  
↓  go  ↓  ask players [updatepayoff] 

↓  ↓  ……..  ;; <= agents update strategyafterrevision  
↓  ↓  ask players [updatestrategy] 
;; <= change of population state  
↓  ↓  tick  
↓  ↓  updatestrategyexpectedpayoffs  
↓  ↓  ……..  ;; <= updategraph  
↓  go  ↓  ask players [updatepayoff] 

↓  ↓  ……..  ;; <= agents update strategyafterrevision  
↓  ↓  ask players [updatestrategy] 
;; <= change of population state  
↓  ↓  tick  
↓  ↓  updatestrategyexpectedpayoffs  
↓  ↓  ……..  ;; <= updategraph  
↓  go  ↓  ask players [updatepayoff] 

↓  ↓  ……..  ;; <= agents update strategyafterrevision  
↓  ↓  ask players [updatestrategy] 
;; <= change of population state  
↓  ↓  tick  
↓  ↓  updatestrategyexpectedpayoffs  
↓  ↓  ……..  ;; <= updategraph 
4.3.5. Final checks
We can conduct a few simple checks to gain confidence in our code. For instance, to see the different payoffs that 0strategists obtain, we can include the following line in procedure to go, right after agents update their payoffs:
show removeduplicates [payoff] of players with [strategy = 0]
to go ask players [updatepayoff] show removeduplicates [payoff] of players with [strategy = 0] ask players [ if (randomfloat 1 < probrevision) [ updatestrategyafterrevision ] ] ask players [updatestrategy] tick updatestrategyexpectedpayoffs updategraph end
With this code in place, please, try to answer the four questions below:
1. Can you find out what we should see in the command center if we click on figure 2?
and then on with the setting shown inSince payofftouse = “playwithonerdagent“, agents play with another agent chosen at random. Thus, most likely, some 0strategists will meet another 0strategist (and obtain a payoff of 0), some will meet a 1strategist (and obtain a payoff of 1), and some will meet a 2strategist (and obtain a payoff of 1). Therefore, almost for certain we will see a list containing these three payoffs [0 1 1], in any order.
2. And if we change payofftouse to = “usestrategyexpectedpayoff“?
The payoff for all 0strategists is now the expected payoff of strategy 0, so we should only see one number in the list. At the initial state, this number should be:
(0*(167/500) + 1*(167/500) + 1*(166/500)) = 1/500 = 0.002
We actually see [0.0020000000000000018] due to floatingpoint errors.
3. What should we observe if we remove with [strategy = 0] from the line we added in procedure to go, still using payofftouse to = “usestrategyexpectedpayoff“?
We should see a list containing the three expected payoffs, in any order. At the initial state, these payoffs are:
( 0*(167/500) + 1*(167/500) + 1*(166/500)) = 1/500 = 0.002
( 1*(167/500) + 0*(167/500) + 1*(166/500)) = 1/500 = 0.002
(1*(167/500) + 1*(167/500) + 0*(166/500)) = 0
We actually see [0 0.0020000000000000018 0.0020000000000000018] due to floatingpoint errors.
4. And if we now change payofftouse to = “playwithonerdagent“?
Almost certainly, we will see a list containing the three payoffs [0 1 1], in any order.
4.4. Complete code of Extension I in the Code tab
Well done! We have now finished the first extension! You can use the following link to download the complete NetLogo model: nxnimitateifbetternoisepayofftouse.nlogo.
globals [ payoffmatrix nofstrategies nofplayers strategynumbers ;; <= new global variable strategyfrequencies ;; <= new global variable strategyexpectedpayoffs ;; <= new global variable ] breed [players player] playersown [ strategy strategyafterrevision payoff ] ;;;;;;;;;;;;; ;;; SETUP ;;; ;;;;;;;;;;;;; to setup clearall nodisplay ;; <= new line to improve efficiency setuppayoffs setupplayers setupgraph resetticks updatestrategyexpectedpayoffs ;; <= new line updategraph end to setuppayoffs set payoffmatrix readfromstring payoffs set nofstrategies length payoffmatrix set strategynumbers range nofstrategies ;; <= new line end to setupplayers let initialdistribution readfromstring nofplayersforeachstrategy if length initialdistribution != length payoffmatrix [ usermessage (word "The number of items in\n" "nofplayersforeachstrategy (i.e. " length initialdistribution "):\n" nofplayersforeachstrategy "\nshould be equal to the number of rows\n" "in the payoff matrix (i.e. " length payoffmatrix "):\n" payoffs ) ] let i 0 foreach initialdistribution [ j > createplayers j [ set payoff 0 set strategy i set strategyafterrevision strategy ] set i (i + 1) ] set nofplayers count players end to setupgraph setcurrentplot "Strategy Distribution" foreach (range nofstrategies) [ i > createtemporaryplotpen (word i) setplotpenmode 1 setplotpencolor 25 + 40 * i ] end ;;;;;;;;;; ;;; GO ;;; ;;;;;;;;;; to go ask players [updatepayoff] ask players [ if (randomfloat 1 < probrevision) [ updatestrategyafterrevision ] ] ask players [updatestrategy] tick updatestrategyexpectedpayoffs ;; <= new line updategraph end ;;;;;;;;;;;;;;;;;;;;;; ;;; UPDATE PAYOFFS ;;; ;;;;;;;;;;;;;;;;;;;;;; to updatestrategyexpectedpayoffs set strategyfrequencies map [ n > count players with [strategy = n] / nofplayers ] strategynumbers set strategyexpectedpayoffs map [ listofpayoffs > sum (map * listofpayoffs strategyfrequencies) ] payoffmatrix end to updatepayoff run payofftouse end to playwithonerdagent let mate oneof other players set payoff item ([strategy] of mate) (item strategy payoffmatrix) end to usestrategyexpectedpayoff set payoff item strategy strategyexpectedpayoffs end ;;;;;;;;;;;;;;;;;;;;;;;;; ;;; UPDATE STRATEGIES ;;; ;;;;;;;;;;;;;;;;;;;;;;;;; to updatestrategyafterrevision ifelse randomfloat 1 < noise [ set strategyafterrevision (random nofstrategies) ] [ let observedplayer oneof other players if ([payoff] of observedplayer) > payoff [ set strategyafterrevision ([strategy] of observedplayer) ] ] end to updatestrategy set strategy strategyafterrevision end ;;;;;;;;;;;;;;;;;;;; ;;; UPDATE GRAPH ;;; ;;;;;;;;;;;;;;;;;;;; to updategraph ; let strategynumbers (range nofstrategies) ; let strategyfrequencies map [ n > ; count players with [strategy = n] / nofplayers ; ] strategynumbers setcurrentplot "Strategy Distribution" let bar 1 foreach strategynumbers [ n > setcurrentplotpen (word n) plotxy ticks bar set bar (bar  (item n strategyfrequencies)) ] setplotyrange 0 1 end
5. Extension II. Implementation of different decision rules
In our second extension, we will implement the following procedures, one for each decision rule:
 to imitateifbetterrule
 to imitativepairwisedifferencerule
 to imitativelinearattractionrule
 to imitativelineardissatisfactionrule
 to directbestrule
 to directpairwisedifferencerule
 to directpositiveproportionalmrule
Note that we have chosen the name of the procedures to match the possible values of parameter decisionrule (plus the suffix “rule”).
5.1. Skeleton of the code
The main procedure we have to modify is to updatestrategyafterrevision, which is where the different decision rules are called from. Figure 3 below shows the skeleton of the new implementation of procedure to updatestrategyafterrevision.
5.2. Interface design
The current interface looks as shown in figure 2. The only change we have to make to the interface is to add the following parameters:
 decisionrule. We can add it as a chooser with possible values “imitateifbetter“, “imitativepairwisedifference“, “imitativelinearattraction“, “imitativelineardissatisfaction“, “directbest”, “directpairwisedifference”, and “directpositiveproportionalm”.
 m. We can add it as a slider (with minimum = 0 and increment = 0.1).
The new interface should look as figure 4 below.
5.3. Code
5.3.1. Global variables
For some decision rules, agents will need access to the minimum and the maximum possible payoffs, and to the maximum possible payoff difference. For this reason, it makes sense to define these as global variables.
globals [ payoffmatrix nofstrategies nofplayers strategynumbers strategyfrequencies strategyexpectedpayoffs minofpayoffmatrix ;; <= new global variable maxofpayoffmatrix ;; <= new global variable maxpayoffdifference ;; <= new global variable ]
Since the value of these global variables will not change over the course of a simulation run, we should set them at a setup procedure. The most appropriate place is at procedure to setuppayoffs.
to setuppayoffs set payoffmatrix readfromstring payoffs set nofstrategies length payoffmatrix set strategynumbers range nofstrategies ;; new lines below set minofpayoffmatrix min reduce sentence payoffmatrix set maxofpayoffmatrix max reduce sentence payoffmatrix set maxpayoffdifference (maxofpayoffmatrix  minofpayoffmatrix) end
5.3.2. Implementation of to updatestrategyafterrevision
Let us start with the implementation of procedure to updatestrategyafterrevision. Given the names we have chosen for the procedures for the different decision rules (see skeleton in Figure 3), can you venture a simple implementation for our new procedure to updatestrategyafterrevision using primitive run?
Implementation of procedure to updatestrategyafterrevision
to updatestrategyafterrevision ifelse randomfloat 1 < noise [ set strategyafterrevision (random nofstrategies) ] [ run (word decisionrule "rule") ] end
Now we just have to implement the procedures for each of the seven decision rules. If you have managed to follow this book up until here, you are most likely ready to implement them by yourself!
5.3.3. Imitative decision rules
Implementation of to imitateifbetterrule
This is the decision rule we had implemented in our baseline model nxnimitateifbetternoise, so we can just copy the code.
Implementation of procedure to imitateifbetterrule
to imitateifbetterrule let observedplayer oneof other players if ([payoff] of observedplayer) > payoff [ set strategyafterrevision ([strategy] of observedplayer) ] end
Implementation of to imitativepairwisedifferencerule
To implement this rule, the code we wrote in chapter III4 will be very useful.
Implementation of procedure to imitativepairwisedifferencerule
to imitativepairwisedifferencerule let observedplayer oneof other players ;; compute difference in payoffs let payoffdiff ([payoff] of observedplayer  payoff) set strategyafterrevision ifelsevalue (randomfloat 1 < (payoffdiff / maxpayoffdifference)) [ [strategy] of observedplayer ] [ strategy ] ;; If your strategy is the better, payoffdiff is negative, ;; so you are going to stick with it. ;; If it's not, you switch with probability ;; (payoffdiff / maxpayoffdifference) end
Implementation of to imitativelinearattractionrule
Can you try to implement this rule by yourself?
Implementation of procedure to imitativelinearattractionrule
to imitativelinearattractionrule let observedplayer oneof players set strategyafterrevision ifelsevalue (randomfloat 1 < ([payoff] of observedplayer  minofpayoffmatrix) / ( maxofpayoffmatrix  minofpayoffmatrix) ) [ [strategy] of observedplayer ] [ strategy ] end
Implementation of to imitativelineardissatisfactionrule
The code for this rule is similar to the previous one. Can you implement it?
Implementation of procedure to imitativelineardissatisfactionrule
to imitativelineardissatisfactionrule let observedplayer oneof players set strategyafterrevision ifelsevalue (randomfloat 1 < (maxofpayoffmatrix  payoff) / (maxofpayoffmatrix  minofpayoffmatrix) ) [ [strategy] of observedplayer ] [ strategy ] end
5.3.4. Direct decision rules
In this section, we implement three direct decisions rules: “directbest”, “directpairwisedifference”, and “directpositiveproportionalm”. In direct decision rules, agents consider different candidate strategies, and choose one of them based on their assigned payoffs. Thus, the first thing we should do is to implement a procedure that assigns payoffs to strategies.
Assigning payoffs to strategies
The payoff assigned to each strategy is determined by parameter payofftouse:
 If payofftouse = “playwithonerdagent“, the payoff assigned to strategy will be the payoff resulting from trying out strategy against one agent chosen at random from the population. Importantly, according to the description of the model, every time a strategy is tested by a reviser, it should be tested against a newly (randomly) drawn agent.
 If payofftouse = “usestrategyexpectedpayoff“, the payoff assigned to strategy will be strategy s expected payoff.
Our goal now is to implement a reporter named toreport payoffforstrategy [], which takes a strategy number as an input, and returns the appropriate payoff for strategy , taking into account the current value of parameter payofftouse. Bearing in mind that strategies’ expected payoffs are stored in global variable strategyexpectedpayoffs, can you implement toreport payoffforstrategy?
Implementation of reporter toreport payoffforstrategy
toreport payoffforstrategy [s] report ifelsevalue payofftouse = "playwithonerdagent" [item ([strategy] of oneof other players) (item s payoffmatrix)] [item s strategyexpectedpayoffs] end
Implementation of to directbestrule
To implement this rule, it is useful to create a list of twoitem lists (i.e., a list of pairs) that we can call pairsstrategypayoff. In this list, there will be one pair for each strategy. The first item in each pair will be the strategy number and the second item will be the payoff assigned to that strategy:
let pairsstrategypayoff (map [ [s] > list s (payoffforstrategy s) ] strategynumbers)
With this list in place, you can use primitive sortby to select one of the pairs with the highest payoff. Remember that ties should be resolved at random. Admittedly, implementing this procedure is not easy, so if you manage to do it, you can certainly give yourself a pat on the back!
Implementation of procedure to directbestrule
to directbestrule let pairsstrategypayoff (map [ [s] > list s (payoffforstrategy s) ] strategynumbers) ;; the following line ensures that ties ;; are resolved (uniformly) at random set pairsstrategypayoff shuffle pairsstrategypayoff let sortedlist sortby [ [l1 l2] > last l1 > last l2 ] pairsstrategypayoff set strategyafterrevision (first (first sortedlist)) end
Implementation of to directpairwisedifferencerule
To implement this rule, the code of procedure to imitativepairwisedifferencerule and primitive removeitem can be useful.
Implementation of procedure to directpairwisedifferencerule
to directpairwisedifferencerule let candidatestrategy oneof (removeitem strategy strategynumbers) ;; compute difference in payoffs let payoffdiff ((payoffforstrategy candidatestrategy)  payoff) set strategyafterrevision ifelsevalue (randomfloat 1 < (payoffdiff / maxpayoffdifference)) [ candidatestrategy ] [ strategy ] ;; If your strategy is the better, payoffdiff is negative, ;; so you are going to stick with it. ;; If it's not, you switch with probability ;; (payoffdiff / maxpayoffdifference) end
Implementation of to directpositiveproportionalmrule
To implement this rule, it is useful to have a look at the code of procedure to directbestrule, load the rnd extension, and use primitive rnd:weightedoneoflist.
Implementation of procedure to directpositiveproportionalmrule
to directpositiveproportionalmrule let pairsstrategypayoff (map [ [s] > list s ((payoffforstrategy s) ^ m) ] strategynumbers) set strategyafterrevision first rnd:weightedoneoflist pairsstrategypayoff [ [p] > last p ] end
To avoid errors when payoffs are negative, we will include procedure to checkpayoffsarenonnegative, just like we did in chapter III4 and chapter chapter IV4.
to checkpayoffsarenonnegative if minofpayoffmatrix < 0 [ usermessage (word "Since you are using decisionrule =\n" "imitativepositiveproportionalm,\n" "all elements in the payoff matrix\n" payoffs "\nshould be nonnegative numbers.") ] end
An appropriate place to call this procedure would be at the end of procedure to setuppayoffs, which would then be as follows:
to setuppayoffs set payoffmatrix readfromstring payoffs set nofstrategies length payoffmatrix set strategynumbers range nofstrategies ;; new lines below set minofpayoffmatrix min reduce sentence payoffmatrix set maxofpayoffmatrix max reduce sentence payoffmatrix set maxpayoffdifference (maxofpayoffmatrix  minofpayoffmatrix) if decisionrule = "directpositiveproportionalm" [ checkpayoffsarenonnegative ] end
5.4. Complete code of Extension II in the Code tab
Well done! We have now finished the second extension!
extensions [rnd] globals [ payoffmatrix nofstrategies nofplayers strategynumbers strategyfrequencies strategyexpectedpayoffs minofpayoffmatrix ;; <= new global variable maxofpayoffmatrix ;; <= new global variable maxpayoffdifference ;; <= new global variable ]
breed [players player]
playersown [ strategy strategyafterrevision payoff ] ;;;;;;;;;;;;; ;;; SETUP ;;; ;;;;;;;;;;;;; to setup clearall nodisplay setuppayoffs setupplayers setupgraph resetticks updatestrategyexpectedpayoffs updategraph end to setuppayoffs set payoffmatrix readfromstring payoffs set nofstrategies length payoffmatrix set strategynumbers range nofstrategies ;; new lines below set minofpayoffmatrix min reduce sentence payoffmatrix set maxofpayoffmatrix max reduce sentence payoffmatrix set maxpayoffdifference (maxofpayoffmatrix  minofpayoffmatrix) if decisionrule = "directpositiveproportionalm" [ checkpayoffsarenonnegative ] end to setupplayers let initialdistribution readfromstring nofplayersforeachstrategy if length initialdistribution != length payoffmatrix [ usermessage (word "The number of items in\n" "nofplayersforeachstrategy (i.e. " length initialdistribution "):\n" nofplayersforeachstrategy "\nshould be equal to the number of rows\n" "in the payoff matrix (i.e. " length payoffmatrix "):\n" payoffs ) ] let i 0 foreach initialdistribution [ j > createplayers j [ set payoff 0 set strategy i set strategyafterrevision strategy ] set i (i + 1) ] set nofplayers count players end to setupgraph setcurrentplot "Strategy Distribution" foreach (range nofstrategies) [ i > createtemporaryplotpen (word i) setplotpenmode 1 setplotpencolor 25 + 40 * i ] end ;;;;;;;;;; ;;; GO ;;; ;;;;;;;;;; to go ask players [updatepayoff] ask players [ if (randomfloat 1 < probrevision) [ updatestrategyafterrevision ] ] ask players [updatestrategy] tick updatestrategyexpectedpayoffs updategraph end ;;;;;;;;;;;;;;;;;;;;;; ;;; UPDATE PAYOFFS ;;; ;;;;;;;;;;;;;;;;;;;;;; to updatestrategyexpectedpayoffs set strategyfrequencies map [ n > count players with [strategy = n] / nofplayers ] strategynumbers set strategyexpectedpayoffs map [ listofpayoffs > sum (map * listofpayoffs strategyfrequencies) ] payoffmatrix end to updatepayoff run payofftouse end to playwithonerdagent let mate oneof other players set payoff item ([strategy] of mate) (item strategy payoffmatrix) end to usestrategyexpectedpayoff set payoff item strategy strategyexpectedpayoffs end ;;;;;;;;;;;;;;;;;;;;;;;;; ;;; UPDATE STRATEGIES ;;; ;;;;;;;;;;;;;;;;;;;;;;;;; to updatestrategyafterrevision ifelse randomfloat 1 < noise [ set strategyafterrevision (random nofstrategies) ] [ run (word decisionrule "rule") ] end to updatestrategy set strategy strategyafterrevision end ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; imitative decision rules ;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; to imitateifbetterrule let observedplayer oneof other players if ([payoff] of observedplayer) > payoff [ set strategyafterrevision ([strategy] of observedplayer) ] end to imitativepairwisedifferencerule let observedplayer oneof other players ;; compute difference in payoffs let payoffdiff ([payoff] of observedplayer  payoff) set strategyafterrevision ifelsevalue (randomfloat 1 < (payoffdiff / maxpayoffdifference)) [ [strategy] of observedplayer ] [ strategy ] ;; If your strategy is the better, payoffdiff is negative, ;; so you are going to stick with it. ;; If it's not, you switch with probability ;; (payoffdiff / maxpayoffdifference) end to imitativelinearattractionrule let observedplayer oneof players set strategyafterrevision ifelsevalue (randomfloat 1 < ([payoff] of observedplayer  minofpayoffmatrix) / ( maxofpayoffmatrix  minofpayoffmatrix) ) [ [strategy] of observedplayer ] [ strategy ] end to imitativelineardissatisfactionrule let observedplayer oneof players set strategyafterrevision ifelsevalue (randomfloat 1 < (maxofpayoffmatrix  payoff) / (maxofpayoffmatrix  minofpayoffmatrix) ) [ [strategy] of observedplayer ] [ strategy ] end ;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; direct decision rules ;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;; toreport payoffforstrategy [s] report ifelsevalue payofftouse = "playwithonerdagent" [item ([strategy] of oneof other players) (item s payoffmatrix)] [item s strategyexpectedpayoffs] end to directbestrule let pairsstrategypayoff (map [ [s] > list s (payoffforstrategy s) ] strategynumbers) ;; the following line ensures that ties ;; are resolved (uniformly) at random set pairsstrategypayoff shuffle pairsstrategypayoff let sortedlist sortby [ [l1 l2] > last l1 > last l2 ] pairsstrategypayoff set strategyafterrevision (first (first sortedlist)) end to directpairwisedifferencerule let candidatestrategy oneof (removeitem strategy strategynumbers) ;; compute difference in payoffs let payoffdiff ((payoffforstrategy candidatestrategy)  payoff) set strategyafterrevision ifelsevalue (randomfloat 1 < (payoffdiff / maxpayoffdifference)) [ candidatestrategy ] [ strategy ] ;; If your strategy is the better, payoffdiff is negative, ;; so you are going to stick with it. ;; If it's not, you switch with probability ;; (payoffdiff / maxpayoffdifference) end
to directpositiveproportionalmrule let pairsstrategypayoff (map [ [s] > list s ((payoffforstrategy s) ^ m) ] strategynumbers)
set strategyafterrevision first rnd:weightedoneoflist pairsstrategypayoff [ [p] > last p ] end
;;;;;;;;;;;;;;;;;;;; ;;; UPDATE GRAPH ;;; ;;;;;;;;;;;;;;;;;;;;
to updategraph setcurrentplot "Strategy Distribution" let bar 1 foreach strategynumbers [ n > setcurrentplotpen (word n) plotxy ticks bar set bar (bar  (item n strategyfrequencies)) ] setplotyrange 0 1 end
;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; SUPPORTING PROCEDURES ;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
to checkpayoffsarenonnegative if minofpayoffmatrix < 0 [ usermessage (word "Since you are using decisionrule =\n" "imitativepositiveproportionalm,\n" "all elements in the payoff matrix\n" payoffs "\nshould be nonnegative numbers.") ] end
6. Exercises
You can use the following link to download the complete NetLogo model: nxngamesinwellmixedpopulations.nlogo.
Exercise 1. Implement a decision rule that you can call “imitatethebest”. This is the imitate the best neighbor rule adapted to wellmixed populations. Under this rule, the revising agent copies the strategy of the individual with the greatest payoff. Resolve ties uniformly at random.
Exercise 2. Implement a decision rule that you can call “imitativepositiveproportionalm“. This is the imitativepositiveproportionalm rule implemented in chapter III4, adapted to wellmixed populations. Under this decision rule, the revising agent selects one individual at random, with probabilities proportional to payoffs raised to parameter m, and copies her strategy. Please do not allow for negative payoffs when using this rule.
Exercise 3. Implement a decision rule that you can call “imitativelogitm“. Under this decision rule, the revising agent selects one individual at random, with probabilities proportional to (where denotes agent ‘s payoff), and copies her strategy.
Exercise 4. Implement a decision rule that you can call “Fermim“. This is the Fermim rule implemented in chapter III4, adapted to wellmixed populations. Under this rule, the revising agent looks at another individual at random and copies her strategy with probability , where denotes agent ‘s payoff and .
Exercise 5. Implement a decision rule that you can call “directlogitm”. Under this rule, the revising agent selects one strategy at random, with probabilities proportional to (where denotes strategy ‘s payoff), and adopts it.
Exercise 6. In this model, we run procedure to updatestrategyexpectedpayoffs at the end of to go, as shown below:
to go ask players [updatepayoff] ask players [ if (randomfloat 1 < probrevision) [ updatestrategyafterrevision ] ] ask players [updatestrategy] tick updatestrategyexpectedpayoffs ;; <= current position updategraph end
Since procedure to updatestrategyexpectedpayoffs updates the expected payoffs, and these are used by agents in procedure to updatepayoffs, it may seem more natural to run procedure to updatestrategyexpectedpayoffs just before asking players to update their payoff, as shown below:
to go updatestrategyexpectedpayoffs ;; <= new position ask players [updatepayoff] ask players [ if (randomfloat 1 < probrevision) [ updatestrategyafterrevision ] ] ask players [updatestrategy] tick updategraph end
What would change if we did this?
Hint
You may compare the two alternatives in a setting where agents play RockPaperScissors (as in figure 2), payofftouse = “usestrategyexpectedpayoff“, probrevision = 1, noise = 0, and decisionrule = “directbest“.
 In chapter II4 we were able to implement a more efficient version of this model, but the speed boost came at the expense of making our code less readable. Here we want to focus on code readability, so we start with the most natural implementation of the model, i.e. nxnimitateifbetternoise. Nonetheless, we can add the primitive nodisplay in procedure to setup, right after clearall, to improve the efficiency of the model at hardly any cost. ↵
 There are decision rules that are neither imitative nor direct. See, e.g., Lahkar and Sandholm (2008). ↵