1. Goal

Our goal in this section is to extend the model we have created in the previous section by adding two features that are crucial to assess the impact of space on evolutionary models:

• The possibility to model different types of neighborhoods of arbitrary size. Besides Moore neighborhoods, we will implement Von Neumann neighborhoods, and both of them of any size.
• The possibility to model other decision rules besides the imitate the best neighbor rule. In particular, we will implement the imitative pairwise-difference rule, the imitate if better rule, the imitative positive proportional rule, and the Fermi rule.^[1]

2. Motivation. The impact of decision rules

In section 2.1 we explored the impact of different assumptions on the robustness of cooperation in spatial settings. However, one assumption we did not change was the decision rule (aka update rule). Roca et al. (2009a, 2009b) conducted an impressive simulation study of the effect of spatial structure in 2×2 games, and discovered that the decision rule can play a major role. In particular, they found that the imitate the best neighbor rule (which Roca et al. (2009a, 2009b) call “unconditional imitation”) favors cooperation in the Prisoner’s Dilemma more than any other decision rule they studied.

In what concerns the Prisoner’s Dilemma, the above results also prove that the promotion of cooperation in this game is not robust against changes in the update rule, because the beneficial effect of spatial lattices practically disappears for rules different from unconditional imitation, when seen in the wider scope of the ST plane. Roca et al. (2009b, p. 9)

In this section we are going to implement several decision rules. This will allow us to replicate Roca et al.’s (2009a, 2009b) results… and many others (see the proposed exercises).

3. Description of the model

The model we are going to develop here is a generalization of the model implemented in the previous section. In particular, we are going to add the following four parameters:

• neighborhood-type. This parameter is used to define the agents’ neighborhood (both for playing and for strategy updating).^[2] The parameter can have two possible values: “Moore” for a Moore neighborhood, or “Von Neumann” for a Von Neumann neighborhood.
• neighborhood-range. This parameter determines the range of the neighborhood. A patch’s Moore neighborhood of range r consists of the patches within Chebyshev distance r. A patch’s Von Neumann neighborhood of range r is composed of the patches within Manhattan distance r. Note that in our descriptions of decision rules below, we do not consider a patch to be a neighbor of itself unless otherwise stated.
• decision-rule. This parameter determines the decision rule that agents will follow. It will be implemented with a chooser, with five possible values:
  • “best-neighbor” (Nowak and May, 1992, 1993). This is the imitate the best neighbor rule, already implemented in our model.^[3]
  • “imitate-if-better-all-nbrs“. This is the imitate-if-better rule (Izquierdo and Izquierdo, 2013; Loginov, 2021) adapted to networks, where agents play with all their neighbors. Under this rule, the revising agent looks at a random neighbor and copies her strategy if and only if the observed neighbor’s average payoff is higher than the revising agent’s average payoff.
  • “imitative-pairwise-difference” (Hauert 2002,^[4] 2006). This is the imitative pairwise-difference rule we saw in section 0.1 adapted to networks. Under this rule, the revising agent looks at a random neighbor and considers copying her strategy only if the observed neighbor’s average payoff is higher than the revising agent’s average payoff; in that case, the revising agent switches with probability proportional to the payoff difference.^[5]
  • “imitative-positive-proportional-m” (Nowak et al., 1994a, b). Under this decision rule, the revising agent copies the strategy of one of her neighbors (including herself, in this case) with probability proportional to their total payoff raised to parameter m.^[6] Parameter m controls the intensity of selection (see below). We do not allow for negative payoffs when using this rule.^[7]
  • “Fermi-m” (Szabó and Tőke, 1998; Traulsen and Hauert, 2009; Roca et al. 2009a, 2009b; Perc and Szolnoki, 2010, Adami et al., 2016). Under this decision rule, the revising agent looks at one of her neighbors at random and copies her strategy with probability , where   denotes agent ‘s average payoff and .^[8] Parameter m controls the intensity of selection (see below).
• m. This is a parameter that controls the intensity of selection in decision rules imitative-positive-proportional-m and Fermi-m (see above). High values of m imply that selection is strongly based on payoffs, i.e. the agents with the highest payoffs will be chosen with very high probability. Lower values of m mean that the sensitivity of selection to payoffs is weak (e.g. if m = 0, selection among agents is random, i.e. independent of their payoffs).

Everything else stays as described in the previous section.

 ^CODE  4. Interface design

We depart from the model we developed in the previous section (so if you want to preserve it, now is a good time to duplicate it).

In the new interface (see figure 1 above), we have to add:

• One chooser for new parameter neighborhood-type (with possible values “Moore” and “Von Neumann“), and a slider for parameter neighborhood-range (with minimum = 1 and increment = 1).
• One chooser for new parameter decision-rule (with possible values “best-neighbor“, “imitate-if-better-all-nbrs“, “imitative-pairwise-difference“, “imitative-positive-proportional-m” and “Fermi-m“), and a slider for parameter m (with minimum = 0 and increment = 0.1).

We have also added a button (labeled show player’s neighborhood) to see any patch’s neighborhood on the 2D view and another button (labeled clear) to clear the display of the neighborhood.

 ^CODE  5. Code

5.1. Skeleton of the code

The skeleton of the code for procedures to setup and to go is the same as in the previous model. In this section we will modify mainly the following two procedures:

• procedure to setup-players, to set the players’ neighborhoods according to parameters neighborhood-type and neighborhood-range (see figure 2).

• procedure to update-strategy-after-revision, to run the decision rule indicated in parameter decision-rule (see figure 3).
  

  

5.2. Extension I. Implementation of different neighborhoods

Recall that patches have the following two individually owned variables:

• my-coplayers, which contains the set of agents with whom the patch plays the game (and is affected by parameter self-matching?), and
• my-nbrs-and-me, which is the set of agents the player considers when revising its strategy.

These two agentsets are the same if self-matching? is true, and differ only in the focal patch if self-matching? is false. We will keep this distinction for any type of neighborhood.

To implement the different neighborhoods, primitive at-points will be very useful. This primitive reports a subset of a given agentset that includes only the agents on the patches at the given coordinates (relative to the calling agent). As an example, the following code gives a patch’s Von Neumann neighborhood of range 1 (including the patch itself):

patches at-points [[-1 0] [0 -1] [0 0] [0 1] [1 0]]

And the following code gives a patch’s Moore neighborhood of range 1 (including the patch itself):

patches at-points [[-1 -1] [-1 0] [-1 1] [0 -1] [0 0] [0 1] [1 -1] [1 0] [1 1]]

Thus, the key is to implement procedures that report the appropriate lists of relative coordinates.^[9] Let’s do this!

Implementation of relative coordinates for Moore neighborhoods

Our goal here is to implement a procedure that reports the appropriate list of relative coordinates for Moore neighborhoods, for any given range and letting the user choose whether the list should include the item [0 0] or not. Let us call this reporter to-report moore-offsets. Note that this reporter takes two inputs, which we can call r (for range) and include-center?.

to-report moore-offsets [r include-center?]
   ;; code to be written
end

Below we provide some examples of what reporter to-report moore-offsets should produce:

• If r = 1 and include-center? is true:

  [[-1 -1] [-1 0] [-1 1] [0 -1] [0 0] [0 1] [1 -1] [1 0] [1 1]]

• If r = 1 and include-center? is false:

  [[-1 -1] [-1 0] [-1 1] [0 -1] [0 1] [1 -1] [1 0] [1 1]]

• If r = 2 and include-center? is true:

  [[-2 -2] [-2 -1] [-2 0] [-2 1] [-2 2] [-1 -2] [-1 -1] [-1 0] [-1 1] [-1 2] [0 -2] [0 -1] [0 0] [0 1] [0 2] [1 -2] [1 -1] [1 0] [1 1] [1 2] [2 -2] [2 -1] [2 0] [2 1] [2 2]]

Note that the effect of input include-center? is just to include or exclude [0 0] from the output list, so we can worry about that at the end of the implementation. To produce the list of Moore offsets for range r, we can start building the list l = [ –r, –r+1, …, r-1, r ] and then build 2-item lists with each element of the list l together with each element of the same list l. Thus, let us build the list l = [ –r, –r+1, …, r-1, r ]:

let l (range (- r) (r + 1))

To build 2-items lists with first element el1 and second element each of the elements of list l, we can use primitive map as follows:

map [el2 -> list el1 el2] l

And now we should make el1 be each of the elements of list l. For this we can use primitive map again:

map [ el1 -> map [el2 -> list el1 el2] l] l

The only problem now is that we have several nested lists. For example, the code above for l = [ -1, 0, 1 ] produces:

[[[-1 -1] [-1 0] [-1 1]] [[0 -1] [0 0] [0 1]] [[1 -1] [1 0] [1 1]]]

Note that the outmost list contains three sublists, each of which contains three 2-item lists. We can get rid of the extra lists using primitives reduce and sentence:

let result reduce sentence map [ el1 -> map [el2 -> list el1 el2] l] l

The code above creates the list of required offsets, including [0 0]. Now we just have to remove [0 0] if and only if input include-center? is false. Thus, the final code for reporter to-report moore-offsets is as follows:

to-report moore-offsets [r include-center?]
  let l (range (- r) (r + 1))
  let result reduce sentence map [ el1 -> map [el2 -> list el1 el2] l] l
  report ifelse-value include-center?
    [ result ]
    [ remove [0 0] result ]
end

Implementation of relative coordinates for Von Neumann neighborhoods

Our goal here is to implement a procedure that reports the appropriate list of relative coordinates for Von Neumann neighborhoods. Let us call this reporter to-report von-neumann-offsets and let us call its inputs r (for range) and include-center?, just like before.

to-report von-neumann-offsets [r include-center?]
   ;; code to be written
end

Note that, for any given r and include-center?, the list of offsets for a Von Neumann neighborhood is a subset of the list of offsets for the corresponding Moore neighborhood. In particular, the list of Von Neumann offsets is composed of the elements [el1 el2] in the list of Moore offsets that satisfy the condition |el1| + |el2| ≤ r . Thus, we can create the corresponding list of Moore offsets and filter those coordinates that satisfy the condition using primitive filter. Do you want to give it a try?

to-report von-neumann-offsets [r include-center?]
  let moore-list (moore-offsets r include-center?)
  report filter [l -> abs first l + abs last l <= r] moore-list
end

Putting everything together

Now that we have implemented to-report moore-offsets and to-report von-neumann-offsets, we can use them in procedure to setup-players to create the right neighborhood for each patch. The only lines we have to modify are the ones highlighted in bold below:

to setup-players
  let initial-distribution read-from-string n-of-players-for-each-strategy
  if length initial-distribution != length payoff-matrix [
    user-message (word "The number of items in\n"
      "n-of-players-for-each-strategy (i.e. "
      length initial-distribution "):\n" n-of-players-for-each-strategy
      "\nshould be equal to the number of rows\n"
      "in the payoff matrix (i.e. "
      length payoff-matrix "):\n"
      payoffs
    )
  ]

  if sum initial-distribution != count patches [
    user-message (word "The total number of agents in\n"
      "n-of-agents-for-each-strategy (i.e. "
      sum initial-distribution "):\n" n-of-players-for-each-strategy
      "\nshould be equal to the number of patches (i.e. "
      count patches ")"
    )
  ]

  ask patches [set strategy false]
  let i 0
  let offsets ifelse-value (neighborhood-type = "Von Neumann")
        [ von-neumann-offsets neighborhood-range self-matching? ]
        [ moore-offsets neighborhood-range self-matching? ]

  foreach initial-distribution [ j ->
    ask n-of j (patches with [strategy = false]) [
      set payoff 0
      set strategy i 
      set strategy-after-revision strategy
      set my-coplayers patches at-points offsets
      set n-of-my-coplayers (count my-coplayers)
      set my-nbrs-and-me (patch-set my-coplayers self)
    ]
    set i (i + 1)
  ]

  set n-of-players count patches
end

Note that variable my-coplayers may or may not contain the patch itself (depending on the value of parameter self-matching?), but variable my-nbrs-and-me will always contain it, since we are setting its value to (patch-set my-coplayers self). This is perfectly fine even if my-coplayers already contains the patch itself, since agentsets do not contain duplicates. Adding an agent a to an agentset that already contains that agent a has no effect whatsoever.^[10]

A nice final touch

Finally, we are going to write some code to let the user see the neighborhood of any patch in the 2D view by clicking on it. This is just for displaying purposes, so feel free to skip this if you’re not really interested. Let us start by implementing a new procedure called to show-neighborhood as follows:

to show-neighborhood
  if mouse-down? [
    ask patch mouse-xcor mouse-ycor [
      ask my-coplayers [set pcolor white]
    ]
  ]
  display
end

You may want to read the documentation for primitives mouse-down?, mouse-xcor and mouse-ycor. Basically, this procedure paints in white the patches contained in the variable my-coplayers of the patch you click with your mouse. However, for this to work, the procedure must be running all the time. To do this, we can run this procedure within the button labeled show player’s neighborhood in the interface, and make this button be a “forever” button.

with-local-randomness [show-neighborhood]

The use of primitive with-local-randomness guarantees that this piece of code does not interfere with the generation of pseudorandom numbers for the rest of the model.

with-local-randomness [ ask patches [update-color] ]

Figure 4 shows a patch’s Von Neumann neighborhood of range 5 (excluding the patch itself) painted in white.

5.3. Extension II. Implementation of different decision rules

The implementation of the decision rule takes place in procedure to update-strategy-after-revision. Note that the effect of noise is the same regardless of the decision rule, so we can deal with noise in a unified way, regardless of which decision rule will be executed. One possible way of doing this is as follows:

to update-strategy-after-revision
  ifelse (random-float 1 < noise)
    [ set strategy-after-revision random n-of-strategies ]
    [ 
      ;; code to set strategy-after-revision 
      ;; using the decision rule indicated by the user
      ;; through parameter decision-rule
    ]
end

To make the implementation of decision rules elegant and modular, we should implement a different procedure for each decision rule. Let us call these procedures: best-neighbor-rule, imitate-if-better-all-nbrs-rule, imitative-pairwise-difference-rule, imitative-positive-proportional-m-rule and Fermi-m-rule. With these procedures in place, the code for procedure to update-strategy-after-revision would just look as follows:

to update-strategy-after-revision
  ifelse (random-float 1 < noise)
    [ set strategy-after-revision random n-of-strategies ]
    [ (ifelse
        decision-rule = "best-neighbor"
                  [ best-neighbor-rule ]
        decision-rule = "imitate-if-better-all-nbrs" 
                  [ imitate-if-better-all-nbrs-rule ]
        decision-rule = "imitative-pairwise-difference"
                  [ imitative-pairwise-difference-rule ]
        decision-rule = "imitative-positive-proportional-m"
                  [ imitative-positive-proportional-m-rule ]
        decision-rule = "Fermi-m"
                  [ Fermi-m-rule ]
      )
    ]
end

Note that primitive ifelse can work with multiple conditions, just like switch statements in other programming languages.^[11]

Now we just have to implement the procedures for each of the four decision rules. Do you want to give it a try? The first one is not very difficult.

to best-neighbor-rule
  set strategy-after-revision 
      [strategy] of one-of my-nbrs-and-me with-max [payoff]
end

Implementing procedure to imitate-if-better-all-nbrs-rule is slightly more difficult, but we believe you can do it. Remember that players consider average payoffs, rather than total payoffs.

to imitate-if-better-all-nbrs-rule
  let observed-player one-of other my-coplayers

  if ([payoff / n-of-my-coplayers] of observed-player) > 
      (payoff / n-of-my-coplayers) [
    set strategy-after-revision ([strategy] of observed-player)
  ]
end

The implementation of procedure to imitative-pairwise-difference-rule is significantly more difficult than the previous one, but if you have managed to read this book until here, you certainly have what it takes to do it!

to imitative-pairwise-difference-rule
  let observed-player one-of other my-coplayers

  ;; compute difference in average payoffs
  let payoff-diff ([payoff / n-of-my-coplayers] of observed-player   
                 - (payoff / n-of-my-coplayers))

  set strategy-after-revision   
      ifelse-value 
       (random-float 1 < (payoff-diff / max-payoff-difference))
         [ [strategy] of observed-player ]
         [ strategy ]
         ;; If your strategy is the better, payoff-diff is negative,
         ;; so you are going to stick with it.
         ;; If it's not, you switch with probability
         ;; (payoff-diff / max-payoff-difference)
end

globals [
  payoff-matrix
  n-of-strategies
  n-of-players
  max-payoff-difference     ;; <== New line
]

to setup-payoffs
  set payoff-matrix read-from-string payoffs
  set n-of-strategies length payoff-matrix

  ;;New lines below
  set max-payoff-difference
     (max-of-matrix payoff-matrix) - (min-of-matrix payoff-matrix)
end

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; SUPPORTING PROCEDURES ;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;;;;;;;;;;;;;;;;
;;; Matrices ;;;
;;;;;;;;;;;;;;;;

to-report max-of-matrix [m]
  report max reduce sentence m
end

to-report min-of-matrix [m]
  report min reduce sentence m
end

To implement procedure to imitative-positive-proportional-m-rule, there is an extension that comes bundled with NetLogo which will make our life much easier: extension rnd. To use it, you just have to add the following line at the beginning of your code

extensions [rnd]

Having loaded the rnd extension, you can use primitive rnd:weighted-one-of, which will be very handy. With this information, you may want to try to implement this short procedure yourself.

to imitative-positive-proportional-m-rule
  let chosen-nbr rnd:weighted-one-of my-nbrs-and-me [ payoff ^ m ]
  set strategy-after-revision [strategy] of chosen-nbr
end

to check-payoffs-are-non-negative
  if min reduce sentence payoff-matrix < 0 [
    user-message (word
      "Since you are using decision-rule = imitative-positive-proportional-m, all elements in the payoff matrix\n"
      payoffs
      "\nshould be non-negative numbers.")
  ]
end

to setup-payoffs
  set payoff-matrix read-from-string payoffs
  set n-of-strategies length payoff-matrix

  ;; New lines below
  set max-payoff-difference 
     (max-of-matrix payoff-matrix) - (min-of-matrix payoff-matrix)
  if decision-rule = "imitative-positive-proportional-m" 
     [ check-payoffs-are-non-negative ]
end

The implementation of procedure to Fermi-m-rule is not extremely hard after having seen the implementation of to imitative-pairwise-difference-rule, and realizing that .

to Fermi-m-rule
  let observed-player one-of other my-coplayers

  ;; compute difference in average payoffs 
  let payoff-diff ([payoff / n-of-my-coplayers] of observed-player   
                 - (payoff / n-of-my-coplayers))

  set strategy-after-revision   
    ifelse-value (random-float 1 < (1 / (1 + exp (- m * payoff-diff))))
      [ [strategy] of observed-player ]
      [ strategy ]
end

5.5. Complete code in the Code tab

The Code tab is ready! Congratulations! With this model you can rigorously explore the effect of space in 2-player games.

extensions [rnd]         ;; <== New line

globals [
  payoff-matrix
  n-of-strategies
  n-of-players
  max-payoff-difference  ;; <== New line
]

patches-own [
  strategy
  strategy-after-revision
  payoff
  my-nbrs-and-me
  my-coplayers
  n-of-my-coplayers
]

to setup
  clear-all
  setup-payoffs
  setup-players
  setup-graph
  reset-ticks
  update-graph
  ask patches [update-color]
end

to setup-payoffs
  set payoff-matrix read-from-string payoffs
  set n-of-strategies length payoff-matrix

  ;; New lines below
  set max-payoff-difference
     (max-of-matrix payoff-matrix) - (min-of-matrix payoff-matrix)
  if decision-rule = "imitative-positive-proportional-m" 
     [ check-payoffs-are-non-negative ]
end

to setup-players
  let initial-distribution read-from-string n-of-players-for-each-strategy
  if length initial-distribution != length payoff-matrix [
    user-message (word "The number of items in\n"
      "n-of-players-for-each-strategy (i.e. "
      length initial-distribution "):\n" n-of-players-for-each-strategy
      "\nshould be equal to the number of rows\n"
      "in the payoff matrix (i.e. "
      length payoff-matrix "):\n"
      payoffs
    )
  ]

  if sum initial-distribution != count patches [
    user-message (word "The total number of agents in\n"
      "n-of-agents-for-each-strategy (i.e. "
      sum initial-distribution "):\n" n-of-players-for-each-strategy
      "\nshould be equal to the number of patches (i.e. "
      count patches ")"
    )
  ]

  ask patches [set strategy false]
  let i 0
  let offsets ifelse-value (neighborhood-type = "Von Neumann")
        [ von-neumann-offsets neighborhood-range self-matching? ]
        [ moore-offsets neighborhood-range self-matching? ]

  foreach initial-distribution [ j ->
    ask n-of j (patches with [strategy = false]) [
      set payoff 0
      set strategy i 
      set strategy-after-revision strategy
      set my-coplayers patches at-points offsets
      set n-of-my-coplayers (count my-coplayers)
      set my-nbrs-and-me (patch-set my-coplayers self)
    ]
    set i (i + 1)
  ]

  set n-of-players count patches
end

to setup-graph
  set-current-plot "Strategy Distribution"
  foreach (range n-of-strategies) [ i ->
    create-temporary-plot-pen (word i)
    set-plot-pen-mode 1
    set-plot-pen-color 25 + 40 * i
  ]
end

to go
  ifelse synchronous-updating?
    [
      ask patches [ play ]
      ask patches [ update-strategy-after-revision ]
        ;; here we are not updating the agent's strategy yet
      ask patches [ update-strategy ]
        ;; now we update every agent's strategy at the same time
    ]
    [
      ask patches [
        play
        ask my-coplayers [ play ]
           ;; since your coplayers' strategies or
           ;; your coplayers' coplayers' strategies
           ;; could have changed since the last time
           ;; your coplayers played
        update-strategy-after-revision
        update-strategy
      ]
    ]
  tick
  update-graph
  ask patches [update-color]
end

to play
  let n-of-coplayers-with-strategy-? n-values n-of-strategies [ i ->
    count my-coplayers with [strategy = i] ]
  let my-payoffs (item strategy payoff-matrix)
  set payoff sum (map * my-payoffs n-of-coplayers-with-strategy-?)
end

to update-strategy-after-revision
  ifelse (random-float 1 < noise)
    [ set strategy-after-revision random n-of-strategies ]
    [ (ifelse
        decision-rule = "best-neighbor"
                  [ best-neighbor-rule ]
        decision-rule = "imitate-if-better-all-nbrs" 
                  [ imitate-if-better-all-nbrs-rule ]
        decision-rule = "imitative-pairwise-difference"
                  [ imitative-pairwise-difference-rule ]
        decision-rule = "imitative-positive-proportional-m"
                  [ imitative-positive-proportional-m-rule ]
        decision-rule = "Fermi-m"
                  [ Fermi-m-rule ]
      )
    ]
end

to best-neighbor-rule
  set strategy-after-revision 
      [strategy] of one-of my-nbrs-and-me with-max [payoff]
end

to imitate-if-better-all-nbrs-rule
  let observed-player one-of other my-coplayers

  if ([payoff / n-of-my-coplayers] of observed-player) > 
      (payoff / n-of-my-coplayers) [
    set strategy-after-revision ([strategy] of observed-player)
  ]
end

to imitative-pairwise-difference-rule let observed-player one-of other my-coplayers

  ;; compute difference in average payoffs
  let payoff-diff ([payoff / n-of-my-coplayers] of observed-player 
                 - (payoff / n-of-my-coplayers))

  set strategy-after-revision 
      ifelse-value (random-float 1 < (payoff-diff / max-payoff-difference))
        [ [strategy] of observed-player ]
        [ strategy ]
      ;; If your strategy is the better, payoff-diff is negative,
      ;; so you are going to stick with it.
      ;; If it's not, you switch with probability
      ;; (payoff-diff / max-payoff-difference)
end

to imitative-positive-proportional-m-rule
  let chosen-nbr rnd:weighted-one-of my-nbrs-and-me [ payoff ^ m ]
   ;; https://ccl.northwestern.edu/netlogo/docs/rnd.html#rnd:weighted-one-of
  set strategy-after-revision [strategy] of chosen-nbr
end

to Fermi-m-rule
  let observed-player one-of other my-coplayers

  ;; compute difference in average payoffs
  let payoff-diff ([payoff / n-of-my-coplayers] of observed-player 
                 - (payoff / n-of-my-coplayers))

  set strategy-after-revision 
    ifelse-value (random-float 1 < (1 / (1 + exp (- m * payoff-diff))))
      [ [strategy] of observed-player ]
      [ strategy ]
end

to update-strategy
  set strategy strategy-after-revision
end

to update-graph
  let strategy-numbers (range n-of-strategies)
  let strategy-frequencies map [ n ->
    count patches with [strategy = n] / n-of-players ] strategy-numbers

  set-current-plot "Strategy Distribution"
  let bar 1
  foreach strategy-numbers [ n ->
    set-current-plot-pen (word n)
    plotxy ticks bar
    set bar (bar - (item n strategy-frequencies))
  ]
  set-plot-y-range 0 1
end

to update-color
  set pcolor 25 + 40 * strategy
end

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; SUPPORTING PROCEDURES ;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

to-report moore-offsets [r include-center?]
  let l (range (- r) (r + 1))
  let result reduce sentence map [ el1 -> map [el2 -> list el1 el2] l] l
  report ifelse-value include-center?
    [ result ]
    [ remove [0 0] result ]
end

to-report von-neumann-offsets [r include-center?]
  let moore-list (moore-offsets r include-center?) 
  report filter [l -> abs first l + abs last l <= r] moore-list 
end

to show-neighborhood
  if mouse-down? [
    ask patch mouse-xcor mouse-ycor [
      ask my-coplayers [set pcolor white]
    ]
  ]
  display
end

to check-payoffs-are-non-negative
  if min reduce sentence payoff-matrix < 0 [
    user-message (word
      "Since you are using decision-rule = imitative-positive-proportional-m, all elements in the payoff matrix\n"
      payoffs
      "\nshould be non-negative numbers.")
  ]
end

;;;;;;;;;;;;;;;;
;;; Matrices ;;;
;;;;;;;;;;;;;;;;

to-report max-of-matrix [matrix]
  report max reduce sentence matrix
end

to-report min-of-matrix [matrix]
  report min reduce sentence matrix
end

6. Sample runs

6.1. Decision rules

Now that we have implemented the model, we can explore the impact of the decision rule on the promotion of cooperation in the spatially embedded Prisoner’s Dilemma. Let us explore this question using the parameter values shown in figure 1 above. We start with the imitate the best neighbor rule. The simulation below shows a representative run.

As you can see in the video above, with the imitate the best neighbor rule, high levels of cooperation are achieved (i.e. about 90% of the patches cooperate in the long run). But, how robust is this result to changes in the decision rule? To explore this question, let us run the same simulation with the other four decision rules we have implemented. The simulation below shows a representative run with the imitate-if-better-all-nbrs rule.

As you can see in the video, with the imitate-if-better-all-nbrs rule there is no cooperation at all. Defection takes over in about forty ticks. Let us see what happens with the imitative-pairwise-difference rule.

With the imitative-pairwise-difference rule, again, there is no cooperation at all in the long run. Let us now try the imitative-positive-proportional-m rule with m = 1:^[12]

We do not observe any cooperation at all with the imitative-positive-proportional-m (with m = 1) either. Finally, let us try the Fermi-m rule with m = 1 (or any other positive value, for that matter):^[13]

And we observe universal defection again! Clearly, as Roca et al. (2009a, 2009b) point out, the imitate the best neighbor rule seems to favor cooperation in the Prisoner’s Dilemma more than other decision rules.

6.2. Neighborhoods

What about if we change the type of neighborhood? You can check that the simulations shown in the videos above do not change qualitatively if we use Von Neumann neighborhoods rather than Moore neighborhoods. However, what does make a difference is the size of the neighborhood. Note that if the size is so large that agents’ neighborhoods encompass the whole population, we have a global interaction model (i.e. a model without population structure). Thus, the size of the neighborhood allows us to study the effect of population structure in a gradual manner. Having seen this, we can predict that, for the imitate the best neighbor rule, if we increase the size of the neighborhood enough, at some point cooperation will disappear. As an exercise, try to find the minimum neighborhood-range (for both types of neighborhood) at which cooperation cannot be sustained anymore, with the other parameter values as shown in figure 1.

6.3. Discussion

We would like to conclude this section emphasizing that the influence of population structure on evolutionary dynamics generally depends on many factors that may seem insignificant at first sight, and whose effects interact in complex ways. This may sound discouraging, since it suggests that a simple general theory of the influence of population structure on evolutionary dynamics cannot be derived. On a more positive note, it also highlights the importance of the skills you are learning with this book. Without the aid of computer simulation, it seems impossible to explore this type of questions. But using computer simulation we can gain some understanding of the complexity and the beauty of these apparently simple –yet surprisingly intricate– systems. The following quote from Roca et al. (2009b) nicely summarizes this view.

To conclude, we must recognize the strong dependence on details of evolutionary games on spatial networks. As a consequence, it does not seem plausible to expect general laws that could be applied in a wide range of practical settings. On the contrary, a close modeling including the kind of game, the evolutionary dynamics and the population structure of the concrete problem seems mandatory to reach sound and compelling conclusions. With no doubt this is an enormous challenge, but we believe that this is one of the most promising paths that the community working in the field can explore. Roca et al. (2009b, pp. 14-15) 

7. Exercises

You can use the following link to download the complete NetLogo model: nxn-imitate-best-nbr-extended.

Exercise 1. How can we parameterize our model to replicate the results shown in figures 1b and 1d of Hauert and Doebeli (2004)?

Exercise 2. How can we parameterize our model to replicate the results shown in figures 1, 2, 6 and 7 of Nowak et al. (1994a)?^[14]

Exercise 3. How can we parameterize our model to replicate the results shown in figure 1 of Szabó and Tőke (1998)? Note that Szabó and Tőke (1998) use total payoffs and we use average payoffs in our Fermi-m rule, but a clever parameterization can save you any programming efforts.

Exercise 4. How can we parameterize our model to replicate the results shown in figure 11 of Nowak and May (1993)?

Exercise 5. To appreciate the impact that neighborhood-type may have, run a simulation parameterized as in figure 1 above, but with decision-rule = “Fermi-m”, m = 10, and self-matching? = true. Compare the results obtained using neighborhood-type = “Moore” and neighborhood-type = “Von Neumann”. What do you observe?

 ^CODE  Exercise 6. The following block of code (in procedure to update-strategy-after-revision) can be replaced by one simple line using run (a primitive that can take a string containing the name of a command as an input, and it runs the command).

[ (ifelse
    decision-rule = "best-neighbor"
              [ best-neighbor-rule ]
    decision-rule = "imitate-if-better-all-nbrs" 
              [ imitate-if-better-all-nbrs-rule ]
    decision-rule = "imitative-pairwise-difference"
              [ imitative-pairwise-difference-rule ]
    decision-rule = "imitative-positive-proportional-m"
              [ imitative-positive-proportional-m-rule ]
    decision-rule = "Fermi-m"
              [ Fermi-m-rule ]
  )
]

Can you come up with the simple line?