2. Spatial interactions on a grid

2.0. Spatial chaos in the Prisoner’s Dilemma

1. Goal

The goal of this chapter is to learn how to build agent-based models with spatial structure. In models with spatial structure, agents do not interact with all other agents with the same probability, but they interact preferentially with those who are nearby.[1]

More generally, populations where some pairs of agents are more likely to interact with each other than others are called structured populations. If, on the other hand, all members of a population are equally likely to interact with each other, the population is said to be well-mixed. Generally, the dynamics of an evolutionary process in a well-mixed population can be very different from that in a structured population. In social dilemmas in particular, population structure plays a crucial role (Gotts et al. (2003), Hauert (2002, 2006), Roca et al. (2009a, 2009b)).[2]

2. Motivation. Cooperation in spatial settings

In the previous chapter, we saw that if agents play the Prisoner’s Dilemma in a well-mixed population, defection prevails. Here we want to explore whether adding spatial structure may affect that observation. Could cooperation be sustained if we removed the unrealistic assumption that all members of the population are equally likely to interact with each other? To shed some light on this question, in this section we will implement a model analyzed by Nowak and May (1992, 1993).

3. Description of the model

In this model, there is a population of agents arranged on a 2-dimensional lattice of “patches”. There is one agent in each patch. The size of the lattice, i.e. the number of patches in each of the two dimensions, can be set by the user. Each patch has eight neighboring patches (i.e. the eight cells which surround it), except for the patches at the boundary, which have five neighbors if they are on a side, or three neighbors if they are at one of the four corners.

Agents repeatedly play a symmetric 2-player 2-strategy game, where the two possible strategies are labeled C (for Cooperate) and D (for Defect). The payoffs of the game are determined using four parameters: CC-payoff, CD-payoff, DC-payoff, and DD-payoff, where XY-payoff denotes the payoff obtained by an X-player who meets a Y-player.

The initial percentage of C-players in the population is initial-%-of-C-players, and they are randomly distributed in the grid. From then onwards, the following sequence of events –which defines a tick– is repeatedly executed:

  1. Every agent plays the game with all his neighbors (once with each neighbor) and with himself (Moore neighborhood). The total payoff for the player is the sum of the payoffs in these encounters.
  2. All agents symultaneously revise their strategy according to the following rule:
    Consider the set of all your neighbors plus yourself; then adopt the strategy of one of the agents in this set who has obtained the greatest payoff. If there is more than one agent with the greatest payoff, choose one of them at random to imitate.

 CODE  4. Interface design

Interface design
Figure 1. Interface design.

To define each agent’s neighborhood, in this chapter we will use the 2-dimensional grid already built in NetLogo, often called “the world”. This will make our code simpler and the visualizations nicer.

The interface (see figure 1 above) includes:

  • The 2D view of the NetLogo world (i.e. the large black square in the interface), which is made up of patches. This view is already on the interface by default when creating a new NetLogo model.

    Choose the dimensions of the world by clicking on the “Settings…” button on the top bar, or by right-clicking on the 2D view and choosing Edit. A window will pop up, which allows you to choose the number of patches by setting the values of min-pxcor, max-pxcor, min-pycor and max-pycor. You can also determine the patches’ size in pixels, and whether the grid wraps horizontally, vertically, both or none (see Topology section). You can choose these parameters as in figure 2 below:

    Model settings
    Figure 2. Model settings.
  • Three buttons:
    1. One button named setup, which runs the procedure to setup.
    2. One button named go once, which runs the procedure to go.
    3. One button named go, which runs the procedure to go indefinitely.
    In the Code tab, write the procedures to setup and to go, without including any code inside for now. Then, create the buttons, just like we did in the previous chapter.
    Note that the interface in figure 1 has an extra button labeled one central D player. You may wish to include it now. The code that goes inside this button is proposed as Exercise 2.
  • Four sliders, to choose the payoffs for each possible outcome (CC, CD, DC, DD).
    Create the four sliders, with global variable names CC-payoff, CD-payoff, DC-payoff, and DD-payoff. Remember to choose a range, an interval and a default value for each of them. You can choose minimum 0, maximum 2 and increment 0.01.
  • A slider to let the user select the initial percentage of C-players.
    Create a slider for global variable initial-%-of-C-players. You can choose limit values 0 (as the minimum) and 100 (as the maximum), and an increment of 1.
  • A plot that will show the evolution of the number of agents playing each strategy.
    Create a plot and name it Strategy Distribution.

 CODE  5. Code

4.1. Global variables and individually-owned variables

We will not need any global variables besides those defined with the sliders in the interface.

Note that in this model there is a one-to-one correspondence between our immobile players and the patches they live in. Thus, there is no need to create any turtles (i.e. NetLogo mobile agents) in our model. We can work only with patches, and our code will be much simpler and readable.

Thus, we can make the built-in “patches” be the players, identifying each patch with one player. These patches already exist in NetLogo, making up the world, so we do not need to create them. Having said that, we do need to associate with each patch all the information that we want it to carry. This information will be:

  • Whether the patch is a C-player or D-player. For efficiency and code readability we can use a boolean variable to this end, which we can call C-player? and which will take the value true or false.
  • Whether the patch was a C-player or a D-player before it revised and updated its strategy. This is needed because we want to model synchronous updating, i.e. we want all agents to change their strategy at the same time. To do this, we need to keep in memory the strategy of every agent before the first agent switches strategy. For this purpose, we may use the boolean variable  C-player-last?.
  • The total payoff obtained by the patch playing with its neighbours. We can call this variable payoff.
  • For efficiency, it will also be useful to endow each patch with the set of neighbouring patches plus itself. The reason is that this set will be used many times, and it never changes, so it can be computed just once at the beginning and stored in memory. We will store this set in a variable named my-nbrs-and-me.
  • The following variable is also defined for efficiency reasons. Note that the payoff of a patch depends on the number of C-players and D-players in its set my-nbrs-and-me. To spare the operation of counting D-players, we can calculate it as the number of players in my-nbrs-and-me (which does not change in the whole simulation) minus the number of C-players. To this end, we can store the number of players in the set my-nbrs-and-me of each patch as an individually-owned variable that we naturally name n-of-my-nbrs-and-me.

Thus, this part of the code looks as follows:

patches-own  [
  C-player?
  C-player-last?
  payoff
  my-nbrs-and-me
  n-of-my-nbrs-and-me
]

4.2. Setup procedures

In the setup procedure we will:

  1. Clear everything up, so we initialize the model afresh, using the primitive clear-all:
    clear-all
  2. Set initial values for the variables that we have associated to each patch. We can set the payoff to 0,[3] and both C-player? and C-player-last? to false (later we will ask some patches to set these values to true). To set the value of my-nbrs-and-me, NetLogo primitives neighbors and patch-set are really handy.
    ask patches [
      set payoff 0
      set C-player? false
      set C-player-last? false
      set my-nbrs-and-me (patch-set neighbors self)
      set n-of-my-nbrs-and-me (count my-nbrs-and-me)
    ]
  3. Ask a certain number of randomly selected patches to be C-players. That number depends on the percentage initial-%-of-C-players chosen by the user and on the total number of patches, and it must be an integer, so we can calculate it as:
    round (initial-%-of-C-players * count patches / 100) 

    To randomly select a certain number of agents from an agentset (such as patches), we can use the primitive n-of (which reports another –usually smaller– agentset). Thus, the resulting instruction will be:

    ask n-of (round (initial-%-of-C-players * count patches / 100)) patches [
      set C-player? true
      set C-player-last? true
    ]
    
  4. Color patches according to the four possible combinations of values of C-player? and C-player-last?. The color of a patch is controled by the NetLogo built-in patch variable pcolor. A first (and correct) implementation of this task could look like:
    ask patches [
      ifelse C-player?
      [
        ifelse C-player-last?
        [set pcolor blue]
        [set pcolor lime]
      ]
      [
        ifelse C-player-last?
        [set pcolor yellow]
        [set pcolor red]
      ]
    ]
    

    However, the following implementation, which makes use of NetLogo primitive ifelse-value is more readable, as one can clearly see that the only thing we are doing is to set the patch’s pcolor.

    ask patches [
      set pcolor
        ifelse-value C-player?
         [ifelse-value C-player-last? [blue] [lime]]
         [ifelse-value C-player-last? [yellow] [red]]
    ]
  5. Reset the tick counter using reset-ticks.

Note that:

  • Points 2 and 3 above are about setting up the players, so, to keep our code nice and modular, we could group them into a new procedure called to setup-players. This will make our code more elegant, easier to understand, easier to debug and easier to extend, so let us do it!
  • The operation described in point 4 above will be conducted every tick, so we should create a separate procedure to this end that we can call to update-color, to be run by individual patches. Since this procedure is rather secondary (i.e. our model could run without this), we have a slight preference to place it at the end of our code, but feel free to do it as you like, since the order in which NetLogo procedures are written in the Code tab is immaterial.

Thus, the code up to this point should be as follows:

patches-own [
  C-player?
  C-player-last?
  payoff
  my-nbrs-and-me
  n-of-my-nbrs-and-me
]

to setup
  clear-all
  setup-players
  ask patches [update-color]
  reset-ticks
end

to setup-players
  ask patches [
    set payoff 0
    set C-player? false
    set C-player-last? false
    set my-nbrs-and-me (patch-set neighbors self)
    set n-of-my-nbrs-and-me (count my-nbrs-and-me)
  ]
  ask n-of (round (initial-%-of-C-players * count patches / 100)) patches [
    set C-player? true
    set C-player-last? true
  ]
end

to go
 
end

to update-color
  set pcolor
    ifelse-value C-player?
     [ifelse-value C-player-last? [blue] [lime]]
     [ifelse-value C-player-last? [yellow] [red]]
end

4.3. Go procedure

The procedure to go contains all the instructions that will be executed in every tick. In this particular model, we will ask each player (i.e. patch):

  1. To play with its neighbours in order to calculate its payoff. For modularity and clarity purposes, we should do this in a new procedure named to play.
  2. To store the current value of C-player? (i.e. the player’s current strategy) in the variable C-player-last?. In this way, the variable C-player-last? will keep the strategy with which the current payoff has been obtained, and we can update the value of C-player? without losing that information, which will be required by neighboring players when they update their strategy.
  3. To update its strategy (i.e. the value of C-player?). To keep our code nice and modular, we will do this in a separate new procedure called to update-strategy.
  4. To update its colour according to their new C-player? and C-player-last? values.

We should also mark the end of the round, or tick, after all players have updated their strategies, using the primitive tick, which increases the tick counter by one, and updates the graph on the interface. Thus, by now the code of procedure to go should look as follows:[4]

to go
  ask patches [play]
  ask patches [set C-player-last? C-player?]
  ask patches [update-strategy]
  tick
  ask patches [update-color]
end

4.4 Other procedures

to play

In procedure to play we want patches to calculate their payoff. This payoff will be the number of C-players in the set my-nbrs-and-me times the payoff obtained with a C-player, plus the number of D-players in the set times the payoff obtained with a D-player.

We will store the number of C-players in the set my-nbrs-and-me in a local variable that we can name n-of-C-players. The number can be computed as follows:

let n-of-C-players count my-nbrs-and-me with [C-player?]

Note that if the calculating patch is a C-player, the payoff obtained when playing with another C-player is CC-payoff, and if the calculating patch is a D-player, the payoff obtained when playing with with a C-player is DC-payoff. Thus, in general, the payoff obtained when playing with a C-player can then be obtained using the following code:

ifelse-value C-player? [CC-payoff] [DC-payoff]

Similarly, the payoff obtained when playing with a D-player is:

ifelse-value C-player? [CD-payoff] [DD-payoff]

Taking all this into account, we can implement procedure to play as follows:

to play
  let n-of-C-players count my-nbrs-and-me with [C-player?]
  set payoff n-of-C-players * ifelse-value C-player? [CC-payoff] [DC-payoff] +
       (n-of-my-nbrs-and-me - n-of-C-players) * ifelse-value C-player? [CD-payoff] [DD-payoff]
end

to update-strategy

In this procedure, which will be run by individual patches, we want the patch to adopt the strategy used by one of the patches with the maximum payoff in the set my-nbrs-and-me. To select one of these maximum-payoff patches, we may use primitives one-of and with-max as follows:

one-of (my-nbrs-and-me with-max [payoff])

Now remember that strategy updating in this model is synchronous, i.e. every player revises his strategy at the same time. Thus, we want each patch to adopt the strategy that was used by the selected maximum-payoff patch when it played the game, i.e. before any strategy revision may have taken place. This strategy is stored in variable C-player-last?. With this, we conclude the code of procedure to update-strategy.

to update-strategy
  set C-player? [C-player-last?] of one-of my-nbrs-and-me with-max [payoff]
end

4.5. Complete code in the Code tab

The Code tab is ready!

patches-own [
  C-player?
  C-player-last?
  payoff
  my-nbrs-and-me
  n-of-my-nbrs-and-me
]

to setup
  clear-all
  setup-players
  ask patches [update-color]
  reset-ticks
end

to setup-players
  ask patches [
    set payoff 0
    set C-player? false
    set C-player-last? false
    set my-nbrs-and-me (patch-set neighbors self)
    set n-of-my-nbrs-and-me (count my-nbrs-and-me)
  ]
  ask n-of (round (initial-%-of-C-players * count patches / 100)) patches [
    set C-player? true
    set C-player-last? true
  ]
end

to go
  ask patches [play]
  ask patches [set C-player-last? C-player?]
  ask patches [update-strategy]
  tick
  ask patches [update-color]
end

to play
  let n-of-C-players count my-nbrs-and-me with [C-player?]
  set payoff n-of-C-players * ifelse-value C-player? [CC-payoff] [DC-payoff] +
       (n-of-my-nbrs-and-me - n-of-C-players) * ifelse-value C-player? [CD-payoff] [DD-payoff]
end

to update-strategy
  set C-player? [C-player-last?] of one-of (my-nbrs-and-me with-max [payoff])
end

to update-color
  set pcolor
    ifelse-value C-player?
     [ifelse-value C-player-last? [blue] [lime]]
     [ifelse-value C-player-last? [yellow] [red]]
end

4.6. Code in the plots

We will use the following color code for the patches:

  • Blue if it is occupied by a C-player and in the previous period it was also occupied by a C-player.
  • Green if it is occupied by a C-player and in the previous period it was occupied by a D-player.
  • Red if it is occupied by a D-player and in the previous period it was occupied by a D-player.
  • Yellow if it is occupied by a D-player and in the previous period it was occupied by a C-player.

To complete the Interface tab, edit the graph and create the pens as in the image below:

Plot settings
Figure 3. Plot settings.

6. Sample runs

We can use the model we have implemented to shed some light on the question that we posed at the motivation above. We will use the same parameter values as Nowak and May (1992), so we can replicate their results: CD-payoff = DD-payoff = 0, CC-payoff = 1, DC-payoff = 1.85, and initial-%-of-C-players = 90.[5] An illustration of the sort of patterns that this model generates is shown in the video below.

As you can see, both C-players and D-players coexist in this spatial environment, with clusters of both types of players expanding, colliding and fragmenting. The overall fraction of C-players fluctuates around 0.318 for most initial conditions (Nowak and May, 1992). Thus, we can see that adding spatial structure can make cooperation be sustained even in a population where agents can only play C or D (i.e. they cannot condition their actions on previous moves).

Incidentally, this model is also useful to see that a simple 2-player 2-strategy game in a two-dimensional spatial setting can generate chaotic and kaleidoscopic patterns. To illustrate this, let us use the same payoff values as before, but let us start with all agents playing C, i.e. initial-%-of-C-players = 100.

When you click on setup, the whole world should look blue, since all agents are C-players. If you now click on go, nothing should happen, since all agents are playing the same strategy and the strategy updating is imitative. To make things interesting, let us ask the agent at the center to play D. You can do this by typing the following code at the Command Center (i.e. the line at the bottom of the NetLogo screen) after clicking on setup:

ask patch 0 0 [set C-player? false]

If you now click on go, you should see the following beautiful patterns:

7. Exercises

You can use the following link to download the complete NetLogo model: 2×2-imitate-best-nbr.

Exercise 1. Let us run a (weak) Prisoner’s Dilemma game with payoffs DD-payoff = CD-payoff = 0, CC-payoff = 1 and DC-payoff = 1.7. Set the initial-%-of-cooperators to 90. Run the model and observe the evolution of the system as you gradually increase the value of DC-payoff from 1.7 to 2. If at any point all the players adopt the same strategy, press the setup button again to start a new simulation. Compare your observations with those in fig. 1 of Nowak and May (1992). Note: To use the same dimensions as Nowak and May (1992), you can change the location of the NetLogo world’s origin to the bottom left corner, and set both the max-pxcor and the max-pycor to 199. You may also want to change the patch size to 2.

 CODE  Exercise 2. Create a button to turn the central patch into a D-player. You may want to label it one central D player. This button will be useful to replicate some of the experiments in Nowak and May (1992, 1993).

Exercise 3. Replicate the experiment shown in figure 3 of Nowak and May (1992).

 CODE  Exercise 4. Implement the following extension to Nowak and May (1992)‘s model, proposed by Mukherji et al. (1996):

With a small probability ε, each player errs and chooses evenly between strategies C and D; with probability 1-ε, the player follows the Nowak and May update rule.

You may wish to rerun the sample run above with a small value for ε. You may also want to replicate the experiment shown in Mukherji et al. (1996, fig. 1).

 CODE  Exercise 5. Implement the following extension to Nowak and May (1992)‘s model, proposed by Mukherji et al. (1996):

During each period, players fail to update their previous strategy with a small probability, θ.

You may wish to rerun the sample run above with a small value for θ. You may also want to replicate the experiment shown in Mukherji et al. (1996, fig. 1).

 CODE  Exercise 6. Implement the following extension to Nowak and May (1992)‘s model, proposed by Mukherji et al. (1996):

After following the Nowak and May update rule, each cooperator has a small independent probability, ϕ, of cheating by switching to defection.

You may wish to rerun the sample run above with a small value for ϕ. You may also want to replicate the experiment shown in Mukherji et al. (1996, fig. 1).


  1. Note that in most evolutionary models there are two types of neighborhoods for each individual agent A:
    • the set of agents with whom agent A plays the game, and
    • the set of agents that agent A may observe at the time of revising his strategy.
    Most often these two sets coincide for each individual agent, but that is not necessarily the case (see e.g. Ohtsuki et al. (2007 a,b).
  2. Christoph Hauert has an excellent collection of interactive tutorials on this topic at his site EvoLudo (Hauert 2018).
  3. By default, user-defined variables in NetLogo are initialized with the value 0, so there is no actual need to explicitly set the initial value of individually-owned variables to 0, but it does no harm either.
  4. The command tick could just as well be included at the very end of procedure to go. We tend to like using tick to separate the main logic of the round from the instructions that deal with plotting and visualization, but this is just a matter of preference.
  5. Some authors make CD-payoff = DD-payoff, so they can parameterize the Prisoner's Dilemma with just one parameter, i.e. DC-payoff. Note, however, that this reduces the range of possibilities that can be studied.

License

Icon for the Creative Commons Attribution 4.0 International License

Agent-Based Evolutionary Game Dynamics by Luis R. Izquierdo, Segismundo S. Izquierdo & William H. Sandholm is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.

Share This Book