EFTA00810742.pdf

Theoretical Population Biology 12S (2019) 38-55 Contents lists available at ScienceDirect Theoretical Population Biology ITSEVIER journal home pa go: ..vww.elscvier.comflocateApb A two-player iterated survival game John Wakeley a.*, Martin Nowak a'b'c 'Department of Organismic and Evolutionary Biology. Harvard University. Cambridge. MA, 02138. USA b Program for Evolutionary Dynamics. Harvard University. Cambridge. MA 02138. USA ' Department of Mathematics, Harvard University. Cambridge, hfA 02138, USA ARTICLE INFO ABSTRACT Ankle history: We describe an iterated game between two players, in which the payoff is to survive a number of steps. Received 22 May 2018 Expected payoffs are probabilities of survival. A key feature of the game is that individuals have to survive Available online 12 December 2018 on their own if their partner dies. We consider individuals with hardwired, unconditional behaviors Keywords: or strategies. When both players are present. each step is a symmetric two-player game. The overall Prisoner's Dilemma survival of the two individuals forms a Markov chain. As the number of iterations tends to infinity, all Survival game probabilities of survival decrease to Zero. We obtain general, analytical results for n-step payoffs and use Iterated game these to describe how the game changes as it increases. In order to predict changes in the frequency of Replicator equation a cooperative strategy over time, we embed the survival game in three different models of a large, well- Moran model mixed population. Two of these models are deterministic and one is stochastic. Offspring receive their parent's type without modification and (finesses are determined by the game. Increasing the number of iterations changes the prospects for cooperation. All models become neutral in the limit (ri co). Further, if pairs of cooperative individuals survive together with high probability, specifically higher than for any other pair and for either type when it is alone, then cooperation becomes favored if the number of iterations is large enough. This holds regardless of the structure of pairwise interactions in a single step. Even if the single-step interaction is a Prisoner's Dilemma, the cooperative type becomes favored. Enhanced survival is crucial in these iterated evolutionary games: if players in pairs start the game with a fitness deficit relative to lone individuals, the prospects for cooperation can become even worse than in the case of a single-step game. 02018 Elsevier Inc. All rights reserved. I. Introduction the temptation, to be the one who comes out ahead even if it is at the expense of other individuals. More than a century ago, Kropotkin (1902) argued that what Indeed, game theory and evolutionary theory have uncovered he called mutual aid should be ranked among the main factors of numerous situations in which cooperative or otherwise helping evolution. aneven more important driver ofevolution than within- behaviors are detrimental t0 the individual or selectively disad- species competition. The idea ofmutual aid is similar to that ofmu- vantageous (Hofbauer and Sigmund, 1998; Mesterton-Gibbons. tualism: partnerships may be beneficial to both partners and thus 2000; Cressman, 2005; Schecter and Gintis, 2016). It has been unconditionally favored. Kropotkin had been deeply impressed shown using a variety of models that such behaviors can be disfa- by the abilities of animals to endure harsh winters and perilous vored even if there are obvious advantages to mutual cooperation. migrations in northern Eurasia by living together and supporting For example, in the classic two-player game called the Prisoner's each other in situations where a lone animal had little chance Dilemma (Tucker, 1950; Rapoport and Chammah, 1965; Axelrod. 1984). cooperation results in a "reward payoff' if one's partner of surviving. Thus, he inferred a connection between cooperative behaviors and survival under adverse conditions. But Kropotkin did also cooperates but a "suckers payoff' if one's partner defects. Defection yields a "temptation payoff" if one's partner cooperates not spell out the ways in which adversity might favor cooperative but a "punishment payoff" if one's partner also defects. The reward behaviors, nor did he consider that cooperative behaviors could be is of course better than the punishment, but it is further assumed disadvantageous. He took the very fact that animals seem to do that the temptation payoff is greater than the reward, and the better in groups than alone as evidence for mutual aid, apparently sucker's payoff is even worse than the punishment This leads to a unaware that any interaction presents the opportunity, or at least paradox: an individual who defects always receives a higher payoff but it is clearly illogical for everyone to defect. * Corresponding author. The Prisoner's Dilemma, in which the payoff for defection is E-mail address: wakeleyefas.harvard.edu U. Wakeley). higher than for cooperation regardless of one's partner behavior, hups://doLorg/10.1016b.tpb.2018.12.001 0040-S809/0 2018 Elsevier Inc. All rights reserved. EFTA00810742  J. Wakeley and M. Nowak/Theoretical Population Biology 125(2019)38-55 39 Table 1 we will call PD for short.Ifa(n) > c(n)and b(n) a d(n), cooperation The payoff, a(n). J(n), c(n) or d(n). that an individual receives in a symmetric has the higher payoff only when one's partner cooperates. This is two-player game depends both on the individual and on the individual's paltrier. In a survival game, these payoffs are probabilities of survival A and B denote the Stag-Hunt class of games, or SH for short. If a(n) e c(n) and two possible strategies or types of individuals (e.g. Dove and Hawk). Payoffs are b(n) > d(n), cooperation has the higher payoff only when one's simultaneously awarded to both players, i.e. each is considered the Individual (and partner does not cooperate. This is the Hawk-Dove class, or HD for the other the Partner) and awards a payoff according to the table. short. Finally, if a(n) > c(n) and b(n) > d(n), cooperation has the Partner higher payoff regardless of the partner. We follow De Jaegher and A Hoyer (2016) and call this a Harmony Game. or HG for short. A o(n) b(n) We refer to the n-step survival game as a repeated or iterated Individual B c(n) d(n) game because the payoff structure in each of then steps is identical and equivalent to the single-step version of the game. The notion of repeated or iterated games is generally predicated on the idea that individuals can act differently in different steps and can react represents one of four possible types of two-player games. It is the polar opposite of what Kropotkin (1902) imagined for mutual to their partner's behavior. When this is true, the relatively sim- aid, that instead it would cooperation that would always yield ple conclusions about which strategy may be favorable based on the higher payoff. Between these extremes lie two other kinds of Table 1 do not necessarily hold. For example, if two individuals games. In the Stag Hunt (Skyrms, 2004) and related games, the play an iterated Prisoner's Dilemma under these conditions, then payoff to an individual is higher ifit matches the partner's behavior reactive strategies like Tit-for-Tat (Axelrod, 1984) or win-stay, regardless of what that behavior is. Cooperators in the Stag Hunt lose-shift (Nowak and Sigmund, 1993) can be favored over the go for the big game, a stag, which two such individuals can catch single-iteration strategy of always-defect. Such repeated games but one alone cannot. Non-cooperators opt out of the big-game allow for the phenomena ofdirect and indirect reciprocity (Trivers. hunt and accept a middling payoff, a hare, which a single individual 1971; Nowak and Sigmund, 1998) and trigger strategies which can catch. In this case, cooperation yields the higher payoff only punish non-cooperation (Osborne and Rubinstein, 1994). These when one's partner also cooperates. The Hawk-Dove game (May- are examples of a general phenomenon of behavioral responsive- nard Smith and Price, 1973; Maynard Smith, 1978) represents the ness (Van Cleve and Alccay, 2014) which is one of a small number fourth kind, in which havinga different behavior than one's partner of mechanisms known to promote the evolution of costly cooper- produces a higher payoff. Doves cooperate by sharing resources but ation (reviewed in Nowak, 20066; Van Cleve and Alccay, 2014). retreat when challenged. Hawks do not cooperate. They are ready We do not consider reactive strategies or behavioral respon- to fight to avoid leaving an interaction empty-handed. A Hawk gets siveness in this work. Individuals have one of two possible simple the entire resource when facing a Dove but suffers badly when fac- strategies: A or B as in Table 1. When both individuals are present, ing another Hawk Hawk and Dove, ofcourse, refer to stereotypical which is always the case initially, their survival probabilities in personalities not animals. In this last case, cooperation yields the each iteration are given by the single-step version of Table 1. i.e. higher payoff only when one's partner does not cooperate. Besides with a( 1), b(1), c(1) and d( 1). We will refer to these single-step Hawk-Dove, other well-studied games of this type are the game survival probabilities as a, b, c and d. The n-step survival proba- of chicken and the snowdrift game (Doebeli and liauert. 2005: bilities, a(n), b(n), c(n) and d(n), will depend on these as well as on Nowak, 2006a). the probabilities of survival when an individual is alone. Although We introduce a two-player survival game which can fall into the game always begins with two individuals, if one dies the other any of these four classes. It may also change, for example from a must continue. In the remaining steps, a loner plays a game against Hawk-Dove game into a Prisoner's Dilemma, depending on one Nature. Specifically, a loner survives a single step with probability key feature, the length of the game. In this game, an initially as if it has strategy A and probability do if it has strategy B. We are sampled pair of individuals confronts a hazardous situation which especially interested in cases in which the single-step, two-player is repeated n times. With reference to Kropotkin (1902). we might game defined by a, b. c and d is of a different type than the n-step imagine that each day of a long journey presents a similar set of game defined by a(n), b(n), c(n) and d(n). challenges which threaten survival. They might, for example, be Several previous works have considered games in which the trying to survive a number of very cold nights or attempting to random survival of individuals is an important factor and envi- defend themselves repeatedly against a predator. How they fare in ronmental conditions may be harsh. Eshel and Weinshall (1988) each step depends on their behavior and their partner's behavior. introduced a model in which payoffs are probabilities of survival. The payoff for an individual is all or nothing: either survive to Payoffs were drawn randomly from a distribution, and the game the end of the game or not. Crucially, an individual must face the was repeated with a fixed probability. As in the model we pro- perilous situation alone for the remainder of the game ifits partner pose, Eshel and Weinshall (1988) allowed that the game may con- dies. Survival payoffs are thus meted out at each step, and this will tinue even if one individual dies. They considered optimal strategy be important for determining total payoffs in the game. Denoting choice by individuals under the assumption that individuals have survival as I and death as 0, the expected total payoff to an perfect knowledge of the game's structure, including the distribu- individual in a given situation(i.e. witha specified partner initially) tion of the payoffs. Eshel and Shaked (2001)described a similar sur- is its probability of surviving to the end of the game. The n-step vivaI game but included a general probability that both members survival game is a symmetric two-player game with total payoffs, of a pair survive, thus allowing for arbitrary synergistic effects of a(n), d(n), c(n) and d(n) as Table 1. Using Table 1 to represent an partnership (Hauert et al.. 2006; Kun et al.. 2006) within each step n-step game that is a Prisoner's Dilemma, with A for cooperation of the game. Eshel and Weinshall (1988) had assumed that pairwise and B for defection, the reward would be a(n), the sucker's payoff survival probabilities were the products of two individual survival b(n), the temptation payoff c(n) and the punishment d(n). probabilities. More generally, depending on a(n), b(n), c(n) and d(n), any Garay (2009) combined the idea of a survival game with the n-step survival game falls into one of the four classes described "selfish herd" theory of Hamilton (1971) to produce a model of above. Ignoring the detail that some payoffs might be identical, cooperation between pairs of individuals in defense against re- these are defined as follows.Ifa(n) < c(n) and b(n) < d(n). cooper- peated attacks by a predator. Individuals were of two possible ation has the lower payoff regardless of the partner. Then the game types, characterized by a probability of helping their partner in falls into the class exemplified by the Prisoner's Dilemma. which defense against attack. Garay (2009) derived the total survival EFTA00810743 40 J. Wakefey and M. Nowak/ Theoretical Population Biology 125 (2019)38-55 probabilities of individuals in different kinds of partnerships given lower payoff given partners of type A and B, respectively. In game a faced number ofattacks, and used these to describe evolutionarily theory and much of evolutionary game theory which treats pop- stable strategies (Maynard Smith and Price, 1973) in an infinite ulations, these two comparisons are relevant because individuals population. Using the example contrast of one attack versus eight choose or modify their strategies based on their knowledge of attacks, this revealed cases in which helping could not invade a the game and its outcomes (Sandholm, 2010). When individuals completely selfish population given a one-attack game but could cannot alter or choose their behaviors but payoffs affect fitness. invade if the number of attacks was larger (Garay. 2009). the same sorts of evolutionary models are used to predict changes Kropotkin's notion that high levels of adversity could favor in the frequency of hardwired behaviors in a population. In this cooperation has also been studied using simulations of structured section, we briefly summarize the evolutionary game dynamics of populations. Harms (2001) found that in a Prisoner's Dilemma co- infinite populations. operators could gain an advantage at the inhospitable margins of a The replicator equation, in which general payoffs are inter- population by colonizing patches cleared by the local extinction of preted as contributions to individual fitness (Taylor and Jonker, defectors. More recent simulations of another model by Smaldino 1978; Hofbauer et al., 1979; Zeeman, 1980; Schuster and Sigmund, et al. (2013) also found cases in which cooperation can be favored 1983; Hofbauer and Sigmund, 1988, 1998) provides an evolution- despite the Prisoner's Dilemma. Specifically, if there is a high cost ary setting for the classification of two-player games. In a survival of living for all individuals which, if unchecked, would lead to the game payoffs are fitnesses, specifically viabilities.If x is the relative extinction of the population and if occasional interactions with frequency of A in an infinite population, the replicator equation cooperators are required for survival, then cooperation can be gives the instantaneous rate of change favored. As in Harms (2001). the success ofcooperators in this case relied on their ability to form clusters (Smaldino et al., 2013). = — x)(la(n)— c(n)Ix lb(n)— d(n)j(1 — x)). (1) De Jaegher and Hoyer (2016) considered two game-theoretic Eq. (I) illustrates the evolutionary consequences of partner- models of the behavior and ecology of a pair of individuals, in dependent payoffs when pain are formed at random in proportion which higher levels of adversity can change the type of the game. to the frequencies ofA and B. When x x 1. so thatA is very common Their Model I includes a degree ofcomplementarity which rescales and nearly all partners are A, it is the sign of a(n) — c(n) that payoffs for mixed-strategy pairs compared to same-strategy pain. determines whether A is favored in the population. On the other and which could represent an environmental challenge for mixed hand, when x x 0, so that nearly all partners are B, the sign of pairs. Their Model 2 includes a number of attacks by a predator b(n) — d(n) is what matters. Beyond this. Eq. ( I ) shows that for any on individuals protecting a common resource. Cooperators are value of x, it is the sign of (a(n) — c(n)ix lb(n) — d(n)1(1 — x) immune to attacks while defectors can sustain at most one at- that determines whether A is favored. Thus. in a population and in tack before the common resource is lost. As in Garay (2009). the evolution the relative magnitudesof both a(n)—c(n) and b(n)—d(n) number of attacks indicates the level of environmental challenge. are important. such that one may dominate the other at a given Cooperation may be disfavored when the number of attacks is value of x. This is not something that can be gleaned directly from small and yet become favored when the number of attacks is large. Table 1. Depending on the other parameters in the model, however, a range Without mutation, the fates ofA and B in this infinite population ofother switches between types of games may occur as the number are entirely determined by Eq. (1). From any starting point, x will of attacks increases. De Jaegher (2017) extended these results to move deterministically toward one of three possible equilibria multi-player games. which are the solutions of k = 0 and which may correspond to Our concerns here are similar to those of Garay (2009) and the evolutionarily stable strategies mentioned in Section 1. Two De Jaegher and Hoyer (2016). We study how the structure of the monomorphic equilibria always exist. z = 0 and 1 = 1. The two-player iterated survival game changes as a function of its polymorphic equilibrium parameters, in particular the number of steps it. We compare the conclusions for single-step games with those for n-step games. — b(n) — d(n) (2) Because the payoffs which are probabilities ofsurvival in this game b(n) — d(n) — a(n) c(n) decrease as n increases. n is a measure of adversity. We present might also exist, depending on the payoffs.It exists, which is to say general results for any level ofadversity, then focus on the possible that Eq. (2) gives a biologically meaningful value, when a(n), b(n), structures of large-n games. We find conditions under which any c(n) and d(n) are such that 0 < it < 1. type of single-step game, be it a Prisoner's Dilemma, a Stag Hunt or Eq. ( I ) defines the same four types of symmetric two-player a Hawk-Dove game. will become a Harmony Game as n increases. games. In the PD case, a(n) — c(n) < 0 and b(n) — d(n) < 0. We identify three other large-n results, one of which shows the and Eq. (1) shows that z < 0 for all values of x E (0, 1). Starting opposite: single-step advantages of cooperation may disappear at any value x < 1, the frequency of A will decrease to zero. The as n grows. To facilitate comparisons, we define A throughout as polymorphic equilibrium given by Eq. (2) does not exist. In the SH the more cooperative type based on the single-step game, so that case, a(n)—c(n) > 0 and b(n)—d(n) < O. ThenA isdisfavored when a > d. Thus, AA pairs survive better than BB pairs. We consider x is below the polymorphic equilibrium Si in Eq. (2) and favored both infinite and finite populations but in neither case is there when x > 2. The polymorphic equilibrium exists and is unstable. spatial or any other kind of structure. We do not develop a detailed In the HD case, a(n) — c(n) < 0 and b(n) — d(n) > 0. Then A is biological. ecological or behavioral model. Akin to what is done in favored when x <2. and disfavored when x > z. The polymorphic models of diploid viability selection, we describe the game and its equilibrium exists and is stable. In the HG case. a(n)— c(n) > 0 and results directly in terms of survival probabilities. Our results are b(n) — d(n) > O. Then k > 0 and A is favored for all x e (0. 1). The applicable to a wide range of specific scenarios in which payoffs polymorphic equilibrium does not exist. Fig. I shows examples of a affect viability (as opposed to fertility or fecundity) and in which Prisoner's Dilemma, a Stag Hunt and a Hawk-Dove survival game. partnerships may either enhance survival or detract from it. depicting payoffs in their contexts (upper panels) and associated shapes of k (lower panels). A Harmony Game is not shown but It Evolutionary dynamics would be another case of directional selection, like Fig. ID but with 1 > 0 for all x e (0, 1). Comparing a(n) to c(n) and b(n) to d(n) in Table I indicates These four types of games defined by the shape of ic over the whether the more cooperative strategy A yields the higher or the interval 10, 1] are identical to what is observed in classical models EFTA00810744  J. Wakeley and M. Nowak/Theoretical Population Biology 125(2019)38-55 41 A 1. Prisoners Dilemma c(n) B 1. Stag Hunt C 1. Hawk-Dove cm) 1 098 096 a(n) a(n) = a 7 o sa ow n) n) k 0.98 0.98 ..................., vz' U^) a(n) 0.94 E -0 094 :E 0.94 2 M^> c • n) d(n) BB M BA Ark - BB MBA AA BB MBA AA Individual-Partner Pair Individual-Partner Pair Individual-Paitner Pair D E Stag Hunt F Hawk-Dove o. 0.001 0. -0.CO1 0. X -0.002 A -0.001 A -0.001 -0.003 -0.002 -0.003 -0.002 0 0.25 0.5 0.75 0 0.25 0.5 0.75 0 0.25 0.5 0.75 1 x x Flg. 1. A and D: Prisoner's Dilemma with payoffs a = 0.97. b = 0.94, c = 0.99, d = 0.95 (a linear transformation of the classic R = 3, 5 = 0, 7 = 5, P = 1 of Axelrod 0984e.g.a = 0.94 + R/100).B and E: Stag Hunt, with a = 0.99.0 = 0.94, c = d = 0.97 (corresponding to a stag value of0.05 and a hare value of0.03 added to a baseline survival probability of 0.94). C and F: Hawk-Dove, with a = 0.97, b = 0.95, c = 0.99, d = 0.94 (corresponding to a cost of fighting of 0.03 and a resource value of 0.04, with a baseline survival probability of 0.95). Line segments in A, B and C connect the survival probabilities for B (left) versus A (right) when each occurs with given type of partner. The curves in D. E and F show k from Eq. (1). In diploid population genetics. these three cases fora are called directional selection (D), underdominance (E) and overdominance (F). of diploid viability selection (e.g. see section 4.2 in Nagylaki, 1992). and the state in which both individuals have died which we denote The difference is that, barring atypical phenomena such as meiotic 0. It is convenient to represent a single iteration using the matrix drive (Dunn, 1953; Sandler and Novitsky, 1957) or segregation- AA AB BB A B 0 distortion (Sandler and Hiraizumi, 1960; Hard. 1974), standard IA a2 0 0 2a(1 - a) O (1 — a)2 1 diploid models always have b(n) = c(n). It is by allowing b(n) # AB 0 be 0 b(1 — r) r(1 — b) (1 — b)( 1 — c) c(n) that two-player games introduce the paradox of the Prisoner's BB 0 0 d2 0 2d(1 — d) (1 — d)2 Dilemma, in which c(n) > a(n) and d(n) > b(n) but a(n) > d(n). In A 0 0 0 a0 O 1 — ao (3) order to have c(n) > a(n) and d(n) > b(n) in a standard diploid B 0 0 0 0 1 — do do model it would be necessary to have a(n) e d(n). Accordingly. O 0 0 0 0 O 1 the Prisoner's Dilemma is the most stringent form of a cooperative dilemma (Doebeli and Hauert, 2005; Hauert et al., 2006; Nowak, with entries equal to the transition probabilities among he six 2012). A cooperative dilemma exists when (i) mutual cooperation states. For example, the transition from state AB to state B means results in a higher payoff than mutual non-cooperation, so that that the B individual survives and the A individual dies. In a single a(n) > d(n) when A represents cooperation, but (ii) there is incen- step, this occurs with probability c(1— b). Note that the transitions tive to be non-cooperative in at least one of three ways: (iia) c(n) > in Eq. (3) include the fates of both individuals but the individuals a(n), (iib) d(n) > b(n) or (iic) c(n) > b(n) (Nowak, 2012). The are not labeled Individual and Partner as they are in Table I. The process described by Eq. (3) is depicted in Fig. 2. State Prisoner's Dilemma includes all three of these incentives. Games O is an absorbing state. There is no possibility of transition be- with fewer barriers to cooperation, such as the Stag Hunt and the tween the paired states M. AB, and BB. Instead each of these Hawk-Dove game, represent relaxed cooperative dilemmas. The feeds either straight into state 0, from which there is no escape. Harmony Game involves no cooperative dilemma and presents no or into one of the loner states. A or B. and from there into 0. barrier to cooperation. Therefore, transitions from any of the starting, paired states to absorption in state 0 involve either one or two changes of state. 2. An n-step survival game between two players Further, all of the transitions that cannot occur in a single iteration (the 0 entries in the matrix) cannot ever occur regardless of the Our survival game always includes n iterations and begins with number of iterations. Because of this simple structure, the n-step a pair of individuals. In each step, both might survive or one, transition probabilities can be calculated directly by conditioning the other or both might die. These same outcomes hold for the on the times these transitions take place, i.e. on their positions in the sequence of n iterations. It is also possible to compute the complete game, only the probabilities of surviving will be smaller. n-step transition probabilities using standard techniques for We consider two unconditional strategies. A and B. When both Markov chains, and this provides a useful framework for decom- players are present, their individual survival probabilities are given posing the process and describing its behavior when n is large. by the single-step version of Table I with a(1) • a, b(1) • b, Details of the matrix approach are given in the Appendix. but c(1) • c and d(1) • d. However, the next iteration must be two key features of it inform our presentation. The first is the fact faced by whoever has survived so far, until all n iterations are done. of the absorbing state (0) in which both individuals have died. It Thus, an individual might have to play alone. Then the survival will be reached eventually, meaning in the limit n oo. For large probabilities become ao forA without a partner and do for B without n the game process will be just the approach to this state. Second, a partner. In all of what follows. A will be the more cooperative when n is large, the rate of approach to state 0 will be given by the strategy, meaning that a > d. largest non-unit eigenvalue of the matrix in Eq. (3). Because it is an The n-step survival game is a stochastic process with six possi- upper triangular matrix, the eigenvalues are simply the entries on ble states: the paired states AA, AB and BB, the loner states A and B, the diagonal. By convention, we call the largest of these 2. = land EFTA00810745 42 J. Wakefey and M. Nowak/Theoretical Population Biology 12S (2019)38-55 Prob(BB 8) = E 2d(1- (14) I 0 = (d2n A„ 2d( 1 d) ) d2 (15) ® The only other possibility is that neither individual survives, so we also have Prob(AA 0) = 1 —Prob(AA AA) — Prob(AA —* A) (16) Prob(AB —a. 0) = 1 — Prob(AB AB) —Prob(AB A) .® Prob(AB B) (17) 0 Prob(BB —* 0) = 1 — Prob(BB —a. BB) — Prob(BB B). (18) a In the Appendix we show how these probabilities are obtained using techniques for Markov chains. Eqs. (8) through (15) make it clear that this paired survival process is one in which the fortunes of individuals may change. rv R 0 perhaps drastically depending on the values of ao and do relative to a, b, c and d. Consider starting state AB. Eq. (12) shows how the probability of beingin state Bat the end ofn iterations is computed. 0 In words, both individuals survive for some time (i-1 steps), then A dies, and B survives the rest of time (n - i steps). If A dies, B trades its individual survival probability ofc. which B enjoys in the Flg. 2. Flow diagram ofthe stochastic process givenby the matrix of Eq. (3).Arrows show all possible transitions among the six states (A4. At BB. A. B. 0).States AA, AB, presence and A, fora loner survival probability ofdo. It could be that BB. A and B are transient. State 0 is absorbing. The process always begins in one of do c c, making B worse off after A dies. In fact, B's fate is closely the three states on the left, M. AB or BB. After n iterations. it may be in any of the tied with A's because this switch could occur quickly if A's survival six possible states. probability, b, is small. Of course. while there is a cost to B when A dies (assuming 4 c c), the death of A in this partnership also represents a rather direct disadvantage toA.In order to understand note that this corresponds to the eventual absorption in state 0. In whether A or B will prevail in evolution, it is necessary to account all, we have for the full dynamics of reproduction in a population, with fitnesses that are determined by this game. We will take this up in Section 3. = 1, a2, bc, d2, ao, . (4) In the n-step game, the differences a(n) - c(n) and b(n) — d(n) give the conditions under which A is favored. The n-step payoffs. Following the discussion of Table 1 and Eq. ( I), we expect the or survival probabilities, are computed by accounting for the two fates of A and 8 in this iterated game to depend on the relative ways an individual may survive the game. An individual survives magnitudes of a versus c and b versus d. Eq. (4) suggests that if both it and its partner survive or if it survives but its partner their fates will also depend on the relative magnitudes of the dies. The total probabilities of individual survival in each kind of pair survival probabilities. a2. be and d2. and on the loner survival partnership are probabilities. ao and 4.and that this dependence may be especially a(n) = Prob(AA AA) + Prob(AA -o A)/2 strong when n is large. We compute the n-step pair survival probabilities directly as _ a2" a—ao + a(1 — a) (19) a2 — ao ao — a2 Prob(AA —* AA) = a2" (5) b(n) = Prob(AB AB) + Prob(AB A) Prob(AB —a. AB) = (bcr (6) b ao b(1 — c) = (kr +4 (20) be - ao ao — be Prob(BB BB) = dm. (7) c(n) = Prob(AB AB) Prob(AB B) Then, by considering the possibility that one of the individuals c — do c(1 — b) = (bcr +4 (21) might die in step 1 < i < ri and the other individual survives to be -d o do — bc the end of the game, we have d(n) = Prob(BB BB) + Prob(BB B)/2 re 2 d dO d — d) —d" + 4 (22) Prob(AA -* A) = E (a2)1-1 2a(1 - a)arl d2 — 4 4 - d2 The transitions AA A and BB B are adjusted by a factor = 02. 4) 2a(1 — a) of 1/2 because they are equally likely to happen by the death 2 — an of the partner as by the death of the focal individual. Eqs. (19) through (22) are our general results, namely the n-step survival Prob(AB -> A) = E(bc)i-1b(1 - c)arl probabilities for individuals of types A and B given each kind of a-t initial partnership. They are exact for any values of a, b, c, d, ao and do greater than zero and less than one, and for any number ((kr d°) 1_ c) of iterations n 1. As expected, when n = 1, they reduce to the It single-step survival probabilities a(1) = a, b( 1) = b, c(1) = c and Prob(AB B) = DbCr I CO — LOCIrj d(1) = d. i-t The two key differences in payoff are then CO — b) a ao (ocr - d rf c do a(n) - c(n)- am ao + 4 Go ——a2 a) a EFTA00810746  J. Waketey and M. Nowak/Theoretical Population Biology 125 (2019)38-55 43 — c(I — b) iterations increases. We present results both for a(n) — c(n) and _ (krc__ (23) be —do ° do — bc b(n) — d(n) and in terms of the unified predictions of Eq. ( I ). bt c) Eq. (1) is a standard replicator equation for a symmetric two- b(n)— d(n)= (bon + aiot player game. In the Appendix, we show how it may be derived bc —ao ao aol — bc for the n-step survival game. This has two notable features: the _ d2nd_ do d(1 — d) (24) game works by removing indi