EFTA00632631.pdf

Cooperate without looking: Why we care what people think and not just what they do Moshe Hoffman', Erez Yoeli', and Martin A. Nowak' I' •Peogram for Evolutionary Dynamics, Oeparunent of Mathematics and 'Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA 02138 Edited by Michael S. Gananiga, University of California, Santa Barbara, CA, and approved December 24. 2014 (received for review September 30, 2014) Evolutionary game theory typically focuses on actions but ignores The payoff matrix is shown in Table I. The strategy pair "always motives. Here, we introduce a model that takes into account the defect" and "alums end" (ALLD) is always a Nash equilibrium: no motive behind the action. A crucial question is why do we trust player can increase her payoff by deviating unilaterally. However, people more who cooperate without calculating the cost? We there are other Nash equilibria. All proofs are in SI Appendix. propose a game theory model to explain this phenomenon. One Of particular interest is the strategy pair where player 1 chooses player has the option to 'look" at the costs of cooperation, and the CWOL and player 2 aids the game if player 1looks. This strategy pair other player chooses whether to continue the interaction. If it is is a Nash equilibrium ifa/(1 — w)a.cip +ch(1 —p).This condition has occasionally very costly for player 1 to cooperate, but defection is a natural interpretation: player I's expected temptation from de- harmful for player 2, then cooperation without looking is a subgame fection is less than the gains from an ongoing cooperative in- perfect equilibrium. This behavior also emerges in population-based teraction. The expected temptation matters because, if player 1 processes of learning or evolution. Our theory illuminates a number were to look, player 2 would end the relationship. Thus, player 1 of key phenomena of human interactions: authentic altruism, why might as well defect. regardless of the temptation. Not looking, in people cooperate intuitively, one-shot cooperation, why friends do a sense, smooths the temptation to defect; the variability in not keep trade of favors, why we admire principled people, Kant's temptations no longer matters. second formulation of the Categorical Imperative, taboos, and love. Another relevant strategy pair is if player 1 CWL and player 2 ends if player 1 defects. This pair is a Nash equilibrium if game theory I evolution I emotion I motive I cooperation a/(1 — w) ≥ ch. This condition has the following interpretation: to sustain CWL, the long-term gains to player 1 from the ongoing C ooperation OMITS when we lake on costs to help others. A key mechanism by which cooperation is sustained is reciprocity: Individuals cooperate with those who have cooperated in the past relationship must suffice for player 1 tocooperate. even if player 1 knows the temptation is high in the current period. For CWL. it is the maximal temptation that matters: because player 1 is not pe- (1-14). However, we care about not only whether others cooperate nalized for looking. she can look at the temptation and choose to but also. their decision-making process: wc place more trust in defect only if it is high. cooperators who do not strategically weigh the arts and make an When it is oa:asionally very costly to cooperate kip +ch(1 —p)≤ effort to collect than before deciding whether to cooperate. For a1(1 —w)<chl, CWL is not an equilibrium. but CWOL is. This example. we are impressed by colleagues who immediately agree expression identifies the region where we should be most likely to proofread a paper but view with suspicion those who ask, to discover CWOL In SI Appendix, we show that the inequality "how many pages does it have?" Intuitively. those who cooperate pb+ (1 —p)d <0 must hold for CWOL to emerge. without "looking" (CWOL) can be trusted to cooperate even in times In SI Appendix. we also address some concerns. First, after wc when there arc large temptations to defect. However. will the added consider a richer strategy set. there might be other equilibria of trust from CWOL be worth missing out on those large temptations? the envelope game, in which player I sometimes does not look. Additionally, which conditions make CWOL a winning strategy? These equilibria might exist undcr different conditions from To address these questions. we develop thc envelope game (Fig. those given above, which may draw into question our statement I). which is a repeated asymmetric game between two players. In that looking matters under these conditions. We show that, whcn each round. player 1 receives an envelope. which contains the we rule out strategies that randomize or depend on the round, magnitude of the temptation to defect. The temptation is low with probability p and high with probability 1 —p. Player 1 can choose Significance to look inside the envelope and, thus, find out the magnitude of the temptation. Then. player 1 decides to cooperate or defect Sub- Why do we trust people more when they do good without sequently. player 2 can either continue or end the game. In the considering in detail the cost to themselves? People who avoid former case there is another round with probability w. "looking" at the costs of good acts can be trusted to cooperate If player I cooperates. her payoff is a. whereas player 2 receives in important situations, whereas those who look cannot We b. If player I defects. her payoff is either cr or ca, depending on find that evolutionary dynamics can lead to cooperation whether the temptation is low or high, respectively, whereas player without looking at costs. Our results illuminate why we attend 2 receivesd. We have thc following inequalities: ca > q> a> 0 and closely to people's motivations for doing good, as prescribed b>0>d. Moreover...we have pb + (1 —p)d O. Therefore. player 2 by deontological ethicists such as Kant, and, also, why we prefers not to interact with a player 1 who only cooperates when admire principled people, adhere to taboos, and fall in love. the temptation is low. Finally. wc assume that low temptation is more likely than high temptation: p> 1/2. Author tontaouticm M.N, E.Y., end IA A N designed restock performed research and To understand thc essence of the game. we need to consider wear the paper. four strategics for player 1 and three strategies for player 2. The The mothers dedere no conflict of interest player 1 strategies am (i) CWOL, (ii) cooperate with looking Thiswild, is e PNAS 0irect Submission. (CWL), (iii) look and cooperate only when the temptation is low. Freely wallal. °name bough the PNAS open aaess opton. and (iv) always defect. The player 2 strategics are (i) end if player 'To wheal correspondence Avid he addressed ETNI: maronficwakeharvard.odu. 1 looks, (ii) end if player 1 defects, and (iii) always end. In SI This article conteire 'updating information online e wAw gine of9gooktokupe1 4 / 104 10. Appendix, wc also explore a richer strategy set. iofirpos.1417904112/4/Gupplemostat vawa.pnas.orgicgirdeirl 0.1073/pnas.1417904112 PNAS Early Edition I 1 of 6 EFTA00632631  Temptation to Defect Low 'e4 High (1) (2) (3) (4) Fig. 1. The envelope game. Cl) The game begins when the temptation to defect is randomly chosen, as indicated by a notice being placed in the envelope. The temptation to defect is low with probability p and high with probability 1 - p. (2) Player 1 receives the envelope and chooses whether to look (open the envelope). (3) Player I decides whether to cooperate or defect Player 1can only condition her action on the realized temptation if she has looked. Each time that player 1cooperates, regardless of whether player I looked, both players benefit from the interaction: player 1 gets a > 0, and player 2 gets b> 0. Player 1 gains even more if she defects. If the temptation is low, player I gets q> a, and if it is high, player 1gets ca >q. In either case, each time that player I defects, player 2 is harmed and gets a negative payoff (cl <0). Moreover, we assume that the harm is substantial Cal c -bp1(1 -p11, and therefore, player 2 prefers not to interact with a player 1 who only cooperates when the temptation is low. (4) Player Z having observed both of player l's choices, decides whether to continue or end. If player 2 continues, there is another round with probability w. ALLD, CWOL, and CWL are the only equilibria of the envelope type 1 can adopt one of four strategies described above. Players game. even with this richer strategy set. of type 2 can adopt one of three strategies described above. Our Second, because player 2 does not directly benefit by attending state space is the product of the simplex Sy and the simplex Sa. A to looking, she might not do so. This concern proves moot. The point in the simplex Sy describes a strategy mix of type 1 players. A intuition is that if there is even a small probability that player 1 point in the simplex 53 describes a strategy mix of type 2 players. looks. player 2 is better off attending to looking. We formalize We randomly seed the strategy frequencies many times and re- this intuition by showing that this equilibrium, as well as ALLD cord the frequency of each strategy after the population has sta- and CWL. can be made subgame-perfect, which is a solution bilized. We observe three possible outcomes that correspond to the concept used to rule out these kinds of concerns (15). Nash equilibria deserted above (Fig. 2). (i) Type I players con- In many cases we do not consciously avoid looking or distrust verge to always defect. whereas type 2 players converge to a tri- those who look. but. rather, are guided to do so by a gut sense, an angular region close to always end. (ii) Type 1 players converge to emotion, or an ideology. That is. looking feels or is wrong. CWOL whereas type 2 players converge to a mixture between end Where do these emotions and ideologies come from? Individuals if player 1 looks and end if player 1 defects. For stability, this mix- do not adopt them rationally or even consciously. Therefore, we ture must contain a minimum fraction of end if player 1looks. (iii) now consider the case where strategies (such as. it feels wrong to Type 1 players converge to a mixture between CWOL and CWL, look) are learned or evolved. whereas type 2 players converge to end if player 1 defects. The We use the replicator dynamic. which is the standard model dynamic stability of those evolutionary outcomes coincides with the for evolutionary dynamics (16-18). and also described rein- criteria for the underlying strategy pairs to be Nash equilibria forcement learning and prestige-biased imitation (19). The rate We now apply the model to shed light on some questions di- of reproduction is proportional to the payoff that a strategy rectly related to cooperation. receives. Because we have two types of players. our simulation First, psychologists and philosophers have long asked the studies coevolutionary dynamics in two populations. Players of following question: is helping others - always and exclusively Table 1. Payoffs for a restricted set of strategies in the envelope game Player 2 Player 1 End If Player 1 looks End If Player 1 defects Always end CWOL TWs a,b * CWL a,b Tfre • eV; ab Look and cooperate only 4P+40 - PMN3 +d(t - P) Ite•41.- 0 aP+40 - PhbP+ - P) when temptation is low Always defect co+ chit —Aci 412+ bin - PLd op+ chil—Acil Player l's strategies are presented in rows, and player Zs strategies are presented in columns. The payoffs at the intersection of a given row and column ate those that the players receive if they play the corresponding strategies. For example, if player 1 looks and cooperates only if the temptation is love and player 2 ends if player 1 defects, then player l's expected payoff is lap+ Ca (I -p)j/il - pwl, and player 2's expected payoff is lisp+ 4(1 -pjj, - pry . Details of regulations leading to payoffs are in SI Appendix. DepencIng on the parameter values, there are up to three Nash equilibria. The pair (ALLD) is always a Nash equilibrium. The pair (CWOL and end if player 1 looks) is a Nash equilibrium if a/(1-w)> qp+ ch(1 -p). The pair (CWL and end if player I defects) is a Nash equilibrium if .1(1- ta)> q,. We refer to these strategy pairs as ALIO, CWOL, and CWL, respectively. 'Nash equilibria of the envelope game. 2 of 6 www.was.wgicgifd:010.1073/was.1417604112 Hoffman et aL EFTA00632632  Equilibrium Classification Player 1 Player 2 041-w) (per • (1-Nol(14.) 1.0 o.e cg 0.6 All D: CWOL E 0.4 U. 0.2 00 1.0 g 0.8 a) 0.6 CWOL: a) 0.4 0' LL 0.2 00 T. t, Imalk " I PL oak CWL: 00 Payoff a Fig. 2. Learning dynamics of the envelope game. We randomly seed the strategy frequencies 10,000 times for 50 values of the payoff value a and record the frequency of each strategy after 1,000 generations. We observe three possible outcomes that correspond to the Nash equiltela identified in Table I. (i) Type I players converge to always defect, whereas type 2 players converge to a triangular region close to always end. Cu) Type 1 players converge to CWOL, whereas type 2 players converge to a mixture between end if player I looks and end if player 1 defects. For stability, this mixture must contain a minimum fraction of end if player I looks. Oil) Type I players converge to a mixture between CWOL and CWI., whereas type 2 players converge to end if player 1 defects. We vary the value of a along the x axis. They axis represents frequencies., and each colored line presents the frequency of each outcome. The parameter region where the corresponding strategy pair is supported as an equilibrium is shaded. Additional detaib are in Si Appendix. All D, all defect; C, converge. motivated by the prospect of some benefit for ourselves, however heuristics to avoid incurring cognitive costs associated with de- subtle" (20) [for example, the conscious anticipation of feeling liberation (29-31). In a world with repeated interactions, it is good (21), avoidance of guilt (22-24). reputational benefits, or usually worthwhile to cooperate. and therefore, individuals may reciprocity (1-14)j. At the extreme, this question amounts to ask- adopt heuristics. such as always cooperate or always cooperate in ing if saintly individuals, such as Gandhi or Mother Teresa. were nonbusiness settings. These same individuals, when serving as motivated thus or if they were authentic altruists who did good laboratory subjects. may apply these heuristics and cooperate even without anticipating any reward and would be altruistic. even in when it is not worthwhile to do so (25, 32.33). the absence of such rewards. Our model suggests that authentic Our model offers an additional explanation for intuitive co- altruism is. indeed. possible: by focusing entirely on the benefits to operation: intuitive cooperation may serve to reduce respon- others, authentic altruists are trusted more, and the benefits from siveness to costs of cooperating in a particular situation. For this this trust outweigh the risk of. for example. dying a martyr's death. explanation to be sensible, it must be the case that whether Second, we address another question of why people are intuitive a decision is made intuitively or deliberately is detectable. In cooperators. That is when people decide rapidly. they are more fact, it is: deliberative decision-making leads to slower reaction likely to cooperate than if they have time to deliberate. Addition- time as well as increased pupil size and heart rate (34) and ally, people who cooperate decide more quickly than those who sometimes, blushing or stammering (35). Our model makes two defect (25-27). Intuitive cooperation underlies extreme acts of predictions that one would not make if the Social Heuristics heroism, acts which, because they place the hero at great personal Hypothesis. by itself, explained intuitive cooperation: decisions risk, are otherwise difficult to explain (2S). The Social Heuristics related to cooperation are more likely to be intuitive than other Hypothesis offers one explanation for this phenomenon: we adopt decisions that are similarly usually worthwhile, and intuitive Hoffman et al. PNAS Early Edition 1 3 of 6 EFTA00632633 cooperators are trusted more than reflective cooperators. There every• decision and will defect when faced with a large temptation, is evidence for the latter: in an experiment eliciting moral whereas someone who is guided by principles is less sensitive to judgments. subjects who read vignettes about people who re- the costs and benefits to themselves and thus, less likely to defect. turned lost wallets judged those who returned the wallets without Imagine that our flip-flopping politician was once against gay hesitation more positively than those who hesitated (36). marriage but supports it now that it is popular. That he only sup- Third. we address the question of why people cooperate in one- ports it when it is popular indicates that the politician is unlikely to shot situations (for example, in laboratory experiments, such as the fight for the cause if it later becomes unpopular with constituents or dictator game) (37). Cooperation in these situations is waling from risks losing a big donor. Note that, not only will gay rights activists the standpoint of models of the evolution of cooperation based on distrust the flip-Hopper but also. women's rights activists will dis- reciprocity. because in that framework. cooperation can only im- trust him, even if the flip-flopping politician has always supported prove one's reputation if actions are observed, and reputations are women's rights, because the flip-Hopper would be likely to end his only valuable if others have an opportunity to reciprocate. Some support for women's issues if it is ever advantageous for him to do have suggested that cooperation in these settings results from so. Of course, we do want our politicians to be strategic about some intuitive cooperation (25, 32), that it can emerge if there is un- things. For example. we would prefer that they carefully consider certainty over the probability of future cooperation opportunities fatalities before invading a foreign country•. Our model suggests (38). or that it is a consequence of the evolution of altruism that we would like pulitkians—and others more generally—to be caused by group selection (37). We offer another potential ex- strategic about the costs and benefits to us (fatalities) but not the planation by considering a variation of our model, in which the costs and benefits to themselves (likelihood of getting reelected). likelihood of continuation varies and player 1 learns this likeli- Our model also teaches us when we will not be bothered if hood when she looks (details in SI Appendix). This model sug- others are strategic: when defections are either not especially gests that subjects learned or evolved to not consider who is tempting top + ch(1 —p)≤a /I —w] or not especially harmful watching so that others can expect them to cooperate. even when [bp + d(1 —p}<0[b]. Contrast the flip-flopping politician with no one is. Unlike the other explanations cited, our explanation a business partner who might have the opportunity to cut you out implies that, when there ends up being an opportunity to interact of your latest deal. As long as such a temptation benefits your again, those who cooperate when they thought no one could partner little relative to losing a valuable long-term partnership, reciprocate will be rewarded more than those who cooperated knowing someone could reciprocate. Indeed, laboratory subjects your partner would never be tempted. and you need not be cooperate more with those who cooperated with a third party bothered if he is strategic. under the presumption that no one would have a chance to re- Next, we discuss why we feel moral disgust by those who use or ciprocate (39). Moreover, our explanation uniquely predicts that manipulate others, as famously condemned by Kant in his second people would feel wrong attending because of the fact that the formulation of the Categorical Imperative: - Act in such a way situation is one shot and that others will judge them harshly if that you treat humanity ... never merely as a means to an end, they behave differently when the situation is one shot. but always at the same time as an end" (46). Consider the well- Fourth, we address the question of why we find it unbecoming known example of dwarf-tossing. Many sec it as a violation of when dose friends keep track of favors or reciprocate favors im- dwarves' basic dignity to use them as a means for amusement. mediately. In experiments, subjects ) do not cooperate more with although dwarves willingly engage in the activity for economic friends who have just given them a gift but do cooperate more with gain. Our aversion to using people may explain many important strangers who have just given them the same gift (40). (ii) take aspects of our moral intuitions, such as why we judge torture as greater care to highlight contributions to strangers than to friends worse than imprisonment or punishment. Our model suggests (41). (iii) are offended when close friends immediately reciprocate that we are repulsed by those who treat others as a means to an kind acts but not when strangers do (42). and (iv) judge friendships end, because they are liable to mistreat their relationship part- as less close when those relationships display immediate reciprocity ners when expedient. even if, currently, the relationship is (43. 44). In fact. relationships have been shown to fall into distinct mutually beneficial. categories. in part characterized by whether favors are tracked (45). The previous two applications are examples of a more general These observations have led researchers to conclude that - the phenomenon: that we judge the moral worth of an action based dynamic of friendship does not fit the logic of models of reciprocity on the motivation of the actor as argued by deontological ethi- and presents a (mini,- for evolutionary• analysis" (43). However, our cists but contested by consequentialists. The deontological ar- model suggests an explanation that is consistent with reciprocity. If gument is famously invoked by Kant (46): close friends CWOL their decision to cooperate is affected not by Action from duty has its moral worth not in the purpose to be a single. recent kind act but rather. only by the distribution of attained by it but in the maxim in accordance with which it is decided payoffs from the relationship in the long run. Moreover, when the upon. and therefore does not depend upon the realization of the beneficiary• of a good deed immediately reciprocates. then either object of the action but merely upon the principle of volition in ac- the beneficiary is looking or the beneficiary thinks that the friend cordance with which the action is done without regard for any object who did the good deed was looking. of the faculty of desire. Fifth. our model gives insight on a number of interesting phenomena not prima facie related to cooperation. These applications illustrate that we attend to motives because Why do we like people who are principled and not like those they provide valuable information on whether the actor can be who are strategic? For example. we trust candidates for political trusted to treat others well, even when it is not in her interest. office whose policies are the result of their convictions and Next, we consider why people dislike considering tradeoffs consistent over time, and we distrust those whose policies arc related to "sacred values" (47). Sacred values are values, such as carefully constructed in consultation with their pollsters and who love. liberty, honor, justice. or life, that people treat - as pos- flip-flop in response to public opinion (as caricatured by the sessing transcendental significance that precludes comparisons. infamous 2004 Republican presidential campaign television ad- tradeoffs, or indeed any mingling with secular values•' (47). Al- vertisement showing John Kerry windsurfing and tacking from though there is variation in what societies consider sacred. vir- one direction to the other). Instead of respecting politicians who tually all societies have a concept of sacredness (47). Sacred flexibly respond to public opinion, we view them as sleazy. values are so strongly imbued in us that we do not find them Our model offers the following potential explanation. Someone puzzling prima fascia. but their existence and origin remain who is strategic considers the costs and benefits to themselves of poorly understood. What makes us treat some values as sacred. 4 of 6 www.ecas.oigkgifdo110.1073/plas.1417904112 Hoffman et S. EFTA00632634 and what differentiates these values from secular values. like free evolve to be in love today and defect tomorrow. Our model time or money. that we more readily trade? requires a different and we think, more realistic constraint: that Our model provides one possible explanation. People who it is impossible to look while one appears as though not looking. calculate costs of trading off against sacred values are less This assumption is justified by the fact that, at least in some trustworthy when it comes to safeguarding these values than contexts, gathering information about the costs and benefits is people who consider them sacred and would never calculate the inherently observable (for example. through reaction time or the costs of trading off against them. Responding with disgust to questions that one asks). these taboo tradeoffs may be one way to prevent us from inter- Existing evidence is consistent with both models: emotions acting with people who make such tradeoffs and hence. are less related to love are observable (53). cannot be faked (54). and are trustworthy and, also. may be a way to signal to others that we relied on by partners when choosing whether to cooperate (55. ourselves would not consider and therefore, make such tradeoffs. 56). There is also reason to believe that love and related emo- Consistent with CWOL, it is taboo to consider the tradeoff. even tions would be hard to fake given their autonomic origins and the if one ultimately makes the right choice. and the longer the costs of placing their activation under conscious control (35,49). tradeoff is considered for, the harsher the judgment by observers It remains to be shown that love, in particular, has these attrib- (47). Importantly, those who consider a taboo tradeoff, such as utes and that we cannot evolve or learn to display love while still selling their own child, pay a reputational cost, because such attending to costs, Thus, additional research is warranted to considerations indicate that one, in general. does not hold sacred differentiate between the model by Frank (49) and CWOL. values and cannot be trusted with, for example. care of other•' Consistent with CWOL, mere discussions of the costs and children, the elderly, or shared resources. benefits of a relationship or a breakup (for example, suggesting If CWOL. indeed, underlies the phenomenon of taboo trade- a prenuptial agreement) damage the relationship. Such dis- offs, then it provides two predictions. First, taboo tradeoffs will cussions indicate that one is looking at the costs of the re- prevail precisely in situations where there is large but infrequent lationship and cast doubt on one's commitment. CWOL also temptation to defect and defection is harmful. such as selling elucidates that falling in or out of love depends on the distribu- a child, betraying a country. or sleeping with someone for tion of temptations but not their immediate realizations, sug- I ce a million dollars. It remains to be shown that taboo tradeoffs gesting that people will fall out of love when there is a permanent show these characteristics. Second, it also provides an important change in alternative mating opportunities or relationship casts policy prescription regarding policies forbidding taboo tradeoffs but not when there is a one-off temptation. For example. one may (for example. the ban on euthanasia): such policies are socially fall out of love with one's partner after becoming unexpectedly suboptimal. because the benefits of cooperating without looking successful. Finally, CWOL clarifies that love comes with a cost— accrue to the individuals who advocate them, but the costs are the cost of ignored temptations—and suggests that this cost must borne by society. We note that the above arguments extend to be compensated for with commensurate investment in the re- taboos in general and explain why they often have the property lationship. Only sometimes is it worthwhile for the recipient of that it is not merely a transgression to violate the taboo but to love to compensate a suitor. which explains why people actively just consider violating it (48). avoid the strung affections of those with whom they do not wish Finally, our model offers an explanation for emotions, such as to have long-term relationships. love, which is closely related to the explanation first proffered by These arguments extend to anger. Anger can be thought of as Frank (49) [precursory insight is given in the work by Schelling - punishing without looking.•' It prevents people from looking at (50); also see the works by Hirshleiter (51). Pinker (35). and the costs of inflicting harm on others after a transgression. Winters (52)]. Love has the property that we behave altruistically thereby deterring future transgressions. toward our partners, regardless of what temptations arise (49). as This paper formalizes a simple intuition first spelled out by illustrated by the wedding vow - for better or for worse, for Trivers (I): richer, for poorer, in sickness and in health.•' For example. love causes individuals to ignore other potential mates, even if those One can imagine. for example. compensating for a misdeed without mates are better than one's current mate, as Shakespeare's Juliet any emotional basis but with a calculating. self-serving motive. Such an individual should be distrusted because the calculating spirit that did when her love for Romeo led her to rebuff the advances of leads this subtle cheater now to compensate may in the future lead the otherwise more suitable Paris. him to cheat when circumstances seem more advantageous (because Why does love have this property? Our model suggests that of unlikelihood of detection, for example. or because the cheated those who are blinded by love can be trusted to stay with their individual is unlikely to survive). partners in sickness and health. because they are not looking at the costs of cooperation in these diverse situations. This expla- We hope that formalizing this intuition has added valuable in- nation for love is different from the explanation by Frank (49). sight on otherwise puzzling aspects of human nature. Frank (49) argues that those who are blinded by love observably ACKNOWLEDGMENTS. This research was funded, in part, by John Templeton commit to staying with their partners That is. those who are in Foundation Grant RFP-12-11 from the Foundational Questions in Evolution- love today do not have the option to defect tomorrow. The ar- ary Biology Fund, National Science Foundation Grant 0905645, and Army gument by Frank (49) has been criticized because one could Research Office Grant W91INF-11-1-0363. I. Wren 81(1931) The evolution of fanatical altruism. 417 Roy tb.0 8511335-57. 9. Aumann ID. shame, LS 119941 long-Term Companion an Game.Thcatetk Analph 2. Friedman 1W (1971) A non-cooperative equilibrium for woectornm Rev (con Stud (springer. Berleil. 38(1)1-12. 10. Now* IAN Sigmund K (199() Evolution of incited reogrotity be image scoring 3. Axelrod RM (198,1) The (volution of Cooperation Manic Books. New York). Nita 393(6685):573-577. 4. fudenberg D, Maskin E 11986) The folk theorem in repeated games with dscounag 11. Nowak MA. Sigmund K (20051 Evolution of inbred retipraily. Neese 43717063): or with incomMete information. Econometrka 540)533-554. 1291-12% & fudenberg D. Mask n E ( 990) Evoluton and cooperation m noisy repeated games. 12. Nowak MA 0006) Pa nies for the evolution of cooperation. Science 314158058 Am Leon Rev SOO:274-279. 1560-1563. 6. (drew, K4 Sanunbon I. (19921 Evolutionary stability in repeated genies played by 13. Ohtsulr.I N, Mesa V (2006) The leading eight: Sodal norms that can maintain co. finite automata. I Icon Theory 57:278-305. operation by indirect reciprocity. 1 MOP gio)239141435-444. 7. Him* MA. Sigmund K 11992) Ti for let in heterogeneous Populatioiwalarive 355: 14. Sigmund K 11010) The Crittuko of Serfishnen (Princeton Univ Press. Princeton/. 250-253. IS. Osborne M1(20031 An Infioduction to Come Meow Oxford link. Press• Hew TON. & Hawk PA, Sigmund K (1993) A strategy of win.stay, lose4rft that outperforms lit. I& Hofbauer 1, Sigmund K (1998) Evolutionary Games and PopuAnlon Oynamin for.tat In the Prisoners Dilemma game NOW. 361(64321:56-58. (Cambridge Univ Press, Cambridge, United Kingdom). Hoffman et al. PAS Early Edda, I S of 6 EFTA00632635 17. Weibul SW (1997) fmkeionary Game Theory DAR Press, Cambridge. MN. 38 Deity" AW. Kresnow MM. Cosmides 1. Took 1120111 Evolution of direct rmitsrodlY IL Nowak MA 120061 Evolvtionary Lynam.= &profile the Equations of Lob (Harvard under uncertainty can explain human gemoosity n oneshot encounters. Proc Nail UMW Press. Cambridge. MA). Aced ()SA 10802)13335-13340. 19. Fudenberg DA (1996) The Theory of Learning in Gamer (MIT Press. Cambridge. MAL 39 Lin It Ong D1201110mersingaltrtionv Type Preferences in the Leboretory. Awrbbre et Vol 2. ices.gmu.odunsp-contentkoploadv2012/01/Deserving-MtruernType.Preferences.in. 20. Batson CD (2014) The Aftnai.vn Question: reword s Sociol.esychologioif Answer theleboratorphy.Ongond-Lin pdf. Accessed January 12.2015. (Psychology Press. Hew York/ 40 Baler Fl. Rodriguez A Ow MG. Mershell L (1995) The relative effechkeness of 21. And eon 1( 990) trope e altruism and donations to public mice A theory of warm. a deed request message and a megMrig message on friends and strangen. Corn- glow giving (con ) (London) 10014011464-477. manic Res 22(4475-484 41. Mills J. Clark MS 11994) Communal and exchange relationships: Controversies and 22. Dana 1, Cain DM, Dawes MA (2006) What you don't know wont hurt me: Costly (but research. Theoretical Frameworks for Personal liotationsNps. ads Mbar R. Gilmour R quiet) exit in dictator genres. Organ Sehev Num Danis Process 100193-201. (Lawrence ErBoeurn Assodeles. Inc. Hillsdele. NO 23. Dellelligne S. List /A. Melinender U 42012) Testing for statism end social pressure in 42. shackellowdllA Buss DM (1996) Betrayal In rnatedups, friends/roc and coalitions. Pen charitable ghing. 0 ) (con 127(01-56. Soc PlyChOrika22(113:1151-1164. 24. Cain DM. Dane J. Newman GE 0014) Giving versus giving in. Aced84, W Ann (Mk 43. Silk /8 (2003) Cooperation without counting- The puzzle of friendship. Genetic 505-533. and CuMmai froamon of CooparatiorL ed Hammerstein P (1ST Press, carnerkga, MAL 25. Rand DG, Greene JO, Novak MA (2012) Spontaneous giving and calculated greed. pp 37-54. Name 4890416)427-m 44 Pinker S. Nowak MA Lee JJ Wag The logic of indirect speech. Prot NatAced ki USA 26. Nielsen UN. Titan JR. Wengstrarn E 0014) Second thoughts on free riding. (con Len 10501833-838. 122128136-139. 4S. Fide AP 11992) The four elementary harms of sociality framework for a united 27. Rubinstein A (2014) A typology of Miners, Between instinc6ve end contemplative. theory of nodal rotations. Psycho/Rev 99(4)489-723. Available at www.arlelrublmtein.tau.acilfpapersrtypology.pdf. Accessed January 46. Kant I (2002) Groundwork for the Metaphysic, of Mora ed Wood AN(Yale UAW PPM, 15,2015. Hew Haven, CO. 28. Rend 0G. Epstein 26 12014) Risking your life without e second thought Intuitive 47. Tetlo PE (2003) Thinking the uMhintet4e: Sacred values and taboo cognitions decisionmatung and extreme altrtism. Available at suncomiabstracte2424036. Trends Coen Sc) 717/320-324. Accessed January IS, 2015. 48. fenhtman C. Gaiety U, Hoffman M 120111 Taboos and identity: Considering the 29. Simon HA (1955) A Wier:aced model of ratio.* choke. 0 ) (con 690119-IIt unthinkable. An (con I Afivoeten 312)119-164. 30. Trotsky A. Kahneman D (1974) Judgment under uncertainty Heuristes and biases. 49. Frank RH 41911111 Passion Within Peyton: The Strategic Rob of the Emotions PAW Horton 16 Co., Hew York). Science 18501571:1124-1131. 50. Schelling TC (19030) The Strategy of Confect eleverd Univ Prep, Cambridge. MN. 31. Kahnemen D. Slavic P. Tvenky A 4191la Aelpment Wider Uncertainty: Heuristic and 51. Hirshleiter I (1967)On the Emotions as Gulfantort Of Threat andProrMeriMIT PIO% Biases (Cambridge WIN Press, Cambridge, United Kingebm). Cambridge, MN 32. Rend D. et el. (2013) Intuitive cooperetkm end the social heseistks hypethesh: 52 VAnten E 12014/ feeling Smart. Why Our &notions Are More Rational Then We Think Evidence from 15 tine constraint studies. Available et sun comfebstred4222683. (Public Athos. New York). Accessed January IS, 2015. 53. Ekman P. Sorenson ER Friesen VN (1969) Pal cultural elements In facial denims of 33. Rend EIG, et el. (2014) Social heuristks shone intuitive cooperation. fiat Continuo emotion. Science 164(36751'.86-88. 51201483677. SA Hunan Pi Davidson N. Friesen WV119901 The Mechem» smile: Emotional repression 34. Tursky B, Shapiro D. Older A, Kahneman D11969) Funnier/ heart rate, and skin re• and bah physiology. II. / Pars Soc Psycho/ 58(2/342-353. 'Mane Menges during a mental talk. Et‘o hydro( 79113:164-167. SS. Reed It Zeglen KN. Schmidt Ki. (2012) Facial expressions es honest Pgnels of co. 35. Pinker 5 (1997) How the *NM Weds (NortoA New York). operetta Intent In a one-shot anonymous prisoners dilemma game. (volNun,!May 36. Catcher CR. Mbar Y. Paarro DA (2013) How quick decisions aire,Mbte moral char. 330)200-209. ester. Soc hydro! Personal So 401108-315. 56 Reed M DeSciali P. Pinker SA (2014) The commitment function of eery facial 37. Feb E, fischbather U 0003) The neon of human attruern.Nana 425169608785-791. expressions. Psycho' So 230):1511-1517. 6 of 6 ' www.pnastaggfcgifdo110.1073rptes.1417904112 Hoffman et aL EFT