Wednesday, September 3, 2014

Behavioral decision making, forecasting, game theory, and role-play



Summary:
In 2002, Green presented evidence that role-playing is a more valuable forecasting tool than game theory.  Since that publication, game theorists have defended their practice,  desperately asserting  that role-playing should not be used as a lone forecasting tool.  As Shefrin points out, game theory ignores the flaws in human judgment.  The forecasting value of role-playing increases if the session, or game, occurs in conditions in which irrationality is expected.  According to the neoclassical view of economics, decision makers always strive to make perfectly rational decision.  However, this view assumes that the decision maker has all available and perfect information guiding his decision.  The relaxed focus on perfect rationality in role-playing more accurately reflects decision making in a world of imperfect information. 

Consider a $1 English-style auction in which the winner receives the $1 and pays his last bet while the loser receives nothing but still has to pay his last bet too.  The betting precedes in $.01 increments.  When this scenario was used in an experiment at Harvard Business School, betting reached $1.  The individual with the last bet reasoned that betting $1.01, winning $1.00, and ultimately losing $.01 was better than losing $.99.  However, that bet triggered and ego- and emotionally-driven race.  The experiment was terminated when betting reach $3.10.

Shefrin asserts that role-playing more accurately portrays and forecasts at least three cognitive biases.  The first is bounded rationality.  Bounded rationality leads decision makers to be narrow-minded.  Contrary to what game theory assumes, humans do not consider every single consequence and/or outcome of their decisions.  Humans will act accordingly to their emotions as they process the limited information they are given.  The second bias is loss aversion.  Humans will accept mathematically and economically irrational offers or bets in order to avoid financial or emotional loss.  The third bias is “overevaluation.”  Humans can incorrectly estimate how much they are losing when they escalate a situation relative to the adversary.  The dollar auction experiment also illustrated this bias.  Players continued to escalate bidding despite only losing $.01 than the other party up to a $1.00.  Shefrin concluded that a game theory model that accounts for cognitive biases is the ideal forecasting tool.

Critique:
While most analytic methods operate under perfect logic with complex calculations and geographic analysis, role-playing is valuable precisely because it embraces irrationality.  It is irrational people with limited information portraying irrational decision makers with similar, if not identical, information. 

If a behavioral game theory model were developed, it would save analysts valuable time.  Admittedly, a model such as this has more potential in the business realm, where situations are less complex than those involving state armies, varying degrees of political consequences, and completely irrational actors (see North Korea).  Using a statistical model to shorten or eliminate the time needed for role-playing sessions is desirable, but unlikely to reflect decision making by imperfect people with 80%+ accuracy.

Source
Shefrin, H. (2002). Behavioral decision making, forecasting, game theory, and role-play. International Journal of Forecasting, 18, 375–382. Retrieved from http://forecastingprinciples.com/files/Shefrin%202002.pdf

4 comments:

  1. You state that one of the strengths of role playing is its tendency to mimic the biases that we, as humans, have during the real situations that the sessions attempt to mimic. As analysts, we learn about these biases and learn how to recognize and mitigate them. Do you believe, based off this research, that role-playing scenarios would be best played out by participants who are not analysts who know of these biases?

    ReplyDelete
    Replies
    1. I would assume that decision makers do possess higher analytical skills than that of a lay person. I would guess that role-playing would be more accurate if analysts did the work - but I have not seen research to back that up.

      Delete
  2. Do you think there is any other biases besides the three outlined in this publication that could prevent students and role-players from obtaining valuable results? It was interesting to see the results since Armstrong concluded that role-playing produced correct predictions for 56 percent verses Green's 80 percent.

    ReplyDelete
    Replies
    1. I am sure there are plenty of biases that affect judgment. I recommend Daniel Kahneman's book Thinking, Fast and Slow for an extended list of decision making biases. I am certain there are many others. To be honest, I was uncomfortable with only three being explored in this article. However, extensive articles are tougher to get published.

      Delete