Monday, August 31, 2015

Forecasting decisions in conflict situations: A comparison of game theory, role-playing, and unaided judgement

Summary:

This article compared the accuracy of game theory, role-playing, and unaided judgement in regards to forecasting decisions . The author provided much background information on the subject, including predicting the behavior of participants in the new competitive market for wholesale electricity after the New Zealand government transferred assets to a new private sector electricity company called Contact Energy Ltd. in 1996. When the managers employed the role-playing method, the results were not consistent with the executives' beliefs so they turned to game theory for answers. What they found out was that the game theory method ended up not being helpful at all and that role-playing accurately predicted the behaviors.

The article continues to mention other tests that have been performed by many other people which ultimately led the author to determine that in regards to predicting human behavior, role-playing is the most accurate, game theory is the second most accurate, and unaided judgment is the least accurate. This is due in part to the fact that game theory cannot take into account the complexities of situations like role-playing can. According to the article, there is also not much proof to suggest that predictive validity for game theory in real conflicts exists since it is normally tested using role-played conflicts. Basically, role-playing is a more accurate method to use to determine human behaviors because there is a much greater degree of realism.

After doing this background research, the author decided to run their own experiment by creating six conflicts to test the accuracy of game theory, role-playing, and unaided judgment. To do this, the author created six conflicts that would be attempted to be solved by the participants using one of the three methods that were being compared. The results ended up being consistent with previous research which means that the author's experiment also found that role-playing is the most accurate method, game theory is the second most accurate method, and unaided judgment is the least accurate method to forecast human behaviors (conflict resolutions).

Critique:

In general, the experiment that was created and ran by the author was not very controlled. As a psychology major as well, this really bothered me. The first issue that I noticed was that the conflicts presented in the experiment were not all made up. Some of the situations came from previous research, one came from television, and another one was a real life situation from a company. The author claims that these situations were probably not going to be recognized by the participants, but if they were, then there could be problems with the accuracy of the experiment. In addition to this, nothing in the experiment (methods used, time given to find a solution, which situation the participants needed to solve, and more) were assigned to the participants randomly. Various other aspects of the tests were not controlled as well.

Source:

"Forecasting decisions in conflict situations: A comparison of game theory, role-playing, and unaided judgement"
By: Kesten C. Green
http://www.umsl.edu/~sauterv/DSS/green.pdf

4 comments:

  1. The author's experiment is the first thing that jumped out at me as well, and your critique pretty much summed up my thoughts on the validity of it. There needed to be much more consistency and randomization throughout the scenarios and groups for the experiment to yield meaningful results.

    ReplyDelete
    Replies
    1. I definitely agree with you there. I really would have liked to see the experiment set up in a much better way. The psychology major in me was pretty disappointed by this study, but I still thought it was an interesting read and could have yielded much more valid results had it been set up properly.

      Delete
  2. Does author detail that how role playing take into account the complexities of the situation and the game playing doesn't? I mean what is the biggest driver that creates the difference between two techniques?

    ReplyDelete
    Replies
    1. The author doesn't go into a ton of detail about this unfortunately. Instead he just talks about many different studies that looked into game theory and how most didn't mention any type of predictive validity. The article basically just states that role-playing is better when predicting human behaviors (such as decisions) because it takes emotions into consideration and game theory does not.

      Delete