Forecasting Decisions in Conflict Situations: A Comparison of Game Theory, Role-playing, and Unaided Judgement

Introduction:

In the International Journal of Forecasting, Kesten C. Green builds upon the role playing research begun by J. S. Armstrong. Written in 2002, this new study compares the forecasting accuracy of game theory, role playing, and unaided judgment when applied to conflict situations. Based on the results, Green concludes that game theory is more accurate than unaided judgment, and that role playing is more accurate than game theory when seeking to predict the outcome of conflict situations.

Summary:

Green defines accuracy as “the proportion of forecasts that match the actual decision.” In order to test the accuracy of unaided judgment, game theory, and role playing Kesten employed a total of six different conflict scenarios. Four of these (Artists’ Protest, 55% Pay Plan, Panalba Drug Policy, and Distribution Channel) were used by J. S. Armstrong during his research. The final two (Zenith Investment and Nurses Dispute) were developed by Green for the purpose of this study.

Unaided judgment participants were generally given a one to two hour time frame to complete their predictions. They were asked to review the full set of information for a single conflict scenario, and then choose the outcome they thought most likely from an exhaustive list. Game theorists were sent email instructions, along with all scenarios, and asked to complete all six. Finally, role players were given a single role in one of the two new conflict scenarios, and were then briefed on the scenario. 30 minutes to an hour were allowed for the role players to act out their scenario, at which time they recorded the decision the group arrived at, or predicted how they believed it would end in the event they were unable to come to a conclusion on time. The numerical results of the study are as follows:

	Chance	Unaided	Game	Role-playb
		judgementa	theorist
Artists’ Protest	17	5 (39)	6 (18)	29 (14)
Distribution Channel	33	5 (42)	31 (13)	75 (12)
55% Pay Plan	25	27 (15)	29 (17)	60 (10)
Zenith Investment	33	29 (21)	22 (18)	59 (17)
Panalba Drug Policy	20	34 (68)	84 (19)	76 (83)
Nurses Dispute	33	68 (22)	50 (14)	82 (22)
Totals (unweightedc)	27	28 (207)	37 (99)	64 (158)

Table 1. Accuracy of unaided judgement, game theorist, and role-play predictions. Percent correct predictions (number of predictions)

Given the totals, unaided judgment was accurate 28 percent of the time, and almost even with chance (27 percent.) Game theorists had an improved, if unimpressive, 37 percent accuracy. Finally, the role players averaged 64 percent accuracy and were better than chance and unaided judgment in all situations. They only scored lower than game theorists on the Panalba Drug Policy scenario. Game theorists, on the other hand, were less accurate than people using unaided judgment in 2 of the 6 scenarios, and were even with chance in 4 of the 6. Green suggests that further research will be necessary to determine if game theory expertise gives any advantage over the unaided judgment of experts on conflicts who are unfamiliar with game theory.

Conclusion:

Given the results of his study, Green concludes by agreeing with Armstrong that “role-playing will provide more accurate forecasts than other methods for forecasting decisions in conflicts because it provides more realistic representations.” In his assessment the cost of conducting these three forecasting methods is similar, which lends further strength to the role playing argument.

Source:

Green, K. C. (2002) Forecasting decisions in conflict situations: a comparison of game theory, role-playing, and unaided judgement. International Journal of Forecasting. Volume 18, Issue 3, July–September 2002. Pages 321-344. http://www.sciencedirect.com/science/article/pii/S0169207002000250