Introduction:
In this article, J. Scott Armstrong reviews years of research on role playing as a method for accurately forecasting decisions and actions by both individuals and groups. Starting with why role playing should work and the basic elements of role playing, Armstrong reviews several experiments and studies that tested the effectiveness of this technique, focusing on five Armstrong conducted. In this version of role playing, participants take on the roles of specific actors or groups and the “decisions” they make are used as forecasts for what real life actors will do.
Summary:
Armstrong, looking back to over forty years of research, notes how subjects presented with the same data but different roles will make significantly different decisions. When several parties are involved in an action-reaction situation, this makes forecasting even more difficult. Role playing allows forecasters to simulate the interactions of these parties, making the role playing a powerful tool for forecasting.
Armstrong then lays out several key elements of role playing. First, realistic casting is helpful but not necessary while using numerous people to represent specific groups encourages improvisation. Second, analysts must describe the roles to participants before the role play while encouraging improvisation and asking them to act as the actors they represent would. Third, analysts should describe the situation accurately and briefly while specifying possible decisions and providing surroundings as realistic as possible. Fourth, analysts should have coding or multiple coders to properly understand the resulting decisions while multiple sessions provide for generating more accurate results. Finally, role playing should be used when there are multiple parties in conflict with the potential to generate significant changes.
While Armstrong cites many studies, he focuses on his own studies to demonstrate the potential accuracy of this method in forecasting. In one study, role players acted as a board of directors for a pharmaceuticals manufacturer whose drug, Panalba, had been recommended for withdrawal by the FDA. Out of 83 predictions, 76 percent accurately forecasted that the drug company would continue marketing the drug while only 34 percent of experts forecasted that outcome. In a distribution plan between a manufacturer and retail store chain, 75 percent of role playing groups forecasted that the store would accept a discount deal offered by the manufacturer while only 3 percent of experts predicted this outcome which actually occurred. In a study on the actions of Dutch artists, 29 percent of role playing sessions accurately predicted that the Dutch government would give in to artists’ demands during a museum while only 3 percent of experts
predicted this outcome. In an additional case on a royalties and contract dispute between the editors of an academic journal and the publisher, 42 percent of role playing sessions accurately predicted that there would be no resolution while only 12 percent of experts predicted such an outcome. In a final study on a dispute between the NFL and the NFL Players Association in
1982, 60 percent of role playing sessions resulted in a players’ strike while only 27 percent of experts predicted the outcome. This last case in interesting because the results of the study were published in 1982 before the players actually went on strike.
Key to all of these studies was that role playing allowed some replication of the actual environment and situation, leading to more accurate forecasting. Instead of just thinking about the situation, role players became involved in the situation and were able to discuss with each other in an interactive way and could dispute issues much like actual decision makers. In addition, the role players had a stronger vested interest in the specific view point of their
respective actors or groups as opposed to seeing the broader picture.
Conclusion:
Based on his own studies and multiple other studies cited in the article, J. Scott Armstrong
presents a compelling case for the effectiveness and accuracy of properly conducted role playing exercises in forecasting decisions in situations involving multiple parties in a dispute. The overall summary notes how role playing had a 56 percent accuracy rate (in Armstrong’s studies) while
consulting expert opinions resulted in only a 16 percent accuracy rate. Role playing’s ability to replicate interaction and, to some degree, conflict between vested parties and actors creates a more realistic environment for studying real life situations, conflicts, and disputes were these interactions play a key role in the final decisions taken.
The article is effective in laying out good ground work for when and how to use role playing
in forecasting decision making in conflict situations, making a useful tool for intelligence professionals. The arguments presented are thoroughly researched including several decades of studies by the author himself. The author presents the results of many studies beyond his own and a synthesis of best practices and situations for using role playing. While the technique is not directly compared to other techniques such as experimentation, game theory or intelligence techniques, Armstrong suggests that this is a weak point in the research and needs further study.
Source:
Armstrong, J.S. (2001) Role playing: A method to forecast decisions. In Principles of Forecasting: A Handbook for Researchers and Practitioners (Ed. J. Scott Armstrong). Kluwer.
Posts
No comments:
Post a Comment