Role Playing: A Method to Forecast Decisions
By: J. Scott Armstrong
By: J. Scott Armstrong
Summary:
This publication outlines the basic elements of role-playing using a series of studies. The author conducted a study that had two groups of two participants each for 80-minute sessions. Each person handled one simulation as an expert and another as a role player. In each of the simulations, each person received closed ended questions ranging in possible decision making conclusions. In each of the role-playing simulations, participants were assigned background information to make the simulation sound realistic while viewing the problem from an outside perspective.
This publication outlines the basic elements of role-playing using a series of studies. The author conducted a study that had two groups of two participants each for 80-minute sessions. Each person handled one simulation as an expert and another as a role player. In each of the simulations, each person received closed ended questions ranging in possible decision making conclusions. In each of the role-playing simulations, participants were assigned background information to make the simulation sound realistic while viewing the problem from an outside perspective.
After reading the material for 20 minutes, each party would meet at a table until they reached a consensus or when the time ran out. After the role-playing was completed, each person answered a set of questions. These questions instructed them to state the consensus as they saw it or, if they did not reach a consensus, what would have happened if they reached a conclusion in the appropriate time.
Also, to reduce the chance of misinterpretation, participants were asked to write their views of the decision down independently at the end of the simulation. This is a great exercise to do since it forces people to agree on the underlying decision.
The author concluded that role-playing provides greater accuracy
when compared to expert opinions. Role-playing produced correct predictions for
“56 percent of the situation versus 16 percent for opinion.” In addition,
“role-playing produced outcomes that were similar to those from seven out of
eight experiments.”
Critique
Some of the advantages of role-playing are that it is less expensive than experiments and it provides additional outcomes not considered by experts on opposing objectives and strategies. It provides individuals to think for themselves. Although the author supports role-playing, this publication has some disadvantages.
Some of the advantages of role-playing are that it is less expensive than experiments and it provides additional outcomes not considered by experts on opposing objectives and strategies. It provides individuals to think for themselves. Although the author supports role-playing, this publication has some disadvantages.
For example:
One of the basic elements of role-playing is to cast roles similar to the people they represent. According to the author, participants should have similar backgrounds, attitudes and objectives to the role-playing simulation. In addition, Ashton and Krammer (1980), “found considerable similarities between students and non-students in studies on decisions making processes.” While valuable, it does not state whether undergraduates partook. If so, how can an undergraduate student have a similar background and attitude towards a role-playing situation if s/he does not have a lot of real work experience and/or an education in the topic of discussion? Therefore, further research needs to show a more in depth correlation stating why students and non-student are showing similar role-playing conclusions.
In addition, the author stated that each role-playing simulation “should run around ten sessions, five using one description and five using another.” The author did not go into detail on why he chose ten sessions versus any other number. Are ten sessions the average used in role-playing simulations, or is that the number he likes to use?
Armstrong, J. S. 2002. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001 edition. Boston, MA: Springer.
Posts
The study by Kresten Green showed similar results to Armstrong's findings with Green's participants achieving 64% correct predictions using role-playing as opposed to 28% correct predictions using unaided judgement. In some of Green's experiments, she was able to place students with leadership experience into leadership roles in the simulation and those with quantitative backgrounds into roles that have quantitative skill sets. Did Armstrong attempt to match anyone in a similar way or were all the roles chosen randomly?
ReplyDeleteArmstrong did not go into detail on how the students were picked- hence one of the flaws in this case study. I would have liked to seen the selection process and how he went about choosing for each simulation.
ReplyDeleteDid the paper describe any of the types of scenarios played out? Also, why was 80 minutes chosen as the amount it would take to complete a trial? Wouldn't it be beneficial to have these trials play out until they agree they are done rather than it be time dependent?
ReplyDeleteNo, the paper did not go into detail about the different scenarios (I wish it did). Armstrong broke the 80 minutes down- 20 minutes for students to read and get into the role and 60 minutes to play the scenarios out. I do agree with you that the scenarios should have lasted until a conclusion was reached.
ReplyDelete