Summary of Findings: Role-Playing (3.5 out of 5 stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the  articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in August 2015 regarding Role Playing as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Role-Playing is an analytic technique in which participants are divided and assigned specific roles to simulate a problem discussion. Each participant must adopt the specific perspective of their assigned role. Each group that is affected by the problem should be represented. The role-players then engage in a discussion of the topic. This technique is useful for both forecasting solutions to problems and for analyzing previously suggested answers.

Strengths:

• It helps the groups to solve the problems in a creative way.
• Role playing can take into account the complexities of the situation.
• It helps to prevent biases if it is implemented as a proper group work.
• It helps the group to gain a different perspective.

Weaknesses:

• In order to solve the problem, it must be launched properly.
• There could possibly be hesitant participants.
• It is extremely important to actually put oneself in the role.
• Important to have background information for participants.
• The method must identify intelligence gaps.

How-To:

1. Assign roles and make sure people embody them
2. Describe the situation
3. Let the situation develop in a free form manner
4. Upon conclusion of the exercise debrief the group

Personal Application of Technique:

Role Playing Exercise involving a potential decision by the Mercyhurst school administration to mandate that all student-athletes live on campus.

For this exercise, we grouped students into three teams: Student-Athletes, School Administrators, and Analysts. The three groups were requested to develop pros, cons, and a conclusion on whether this new policy is good for their specific group. The teams were given 10 minutes to develop their arguments. After the initial brainstorming period, the teams presented their ideas, and a list of similarities and differences in points of view was produced.

To improve this exercise, an additional group (non-student-athlete) should have been added. Additionally, all roles should have been assigned prior to the problem statement being given.

For Further Information:
Forecasting decisions in conflict situations
Wisdom of group forecasts: Does role-playing play a role?
Role-Playing our way to solutions

Forecasting decisions in conflict situations: A comparison of game theory, role-playing, and unaided judgement

Summary:

This article compared the accuracy of game theory, role-playing, and unaided judgement in regards to forecasting decisions . The author provided much background information on the subject, including predicting the behavior of participants in the new competitive market for wholesale electricity after the New Zealand government transferred assets to a new private sector electricity company called Contact Energy Ltd. in 1996. When the managers employed the role-playing method, the results were not consistent with the executives' beliefs so they turned to game theory for answers. What they found out was that the game theory method ended up not being helpful at all and that role-playing accurately predicted the behaviors.

The article continues to mention other tests that have been performed by many other people which ultimately led the author to determine that in regards to predicting human behavior, role-playing is the most accurate, game theory is the second most accurate, and unaided judgment is the least accurate. This is due in part to the fact that game theory cannot take into account the complexities of situations like role-playing can. According to the article, there is also not much proof to suggest that predictive validity for game theory in real conflicts exists since it is normally tested using role-played conflicts. Basically, role-playing is a more accurate method to use to determine human behaviors because there is a much greater degree of realism.

After doing this background research, the author decided to run their own experiment by creating six conflicts to test the accuracy of game theory, role-playing, and unaided judgment. To do this, the author created six conflicts that would be attempted to be solved by the participants using one of the three methods that were being compared. The results ended up being consistent with previous research which means that the author's experiment also found that role-playing is the most accurate method, game theory is the second most accurate method, and unaided judgment is the least accurate method to forecast human behaviors (conflict resolutions).

Critique:

In general, the experiment that was created and ran by the author was not very controlled. As a psychology major as well, this really bothered me. The first issue that I noticed was that the conflicts presented in the experiment were not all made up. Some of the situations came from previous research, one came from television, and another one was a real life situation from a company. The author claims that these situations were probably not going to be recognized by the participants, but if they were, then there could be problems with the accuracy of the experiment. In addition to this, nothing in the experiment (methods used, time given to find a solution, which situation the participants needed to solve, and more) were assigned to the participants randomly. Various other aspects of the tests were not controlled as well.

Source:

"Forecasting decisions in conflict situations: A comparison of game theory, role-playing, and unaided judgement"
By: Kesten C. Green
http://www.umsl.edu/~sauterv/DSS/green.pdf

Wisdom of group forecasts: Does role-playing play a role?

Summary:

The article is trying the find out the level of the efficacy of role playing technique in 'forecasting' by applying it to a sales forecasting case. Since one of the determinants of a good analyzing techniques is not to harm the forecasting accuracy, this study aims to figure out the role of role played groups in the forecasting accuracy of sales rates in the future. Hence, the authors form 7 groups ( each group has 3 participants ) of participants and randomly label them as role-playing groups, while they form 6 groups as no-role-playing groups. Totally, they have 13 groups and and 39 participants.

The no-role-playing groups are asked to make groups discussions and come up with consensus forecasts for each product’s sales for the next period. The group rules prohibited any member acting as a group leader and asked the participants to: (1) act with due consideration for all group members; (2) let the member given the Q identifier (is selected randomly) introduce the initial forecast; (3) record their levels of satisfaction with each of the consensus forecasts; (4) record their preferred forecast (which would be equal to the consensus forecast only if they fully agreed with the group consensus); and (5) evaluate each of the group members along with a self-evaluation upon task completion.

The role-playing groups are asked to draw out unmarked envelopes for their roles. These roles are the Forecasting Executive, Marketing Director, or Production Director. They all are given scripts in which their role descriptions identified. The set of rules given to each group prohibited any member acting as a group leader while asking the participants to: (1) act out their given roles as they believed it would be performed in an organisation; (2) act with due consideration for all group members; (3) let the forecasting executive introduce the initial forecast; (4) record their levels of satisfaction with each of the consensus forecasts; (5) record their preferred forecast (which would be equal to the consensus forecast only if they fully agreed with the group consensus); and (6) evaluate each of the group members along with a self-evaluation upon task completion.

Results:

The study doesn't reveal a significant difference between the no-role-playing groups and role-playing groups regarding the accuracy of consensus forecast. The study also couldn't find no significant difference in forecasting accuracy. However, the study shows that the commitment of no-role-playing groups members is stronger than the role-playing group members. This is due to, the role-playing groups members' commitment to their assigned roles and scripts.

Critique:

The group members' lack of subject matter expertise, lack of background  information regarding those sale products and so forth are very important determinants for a person while judging about those products' future sale figures. Thus, the control groups in role playing technique are always susceptible to misleading the experimenters due to lack aforementioned skill sets or background information. This may have been overcame if they could conduct their experiment with an actual business organization's members.

Source:
http://www.sciencedirect.com.ezproxy.mercyhurst.edu/science/article/pii/S0305048311001459?np=y

Role-Playing our Way to Solutions

"Role-Playing our Way to Solutions"
By: Miriam Axel-Lute
National Housing Institute
Source: https://www.bostonfed.org/commdev/c&b/2013/winter/role-playing-our-way-to-solutions.pdf

Summary:

The article discusses a community development issue that used a megacommunity simulation as a way to find solutions and methods. In this situation, the overall goal is to reduce the state of Connecticut's energy use by 25% by the year 2030. Under the leadership of the Housing Development Fund (HDF) and the guidance of the consultant company Booz Allen Hamilton, this large community role-playing simulation was formed in March 2012.

Participants from across the sphere of the energy industry and the Connecticut community were selected to participate. Examples of those represented are energy suppliers, government agencies, non-profits, private residents, the financial sector, and many more. The goal of the simulation was to find avenues in which all parties could agree on to increase Connecticut household energy efficiency. According to the article, in the beginning the participants were uneasy in regards to fitting into the role-playing process but eventually were able to get comfortable.

As expected, the simulation allowed participants to think differently and many new relationships for future collaboration were formed. Participants mentioned how this simulation awoken them to how many aspects of the issue there were and how many sectors play a role in reaching the overall goal of increased energy efficiency. One interesting comment about the potential of role playing was made by Booz Allen VP Gary Rahl, He said, “You never want to pick a goal that can simply be met by an analytical solution—figure out who needs to do what. You need a goal where, to meet it, there will be tensions between participants and no single way of getting there.” I find that very intriguing and I think it is worthwhile to note about future role-playing simulations.

Critique:

The article discussed it, but a major issue with this study was the lack of one major participant, low-income householders. To get the full benefit of a role-playing simulation all aspects of participation need to be represented. The other negative of the study was that most of the results were that of compromise from previous ideas. While compromise and collaboration are good and important, I would have liked to see more innovation in a role-playing simulation. I realize it is difficult to achieve due to every side have their own goals and agenda, however true innovation would be the best result in my opinion. The benefit of role-playing is it increases the potential for innovation, there is no clear cut answer as Booz Allen VP Gary Rahl said. His statement as mentioned above I believe is very interesting and something we all should keep in mind when discussing role-playing.