Summary:
The article is trying the find out the level of the efficacy of role playing technique in 'forecasting' by applying it to a sales forecasting case. Since one of the determinants of a good analyzing techniques is not to harm the forecasting accuracy, this study aims to figure out the role of role played groups in the forecasting accuracy of sales rates in the future. Hence, the authors form 7 groups ( each group has 3 participants ) of participants and randomly label them as role-playing groups, while they form 6 groups as no-role-playing groups. Totally, they have 13 groups and and 39 participants.
The no-role-playing groups are asked to make groups discussions and come up with consensus forecasts for each product’s sales for the next period. The group rules prohibited any member acting as a group leader and asked the participants to: (1) act with due consideration for all group members; (2) let the member given the Q identifier (is selected randomly) introduce the initial forecast; (3) record their levels of satisfaction with each of the consensus forecasts; (4) record their preferred forecast (which would be equal to the consensus forecast only if they fully agreed with the group consensus); and (5) evaluate each of the group members along with a self-evaluation upon task completion.
The role-playing groups are asked to draw out unmarked envelopes for their roles. These roles are the Forecasting Executive, Marketing Director, or Production Director. They all are given scripts in which their role descriptions identified. The set of rules given to each group prohibited any member acting as a group leader while asking the participants to: (1) act out their given roles as they believed it would be performed in an organisation; (2) act with due consideration for all group members; (3) let the forecasting executive introduce the initial forecast; (4) record their levels of satisfaction with each of the consensus forecasts; (5) record their preferred forecast (which would be equal to the consensus forecast only if they fully agreed with the group consensus); and (6) evaluate each of the group members along with a self-evaluation upon task completion.
Results:
The study doesn't reveal a significant difference between the no-role-playing groups and role-playing groups regarding the accuracy of consensus forecast. The study also couldn't find no significant difference in forecasting accuracy. However, the study shows that the commitment of no-role-playing groups members is stronger than the role-playing group members. This is due to, the role-playing groups members' commitment to their assigned roles and scripts.
Critique:
The group members' lack of subject matter expertise, lack of background information regarding those sale products and so forth are very important determinants for a person while judging about those products' future sale figures. Thus, the control groups in role playing technique are always susceptible to misleading the experimenters due to lack aforementioned skill sets or background information. This may have been overcame if they could conduct their experiment with an actual business organization's members.
Source:
http://www.sciencedirect.com.ezproxy.mercyhurst.edu/science/article/pii/S0305048311001459?np=y
Subscribe to:
Post Comments (Atom)
Posts
As mentioned in your critique, one of the problems with role playing is experience with the problem that is being explored. An untrained or inexperienced participant may come up with theories or conclusions that are inconsistent with reality. It seems it is very important to have role players with subject matter expertise on the topic. However, from a scholarly perspective, if I understand the results correctly, you can’t have a control group with subject matter expertise and remove bias or group think simultaneously from the test.
ReplyDeleteActually, the study doesn't state that it is impossible to have a control group with subject matter expertise. It only indicates that they couldn't find a result that would support the role-playing is an effective technique 'assumption' for their very unique study. Moreover, they identify the expertise as 'mental models'. Individual competencies, skills, and knowledge of the group members constitute the mental models. Therefore, these have the biggest impact on their judgments. And since they adhere to their identified roles (they tend to think for the benefit of DM) the biggest impact comes from the decision maker's mental model. (However, it is a rule that there is no leader in this experiment.)
DeleteThe article mentions that the forecasts from the role-playing groups yielded consistently lower errors, while the non-role-playing groups varied in accuracy. Do you think increasing the sample size of the groups or total participants could show for more statistical significance?
ReplyDeleteAs you may know, the results of an experiment don't have to be statistically significant. This can be interpreted as that the experimenter(s) couldn't find enough support for his/her discussion questions at the end of the study. However, in this study, the authors encourage the future studies to be held with increased sample sizes. Probably, that would make the results more concise.
Delete