Tuesday, March 27, 2012

Wisdom of group forecasts: Does role-playing play a role?


Introduction

The objective of this study was to identify the effect of group dynamics and role playing on forecasting accuracy. Although businesses increasingly use groups of employees to create consensus sales forecasts, little research has been done to determine the accuracy of these group forecasts.  The participants in this study were divided into role-playing (RP) groups and non-role-playing (NRP) groups in order to see how role playing affected forecasting accuracy of the groups.

Discussion

Researchers split the group of participating MBA students into 13 groups of three, seven of which were assigned to be RP groups, and six of which were NRP groups Students in the RP groups were randomly assigned roles as forecasting executives, marketing directors, or production directors, and depending on their roles, they were given varying descriptions and instructions about the products used in the role playing exercise. They were also given initial model-based forecasts, and researchers tracked how each group adjusted these forecasts. 

Although the RP groups showed consistently higher accuracy than the NRP groups, researchers did not find any statistically significant results, possibly due to the small sample and group-specific factors, such as extroversion and expertise. 

Researchers did find that RP group members were less likely to agree with their group’s consensus forecast.  When asked individually after group discussion, only 44 percent of RP group members gave answers that matched their group forecasts, compared to 78 percent of the NRP group members.
The study also showed that role-playing influenced the likelihood that a group would adjust the initial model-based forecasts. RP groups made far fewer adjustments to the initial model-based forecasts than the NRP groups. Groups that did not have roles or scripts adjusted 92 percent of their initial forecasts, while only 67 percent of RP groups adjusted the initial forecasts.

Conclusion & Comments

At several points throughout the paper, the researchers highlighted the fact that more research needs to be done on group forecasting. While it is becoming a prevailing method for organizations to create forecasts, there is not research supporting the accuracy of using groups. Although the researchers seemed primarily interested in findings related to group forecasting accuracy, the study produced several interesting findings about how the role playing technique influences groups. It would be helpful to see more research done on the topic using more participants, and more discussion about the implications of the differences between RP groups and NRP groups.

Reference

 Onkal, D., Sayim, K.Z., & Lawrence, M. (2011). Wisdom of group forecasts: Does role-playing play a role? Omega 40 p. 693-702.

10 comments:

  1. Is it possible that the reason the NRP group saw more consensus was due to group think setting in and the desire of the group to promote harmony?

    ReplyDelete
  2. My research had the same conclusion in that RP methods/analyses are lacking critical research and in the incubation stage of a concrete methodology. Also, I think it's pretty interesting that the RP group members were less likely to agree with their group's consensus forecast. Why did this happen? Were they a little over zealous with regard to their roles?

    ReplyDelete
  3. Two comments:
    First, what roles did the role playing group play? Was there a real life conflict to call for role playing? Was there genuine interaction between groups of role players or did the role players work independently and then have to create a group solution.

    Second, the NRP groups sound like they did actually experience a form of groupthink, as David mentioned. In the NPR groups, members came together on premature consensus to either finish the work early or avoid a foreseeable conflict while RP group members probably had a stronger vested interest in their individual roles and objectives.

    ReplyDelete
  4. Great job in finding an article that reflected forecasting. Perhaps you'd like to share your researching skills as I struggled to find good material for this week's blog post.

    ReplyDelete
  5. Dean I had the same problem. But I think that speaks to the problems with role playing as an analytic technique? It seems like Jo's study was inconclusive, as was mine.

    ReplyDelete
  6. I think we can implement the RP and NRP groups in the stock game we played last term on Weseed. The site has an option to work in teams and manage one portfolio. With enough teams, I think the outcomes can give us a measurable, legit, answer to whether this works.

    ReplyDelete
  7. Dean - it took me a while to find this article. I agree with Shawn, I think it was mostly because this doesn't seem to be a very popular technique.

    ReplyDelete
  8. Puru - I'm not sure about that, it seems like it could be too complicated for students who are generally unfamiliar with the stock market. If we had a group of people with a better background for it, I think that it might work pretty well.

    ReplyDelete
  9. David - I definitely think that it was because of groupthink. But perhaps it was less about promoting group harmony than the lack of a reason to disrupt it. In a study with no or few consequences for the participants, why try to disagree?

    Also, the researchers said the study demonstrated how the RP groups were more likely to use divergent thinking. This seems to apply to our discussion about convergent/divergent thinking in the class about brainstorming. I wonder what would happen if someone tried to integrate role-playing into a brainstorming session... that could produce some interesting results.

    ReplyDelete
  10. Sam - Participants in the RP groups were randomly assigned to be either Forecasting Executives, Marketing Directors, or Production Directors. There was a great deal of interaction between the group members, as they were told they had to come up with a group consensus.

    ReplyDelete