This article by James Hunton and Anna Gold of Erasmus University describes an experiment in which the researchers tested the application of three brainstorming procedures - nominal group, round robin, and open discussion - to determine the effectiveness of their application to the sample situation of brainstorming possible fraud risks.
The experiment involved 150 audit clients and 2614 auditors broken into hierarchical audit teams. Each team was assigned to a specific method of brainstorming and taken through a preparatory phase in which they individually contemplated possible fraud risks. This was to reduce 'free riding' by causing each individual to come to the group with ideas already in mind, rather than coming up with them after the discussion unfolded. The three types of brainstorming were described as follows:
- Nominal Groups - Individuals generate fraud risk lists on their own and would not actually meet with their team members. The lists would be compiled together and that final list would represent the group's brainstorming output. There would be no discussion at all and complete anonymity.
- Round Robin - After the preparatory phase, the group would meet and go through two rounds of discussion in which each participant would read their list aloud. The second round would be for new ideas that occurred while listening to the first round. There would be no unstructured discussion and no anonymity.
- Open Discussion - After the preparatory phase, the group would meet and participate in open, unstructured discussion about their ideas. Contribution to the discussion would be on a volunteer basis, rather than mandatory as in the Round Robin groups.
The researchers actually looked at hypotheses involving multiple stages of the experiment. Their findings include:
- Open discussion groups spent less time in the preparatory phase than the other groups AND generated fewer unique fraud risks on their preparatory lists.
- Open discussion groups' interaction phase tended to result in lost ideas from the preparatory phase as some members were not comfortable speaking up in the discussion.
- Nominal group and Round-Robin brainstorming achieved an apparent 'tie' in end results, both more effective than open discussion on a statistically-significant level.
Hunton, J. E., & Gold, A. (2010). A Field Experiment Comparing the Outcomes of Three Fraud Brainstorming Procedures: Nominal Group, Round Robin, and Open Discussion. Accounting Review, 85(3), 911-935. Accessed through EBSCOHost database. Abstract available at http://aaajournals.org/doi/abs/10.2308/accr.2010.85.3.911?journalCode=accr