Comparing the Outcomes of Three Fraud Brainstorming Procedures

Introduction
 This article by James Hunton and Anna Gold of Erasmus University describes an experiment in which the researchers tested the application of three brainstorming procedures - nominal group, round robin, and open discussion - to determine the effectiveness of their application to the sample situation of brainstorming possible fraud risks.

Summary
 The experiment involved 150 audit clients and 2614 auditors broken into hierarchical audit teams. Each team was assigned to a specific method of brainstorming and taken through a preparatory phase in which they individually contemplated possible fraud risks. This was to reduce 'free riding' by causing each individual to come to the group with ideas already in mind, rather than coming up with them after the discussion unfolded. The three types of brainstorming were described as follows:

1. Nominal Groups - Individuals generate fraud risk lists on their own and would not actually meet with their team members. The lists would be compiled together and that final list would represent the group's brainstorming output. There would be no discussion at all and complete anonymity.
2. Round Robin - After the preparatory phase, the group would meet and go through two rounds of discussion in which each participant would read their list aloud. The second round would be for new ideas that occurred while listening to the first round. There would be no unstructured discussion and no anonymity.
3. Open Discussion - After the preparatory phase, the group would meet and participate in open, unstructured discussion about their ideas. Contribution to the discussion would be on a volunteer basis, rather than mandatory as in the Round Robin groups.

The second stage was the interaction stage, in which each team met (or compiled their lists, as with the nominal groups) and engaged in their various forms of brainstorming discussion, the end-goal being to generate a unique list of potential fraud risks. Finally, the evaluation stage was where the teams determined whether their identified list of risks warranted a change in the audit procedures of their clients - increased audit hours, tests, altered timing, etc.

Conclusions
 The researchers actually looked at hypotheses involving multiple stages of the experiment. Their findings include:

• Open discussion groups spent less time in the preparatory phase than the other groups AND generated fewer unique fraud risks on their preparatory lists.
• Open discussion groups' interaction phase tended to result in lost ideas from the preparatory phase as some members were not comfortable speaking up in the discussion.
• Nominal group and Round-Robin brainstorming achieved an apparent 'tie' in end results, both more effective than open discussion on a statistically-significant level.

The researchers go on to briefly discuss the practical implications of these findings, most notably that the commonly-used open discussion brainstorming was least effective and that some combination of nominal group and round robin procedures might yield the best of both - such as having nominal group members meet in person after submitting their lists for a round-robin discussion that might generate additional ideas.

Source
Hunton, J. E., & Gold, A. (2010). A Field Experiment Comparing the Outcomes of Three Fraud Brainstorming Procedures: Nominal Group, Round Robin, and Open Discussion. Accounting Review, 85(3), 911-935. Accessed through EBSCOHost database. Abstract available at http://aaajournals.org/doi/abs/10.2308/accr.2010.85.3.911?journalCode=accr