Summary of Findings (Green Team): Brainstorming (3.5 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in May 2012 regarding Brainstorming specifically. This technique was evaluated based on its overall validity, simplicity, flexibility, its ability to effectively use unstructured data and its ease of communication to a decision maker.

Description:
Brainstorming is a group or individual creativity modifier by which efforts are made to generate a list of ideas spontaneously contributed by its member(s). There are many types of brainstorming, including nominal, round robin and open discussion. Brainstorming works best in a partially structured environment, where participants follow [structured] rules that govern the group process. The modifier is best used in the beginning of the project, as this is the point at which new ideas are most valuable. Findings can be added throughout the project. This is a classic Divergent technique.To further the success of brainstorming, convergent techniques can be used to prioritize, group and filter ideas.

The technique is effective at idea generation, simple to use (although difficult to use effectively), applicable to a variety of problems/projects, and does effectively use unstructured data, though the modifier may engender groupthink and hinder creativity.

Strengths:

• Easy method to conduct
• Many methods of brainstorming
• Can be customized to different personality types
• Once done can be easily expanded
• If participants come prepared, there are improved results

Weaknesses:

• Subject to groupthink in open discussion
• Easy to become sidetracked/distracted
• Critics may discourage participation
• Strangers within the group make it harder to stimulate discussion and idea generation
• Subject to anchoring based upon other ideas

How-to:
The CIA offers an excellent run-down of their process of brainstorming. A step by step summary is available here.

Personal Application of Technique:
The exercise in class compared two methods of brainstorming: nominal and open discussion. Both groups were asked the same question: “What would you like to see in the new Intel building?”

The nominal team had ten minutes to compile individual lists. Once the ten minutes were done, the lists were combined. The open discussion group spent the same ten minutes as a group discussing the question.

At the end of the ten minutes, the open discussion team had compiled a list of 34 items, while the nominal team had a list of over 125 items. The results of the application confirmed the findings from many of the readings of the week.

Rating: 3.5 out of 5

If done wrong, the evidence suggests it is not worthwhile. However, when used with both divergent and convergent techniques it can be useful.

Effectiveness of Game-Based Brainstorming

Introduction
Researchers in Taiwan used versions of a computer-based brainstorming game to test the effectiveness of different types of brainstorming in an online collaborative environment. The article outlines how the game Idea Storming Cube (ISC) was used to test how high school students responded to information-based, game-based, and game-based with peer-like intelligent agent methods of idea generation. The article also details the game-design techniques used to create these tools. I think this is an interesting addition to the class's discussions on game-based learning.

Discussion

ISC, the game used to conduct the experiment, is inspired by the idea of a rubics cube. Users are told to write on their own face of the cube. When users come up with an original idea, the game rewards them by giving them points and turning the cube and allowing them to see answers from other participants. Users win the game by coming up with the more valid ideas than the other participants.

Researchers used variations of this game to test the different brainstorming methods. ISCinfo tested information sharing by adding a window that let users see all of the group members' ideas as soon as they are created, thereby decreasing the reward for original ideas. ISCgame tested how users performed when the competitive rules applied and they were rewarded for new ideas by being shown another user's list. ISCgame_agent added a peer-like intelligent agent, essentially an idea-generating pseudo-user, to inspire users to think more creatively when they were "idea blocked."  The peer-like intelligent agent was designed to encourage paradigm modification, encouraging students to think outside the bounds of the context.

Conclusion

The game-based learning environment was successful in improving students' generation of ideas. Students who played ISCgame_agent had the best performance. The authors of the study found that the peer-like intelligent agent helped students to think about unexplored categories and generate more ideas about categories rarely covered by students playing ISCgame and ISCinfo.

Reference

Huang, C.-C., Yeh, T.-K., Li, T.-Y., & Chang, C.-Y. (2010). The Idea Storming Cube: Evaluating the Effects of Using Game and Computer Agent to Support Divergent Thinking. Educational Technology & Society, 13 (4), 180–191.

Effects of task instructions and brief breaks on brainstorming

Introduction:

In this article, Paulus et al use three separate experiments to measure the effectiveness of facilitators, instructions, and breaks in the process of brainstorming. This research was conducted on both group brainstorming and nominal brainstorming, partially as an attempt to increase the effectiveness of group brainstorming creativity, which studies have shown to be less effective than nominal brainstorming. Experiment 1 was conducted in person, while experiments 2 and 3 measured the impact of additional rules for individuals who wrote or used a computer, respectively.

Summary:

Experiment 1 measured the difference made by the presence of a facilitator (in group settings) and the introduction of additional brainstorming rules (to both groups and nominal participants). The main finding of this first experiment was that both individuals and groups who were given additional rules experienced an increase in their performance. Facilitators, on the other hand, did not influence group performance other than to limit the number of extraneous words used. The authors did note that further research would be necessary to determine if a more active facilitation style could lead groups to surpass nominal brainstorming in productivity.  

Experiment 2 separated individuals into four groups: 1) a traditional group with no rules and no breaks; 2) a 36 minute brainstorming session with additional rules but no break; 3) a six-minute break after 15 minutes, and; 4) and two three-minute breaks after 10 and 23 minutes. As the experimenters predicted, the second and third group showed improvement over the first group during the brainstorming session. While the fourth group also had better production, they did not receive any additional benefit from the extra break.

Experiment 3 was broken into the same four categories, although the participants were brainstorming alone on a computer. The results of this experiment indicated that while the additional rules once again improved the brainstorming session, the breaks had no additional effect on electronic participants. Researches hypothesized that this was due to electronic brainstorm being both less fatiguing and also easier to keep track of (as the words are visible on screen at all times).

Source:

http://web.ebscohost.com/ehost/detail?vid=5&hid=113&sid=606d4a79-3b66-41e6-be98-73afebcde869%40sessionmgr114&bdata=JnNpdGU9ZWhvc3QtbGl2ZQ%3d%3d#db=pdh&AN=gdn-10-3-206&anchor=c19

A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis.

The CIA published its understanding of brainstorming as an analytic technique in the white paper, A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis. The document provides a run-down of diagnostic, contrarian, and imaginative thinking techniques used and recommended by the CIA for its analysts. Brainstorming falls under the imaginative thinking techniques section, along with outside-in thinking, red team analysis, and alternative futures analysis.

According to the primer, brainstorming is an unconstrained group process designed to generate new ideas and concepts. The technique should be used in the beginning of the the process or at critical points among a group of analysts in order to take advantage of a variety of perspectives to help structure a problem. Although an individual may act alone to produce a wider array of ideas, he or she may have difficulty breaking free of his or her cognitive biases without the benefit of a diverse group.

The brainstorming process should be very structured, in order to break down mind-sets and produce new insights. There are two phases: a divergent thinking phase to generate and collect new ideas and insights, and a convergent phase in which new ideas are grouped and organized around key concepts. During the session, which should take over an hour, ideas should never be censored, no matter how unconventional they may seem. The conventional wisdom concerning the topic or project should be exhausted in order to develop new ideas or important connections between the topic and unstated assumptions. The ideal group should incorporate at least one 'outsider who does not share the same background, culture, or technical knowledge as the core group. A mix of junior and senior analysts are also important, as less experienced analysts may not carry the same biases and prejudices about the topic. The recommended technique is as follows:


Divergent Thinking Phase:
• Distribute “Post-It” notes and pens or markers to all participants. Typically, 10-12 people works best.
• Pose the problem in terms of a “focal question.” Display it in one sentence on a large easel or whiteboard.
• Ask the group to write down responses to the question, using key words that will fit on the small “Post-It” note.
• Stick all the notes on a wall for all to see—treat all ideas the same.
• When a pause follows the initial flow of ideas, the group is reaching the end of their conventional thinking and the new divergent ideas are then likely to emerge.
• End the “collection stage” of the brainstorming after two or three pauses.

Convergent Thinking Phase:
• Ask the participants as a group to rearrange the notes on the wall according to their commonalities or similar concepts. No talking is permitted. Some notes may be moved several times as notes begin to cluster. Copying some notes is permitted to allow ideas to be included in more than one group.
• Select a word or phrase that characterizes each grouping or cluster  once all the notes have been arranged.
• Identify any notes that do not easily fit with others and consider them either useless noise or the beginning of an idea that deserves further attention.
• Assess what the group has accomplished in terms of new ideas or concepts identified or new areas
that need more work or further brainstorming.
• Instruct each participant to select one or two areas that deserve the most attention. Tabulate the votes.
• Set the brainstorming group’s priorities based on the voting and decide on the next steps for analysis.



Source:


U.S. Government (2009). A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis. Retrieved from: https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/Tradecraft%20Primer-apr09.pdf

Brainstorming - Yay or Nay?

Summary

In the article, In the defense of brainstorming: against Lehrer’s New Yorker article, Scott Berkun talks about the method of brainstorming and defends its uses in the business community. Opposing Groupthink: the brainstorming myth, the author looks at how brainstorming can be a useful tool and puts it in the context of the process rather than as a standalone tool.

Technique

The technique is useful for facilitating group thinking and coming up with new and perhaps unthought-of of ways of approaching different scenarios and situations. The author details how a facilitator is necessary to keep the group on task and to the point. There should also be a great deal of preparation done in order for it to be effective. This includes; a topic for ideas around a simple question, invitations to select individuals to join in on the process and a note in advance in order to give people time to think ahead.

            A criticism free environment should be implemented in order to get the best out of the participants. The facilitator should then introduce the problem that needs to be solved to those involved. Participants should explain their idea and why it is necessary and others should feel free to oppose or build on those ideas – always stating why in order for progression. Once the brainstorming session is over – the ideas should be revisited and discussed in more detail.

Strengths

• ·         It is designed for a volume of ideas
• ·         Useful for a wide variety of situations
• ·         Once done can be easily expanded
• ·         It is easy and requires little preparation

Challenges

• ·         May be easy to get off topic
• ·         Critics can make it hard to stimulate ideas
• ·         It is only a small part of a larger process
• ·         People should know each other beforehand. Strangers make it hard to stimulate discussion and ideas.

Conclusions

Brainstorming has many critics and it is hard to measure its utility. There are a number of arguments for and against it. However, it is simple to implement and if the preparation and right conditions are there then it is a great tool for generating a lot of ideas – no matter how good they are – for later discussion and use.

Bibliography

Berkun, Scott. In the defense of brainstorming: against Lehrer’s New Yorker article;  http://www.scottberkun.com/blog/2012/in-defense-of-brainstorming-2/

Momentum: Getting and Staying on Topic During a Brainstorm

Introduction:

The authors suggest in their study that using a creativity support tool to encourage small amounts of individual preparation prior to a group brainstorming session results in more structured use of the time.

Summary:

Bao, Gerber, Gergle and Hoffman created an online tool called Momentum to deliver short answer questions to participants of a brainstorming team each day of the week prior to the session. The questions are intended to be open and somewhat unexpected in order to encourage thinking about the brainstorm topic in unusual ways.

Momentum is a web-based system that generates a series of textual prompts based on a given brainstorm topic. Prompts can be modified before they are sent by e-mail to all group members at specified intervals in the days leading up to the brainstorming session. Group members are asked to quickly and individually respond to the prompts with text messages or photo attachments. These replies are stored in the system until the start of the brainstorming session, at which point all responses are unveiled as a large-scale visualization visible to the entire group.

During the brainstorming session, the answers given from the pre-session prompts are clustered so that all answers are at least partially visible on a digital background. The prompts can be accessed by the participants during the brainstorming session.

The experiment included a group that received pre-session prompts and a similar group who did not receive any prompts. Both groups came up with an equal number of new ideas; however, the group who used prompts were able to stay on target during the session. The prompt group also used several brainstorming strategies during their session. The prompts also aided the group in using storytelling and visualization during the session, speeding up the process.

Conclusion:

Results suggest that participants who used Momentum found it useful for maintaining task focus and providing inspiration during group brainstorms. Although Prompted and Unprompted groups performed similarly according to traditional metrics such as the quantity and quality of ideas, the researchers observed unique patterns in their use of time and idea creation processes. Prompted groups were quick to generate quality ideas and largely avoided unrelated discussion. Even though storytelling was accompanied by idea creation in both Prompted and Unprompted groups, the shift from story to idea appeared to be quicker for the former, especially with the aid of the visualization.

Source:

Bao, P.; Gerber, E.; Gergle, D.; & Hoffman, D. (2010). Momentum: Getting and Staying on Topic During a Brainstorm. CHI 2010, April 10-15, 2010, Atlanta, GA.  Retrieved from http://collablab.northwestern.edu/patti/pubs/CHI2010-BaoEtAl-Momentum.pdf

Group Preference and Convergent Tendencies in Small Groups: A Content Analysis of Group Brainstorming Performance

Introduction:

In their study on brainstorming, Timothy Larey and Paul Paulus took a different approach to the subject. Instead of looking at the effectiveness of the technique as a whole, they looked at how different group dynamics impact the results of brainstorming. The study looks at differences between people who prefer working in groups or alone as well as actually brainstorming in groups or brainstorming alone.

Summary:

The basic hypothesis of the study is that, while people who prefer working in groups will do better than people who prefer to work alone when working in a group, people who brainstormed alone will outperform people who worked in groups.  A key point of this argument rests on the idea of “groupthink” that originated with Jarvis. The idea is that groups, in a desire to reach consensus quickly, will avoid divergent thinking and instead prefer to agree more quickly than would otherwise be the case. This convergent thinking would limit the performance of small groups in developmental and broad practices such as brainstorming, which demand a certain level of openness, creativity, and even divergence to be effective.

Using 144 volunteers from the University of Texas at Arlington, Larey and Paulus first tested for introverted or extraverted preferences, and then generated groups of four or individuals based on the results of the first test. This allowed for interactive groups and nominal groups of those who either preferred to work in groups or work alone. The experimenters then provided a brainstorming activity for all groups that discussed ways to improve the campus. Measurements were taken on the total number of ideas, the number of different categories discussed (categories being things like improving parking, classes, student activities, etc.), and how much detail was covered in each category.

The results confirmed the original hypothesis of Larey and Paulus by demonstrating that individual brainstorming activities were more effective than interactive small groups. While interactive groups made up of members that preferred working in groups did perform marginally better than groups of members that preferred to work alone, both fell far short of the nominal individual brainstorming groups. The average total number of ideas that interactive groups generated were 23.5 and 20.0 while those that nominal groups generated 45.6 for nominal high interactive preference and 53.2 for nominal low preference. Another key side hypothesis that was proven was that interactive groups discussed fewer categories in depth, while also not moving between categories. Rather, they preferred to spend more time discussing a few categories by repeating ideas. The nominal group, on the other hand, changed categories much more frequently and readily.

Conclusions:

The ultimate conclusion of the study is that brainstorming is most effective at generating ideas when it is done by an individual. The authors believe that this is because people tend to prefer convergent thinking and can easily fall prey to “groupthink” mentalities when in small groups. This mentality discourages divergent or “free-wheeling” ideas and limits both creativity and real depth of discussion as people who interact would rather avoid conflict.

This study has some significance for the intelligence field. As brainstorming is a common step early in the modeling and research phases of intelligence, and most intelligence is done in small groups, it is important to know how these two factors interact. According to this study, a direct and interactive brainstorming session is both remarkably ineffective and a waste of the time of individuals. An alternate suggestion would be to have each group member individually brainstorm ideas for a given amount of time. At the end of this period, the group could come together and collectively present their own individual reports to a facilitator through a round-robin session where each idea is considered.

Source:

Larey, T.S., Paulus, P.B. (1999). Group preference and convergent tendencies in small groups: A content analysis of group brainstorming performance. Creativity Research Journal, 12 (3) 175-184.

Retrieved from:
http://www.tandfonline.com/doi/abs/10.1207/s15326934crj1203_2

Perceived Effectiveness of Brainstorming

Introduction: Researchers at the University of Arlington used a study to compare the perceived productivity of individuals brainstorming alone, and when with a group. Using a survey format, they were able to study the same group of individuals in different brainstorming settings. Summary: In this article, the researchers took a survey of perceived productivity in a group setting through brainstorming. When surveyed, individuals were asked about the effectiveness of their performance when they worked alone than when they worked with a group. The individuals perceived their performance much more favorably in a group setting, than when they were in a setting alone, although the conclusions of this study state that no differences were made in the output of material. It was also found that the amount of work perceived to have been completed when working in a group setting, was much greater than when working/brainstorming individually. Conclusion: The authors concluded that while individuals perceived their contributions to more than one other individual while brainstorming to be greater and more effective in a group setting, their additions were no more effective than had they practiced the same brainstorming activity alone. It also appears that in a group setting, individuals take credit for a disproportionate amount of work . Source: Pauhus, P. (1993). Perception of performance in group brainstorming: The illusion of group productivity. Personality and Social Psychology Bulletin, 19(1), 78-89.

Brainstorming Rules as Assigned Goals: Does Brainstorming Really Improve Idea Quantity?

Introduction:

The author, Robert Litchfield, set out to see if rules assigned to brainstorming groups increased or decreased the volume of idea output. He found that “brainstorming rules alone do not convey an advantage over even a vague quantity goal presented alone for enhancing the number of ideas generated in to common tasks.” On the contrary he found that “evidence…that brainstorming rules are useful adjuncts to specific, difficult quantity goals.” Translation: whether or not you to people to brainstorm up X number of ideas doesn’t matter; they’ll always disappoint you no matter what.

Summary:

This paper consists of two studies. Study 1 requested brainstormers to generate 30 ideas in 10 minutes regarding this question: If everyone after 2006 was born with two thumbs on each hand and all of the original fingers, what advantages would that give the person? Participants in Study 2 were told to generate any amount of ideas in 10 minutes surrounding the question, not specifically 30.

In the first study there were 4 separate groups. The first group used brainstorming and only brainstorming (standard rules apply), the second group was with vague quantity goal (produce as many ideas as possible), the third was with a specific but difficult quantity goal (30 ideas in 10 minutes), the last group was with both standard brainstorming techniques and the specific but difficult quantity goal.

The result: the goal of 30 was not achieved in either group. The highest number of ideas was 25. Their results looked like this: (Where n= the number of participants in each group.)

The second study used nearly identical but recreated the “hanger test” used by Gerlach in their study in 1964. These results were much much more interesting. As the charts below show, the range in the number of ideas were higher in groups that needed to produce specific quantities of ideas. The role that ‘brainstorming rules’ played was also quite interesting. The presence of brain storming rules and the presence of a quantity goal produced nearly equal quantities of results while the absence of brainstorming rules and the presence of quantity goals produced equally high results.

Conclusion:

The whole test can be summarized in this table:

There is a time and a place for brainstorming rules. They do play a very active part in volume idea creation. Also, a known goal in a time pressed situation plays an equally important role. The two combined performed the highest in both studies. This says nothing of quality, only quantity. If you’re trying to pump out new ideas, this is certainly the way to go.

Source:

Litchfield, R. (2009). Brainstorming rules as assigned goals: Does brainstorming really improve idea quantity. Retrieved from http://www.springerlink.com/content/t781187427243t17/

This Looked Interesting....

Here's a few interesting articles and a short podcast on the topic of brainstorming. It helped me brainstorm up some ideas for my blog post which is forthcoming.

http://www.newyorker.com/reporting/2012/01/30/120130fa_fact_lehrer

http://www.newyorker.com/online/2012/01/30/120130on_audio_lehrer

http://www.amazon.com/Imagine-Creativity-Works-Jonah-Lehrer/dp/0547386079

Quality, Conformity, and Conflict: Questioning the Assumptions of Osborn's Brainstorming Technique

Introduction:
In the paper, Quality, Conformity, and Conflict: Questioning the Assumptions of Osborn’s Brainstorming Technique, the authors, Olga Goldenberg and Jennifer Wiley, review empirical evidence related to the effectiveness of group brainstorming techniques.  They also question whether some features and assumptions of the traditional brainstorming approach should be reconsidered.

Summary:
In 1963, Alex Faickney Osborn asserted that the key to a successful brainstorming exercise is “quantity, quantity, and more quantity.”  His assumption was based on the idea that better-quality ideas are produced later rather than earlier in the brainstorming session.  Osborn thought that brainstorm participants needed to rid their minds of common, ordinary ideas before the participants can uncover or think up more original, creative solutions to a problems.   

Brainstorming has been criticised because there is little evidence that supports the idea that group brainstorming is superior to individual creative idea generation.  Also, a body of research has emerged that counter the original set of brainstorming rules that ideas are not criticised during the generation process.  It has been suggested that there is an important role of dissent and evaluation in idea generation.   Of course, the literature says there must be a balance between an individual’s fear of their ideas being criticised and the benefits of accountability and constructive criticism in the group setting.  

An emergent theme in brainstorming research is that manipulations of social and situational variables aimed at increasing the number of creative ideas in group interaction may not always improve idea quality.  Some researchers have suggested that there are more direct ways to increase originality and variety of ideas rather than through sheer quantity. Goldenberg and Wiley say that future research into brainstorming include assessment of idea quality in addition to quantity to gain more insight into how each of these indicators are affected by any variable or manipulation that is in question. The authors go on to say that measuring the quality of responses in creative idea generation research offers the potential to learn something above and beyond what can already be gauged with idea quantity.  

Creativity researchers have argued that the generation of a large number of ideas that is achieved by the brainstorming technique is one of many stages in the creative process.  Idea generation is an initial phase that may follow problem identification, but in order for innovation to occur, it must be followed by phases of idea evaluation, selection, and implementation.  Osborn, in 1963, recognized a need for these later phases, but little attention has been paid to them.

Finally, the authors state that most studies into brainstorming are weak because they have been performed using groups of undergraduate students assembled for the purposes of the study. The authors state that more systematic work is needed in an actual organizational setting to better understand innovation and how the climate or culture of an organization may affect brainstorming.

Source:
Goldenberg, O., & Wiley, J. (2011). Quality, conformity, and conflict: Questioning the assumptions of osborn's brainstorming technique. Journal of Problem Solving, 3(2), 96-118.

Cultural Difference and Adaptation of Communication Styles in Computer-Mediated Group Brainstorming

Introduction: 

In the article, “Cultural Difference and Adaptation of Communication Styles in Computer-Mediated Group Brainstorming” the authors examine how culture and medium shape the underlying brainstorming process in today’s society where technology makes it easy for international and intercultural group members to brainstorm together remotely. The authors state that as people’s behavior during group brainstorming is affected by peer evaluation and social conformity, culture and media may influence the extent of evaluation pressure people perceive and change the dynamics of group brainstorming.

Summary:

The authors conducted a laboratory study in which three-person groups performed two brainstorming tasks, one using a text-only chatroom and one using a video-enabled chatroom that showed a view of other group members’ faces. Three person groups were asked to perform two structural similar brainstorming tasks, one via text chatroom and one via video chatroom. American and Chinese participants were assigned to one of four group compositions:

            -Three Americans

            -Three Chinese

            -Two Americans and one Chinese

            -One American and Two Chinese

23 participants were American born with English as their first language. The remaining participants were international students boring in China, Hong Kong or Taiwan whose first language was Chinese but were all fluent or nearly fluent in English. All participants were studying at a U.S. university.

Two brainstorming tasks of equivalent difficulty were created: the “extra thumb” question and the “extra eye” question. The participants were brought to the laboratory and instructed about the brainstorming topics and rules which included (1) the more the ideas the better; (2) the wilder the ideas the better; (3) combination and improvement of ideas are sought; and (4) avoid evaluating others’ ideas. Groups were given about 15 minutes for each brainstorming task. The authors analyzed participants on talkativeness, responsiveness, and individualism/collectivism scale.

Conclusion:

The authors concluded that although Chinese were less talkative in general, the use text-only chatroom increased Chinese participants’ talkativeness. Interestingly, there was cultural adaptation such that Chinese participants became as responsive as Americans when working in mixed-culture groups. The findings demonstrate how cultural factors and medium jointly shape group brainstorming conversations.

Source:

Wang, H-C. , Fussell, S. R., & Setlock, L. D. (2009). Cultural difference and adaptation of communication styles in computer-mediated group brainstorming. Proceedings of CHI 2009. Retrieved from http://sfussell.hci.cornell.edu/pubs/Manuscripts/Wang-CHI2009.pdf