Collaborative Intelligence Analysis with CACHE: Bias Reduction and Information Coverage
By: Matt Haines
Summary:
Gregorio Convertino, Dorrit Billman, Peter Pirolli, JP Masur, and Jeff Shrager created a virtual environment to conduct standard analysis of competing hypotheses(ACH) and then they analyzed the effects of that environment. The authors begin by defining CACHE as a collaborative analysis of competing hypotheses environment. Then, they explain the difficulties an analyst faces in everyday intelligence tasks. An analyst is faced with tasks that span a vast multitude of areas of expertise on a daily basis and biases influence all of those analytical products. The authors then go into detail of what the CACHE framework actually does in order to combat this challenge. CACHE allows a user to search through all available evidence, input that evidence into a personal ACH matrix, view other team members ACH matrix, and communicate with other team members through an instant messaging system.
Before completing the actual test of the CACHE framework, the authors hypothesized that:
Heterogeneous groups would show less confirmation bias than Homogeneous groups. Because CACHE supports sharing information among participants, the differing views in the heterogeneous groups should mitigate cognitive biases by 1) exposure to more, and less-biased, evidence and 2) access to alternative analyses provided by partners.
and that, “Heterogeneous groups would show no net process loss relative to the Solo/Nominal Group. CACHE will mitigate the process costs, producing equivalent or better performance in heterogeneous groups.” The results of the experiments were concurrent with the authors’ hypotheses.
Critique:
The CACHE framework is a great idea and prototype for groups where not every person can be in the same location at the same time. However, the authors of this paper did not do much to actually prove anything. This paper laid out a product. It did not add to the ACH process nor did it attempt to contest normal assumptions of ACH. However, CACHE has achieved something just by allowing analysts to be in two different places at once and collaborate with each other. This feature can help eliminate some group think biases because it takes some power away from those members of the team who are better presenters. For example, one of the major complaints many international students have, is that they feel like their ideas are not heard, because they cannot vocally command a room. By allowing analysts to work remotely, international students can have the same voice as a native english speaker.
Posts
I agree with the premise of using CACHE as a means of easing communication for international students. However, unless the presentation is via internet, communication issues may still occur due to the audience's possible inability to understand various accents. This would be a good team exercise though.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteI like how you mention that one of the positive features of CACHE is that it allows analysts to be in different locations and have their voice heard. I definitely agree that can help eliminate group think and make it so the group does not listen to only the loudest person in the room. This should allow everyone to be able to state their views, as overcoming group think can be very difficult.
ReplyDeleteThe CACHE system seems to be a re-branded version of nominal group technique and used solely on ACH matrices. While nominal group technique is a great way to hear everyone’s voice equally and group important ideas. However by using this technique it would be interesting to see if by using this brainstorming technique along with the ACH matrices if it eliminates some of the limitations of both methodologies. For example would plugging information brainstormed through nominal group technique help analysts more effectively avoid group-think, confront biases and prioritize information faster or better than before?
ReplyDeleteWhile reading the article, I also felt that the author was trying to sell the product instead simplifying or adding to the ACH process. As mention before in other post, bias is a weakness that factors into the ACH process. With the CACHE framework, it could help reduce group think biases and add value to other opinions or ideas.
ReplyDeleteWe actually did CACHE in Mills (I was with Pouch and Kelsey), just not virtually. It works well as long as it removes biases through good discussion and doesnt add new biases. It would be good to test with a few dozen teams of four and see how the control groups working alone do compared to the CACHE groups.
ReplyDeleteMatt, I would have to agree that this did not contribute much to the process of ACH itself. The hypothesis that heterogeneous groups will have less bias than homogeneous groups is not a strong, or novel theory at all. There is power in diversity of thought, this has been proven time and again. Overall, I think heterogeneous groups should be all but mandated in ACH and analysis in general to reduce bias as a whole.
ReplyDelete