Creation Of A National Institute For Analytic Methods

Steven Rieber and Neil Thomas

Article Summary
Rieber and Thomas begin the article with criticism of recent commissions from the Senate Select Committee on Intelligence and the Special Presidential Commission on Iraq Weapons of Mass Destruction. Both commissions recommended creating positions for "mission managers" or subject matter experts. Rieber goes on to explain and give examples of how expert opinions are often wrong despite years of experience and that the only way to know if "conventional wisdom" is correct, there must be in depth scientific study of the subject matter. Both commissions mentioned above advocate greater use of devil's advocacy in intelligence analysis however they do not cite any research validating the method. Rieber cites the book Groupthink by Irving Janis who argues that devil's advocacy can create a false sense of comfort that their decision making is sound just because they considered an opposite viewpoint and that their original policy choice has stood-up to scrutiny.

Rieber and Thomas argue that "The first element in improving the process of improving analysis is to find out what the existing scientific research says." From their research they identify argument mapping as one of the most promising method to "improve human judgment."

The way a method should prove itself is by being subjected to scientific studies that to control for outside influences, point out causation from correlation, and reveal significant facts. It also needs to determined to what extent the method improves analytical judgements and in which domains (political, economic, military,). The method needs to be teachable, and analysts must be willing to use the method.

The article continues by advocating for the creation of a National Institute for Analytical Methods (NIAM). The institute's function would be similar to that of the National Institute of Health (NIH). The NIH conducts its own research as well as funds research. With the right funding and staffing a NIAM could provide continued evidenced based insight about various methods of analysis that could show potential for the intelligence community.

Author's Note: At the time of this article's publication in 2005 Steven Rieber was a scholar at the Kent Center for Analytic Tradecraft. He co-wrote this article with Neil Thomas who is a lecturer in Philosophy of Science and History at the University of Melbourne. It is important to note that Tim van Gelder is Principal Fellow in the Philosophy Department at the University of Melbourne and is an instrumental force in the development of critical thinking and argument mapping teaching methods and software. His work is cited several times in this article.

During a class discussion Professor Wheaton asked if any of us had come across the name Steven Rieber in our research of articles relating to argument mapping. He then explained to the class that Steven Rieber currently works as an analytic methodologist at the Office of Analytic Integrity and Standards within the Office of the Director of National Intelligence (DNI). Rieber has instituted the teaching of argument mapping into the DNI analysts training curriculum. Because the above article was published in 2005 and that argument mapping was only method mentioned more than once, out the seven suggested, it is highly likley that argument mapping passed most of the criteon mentioned above for being a viable method for better intelligence analysis.