Source: https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol20no2/html/v20i2a03p_0001.htm
Summary:
Nicholas Schweitzer introduces this report for the CIA by briefly discussing one problem intelligence analysts face today: the 'information explosion'; intelligence analysts struggle to filter through the copious amount of information with their limited resources. Another problem Schweitzer acknowledges is the difficult questions analysts have to answer in an increasingly complex world. One way to ease these problems is through the application of analytic methods.
Schweitzer introduces the Bayesian analysis as a simple mathematical equation: A = B x C. He explains it further as "the probability of an underlying cause (hypothesis) equals its previous probability multiplied by the probability of the observed event was caused by that hypothesis". Schweitzer describes how analysts use Bayes to predict complex political outcomes. Analysts use estimated probabilities since the exact probability is impossible to evaluate. To generate these probabilities, analysts can use historical or expert opinion. Schweitzer even says analysts can incorporate the Delphi method, which uses expert opinions, alongside Bayes. The approximated probabilities provide a "starting point" of analysis
Strengths:
- Provides a wide range of expertise instead of depending on a single analyst
- More information can be extracted from data
- The process is reproducible
- The end result can be displayed on a numerical scale
- Analysts are forced to look at other perspectives
Weaknesses:
- Can only be applied to certain questions (i.e. war or no war?)
- Must be expressed as a specific set of hypothetical outcomes
- Must follow mathematical principles
- Needs to have a large amount of data which increases the probability of unreliable or even negative information
- Can be manipulated by cognitive bias
- Cannot be applied to crisis situations due to the need for time
Critique:
The author uses great examples, especially to stress the complexity of intelligence questions and defining the kinds of questions that work with Bayes. I also liked how Schweitzer discussed its applicability with the Delphi method because it is unlikely an analyst will only use one technique to come to a final conclusion on a difficult issue. However, there was not much explanation into the process of setting up Bayesian analysis (especially regarding the examples of the Middle East), and interpreting the results. I think this report does provide readers with a basic grasp on how Bayes works, but would not be enough of an explanation for analysts to use as a how-to guide.
Posts
We discussed in the class that prediction market analysis are not enough to respond complex "intelligence questions" of the modern world. What is your opinion regarding Bayesian Analysis in terms of responding the complex intelligence questions?
ReplyDeleteFrom what I've read about Bayes, it seems just like prediction markets, Bayesian analysis can only be applied to zero-sum questions. I listed the example that the author gave in the article with will there be a war or no war at all? Unfortunately in the intelligence community, we do not always deal with those kinds of questions. I would use this as a method to provide a starting point for analysis but perhaps not as a concluding estimate.
DeleteI agree with Osman, Bayesian Analysis is similar to predication markets in that it attempts to answer a “yes or no” answer. The difference between the two seems that one has a statistical formula behind the question, while the latter is an aggregate of beliefs. It appears that the Bayesian is only useful in certain complex applications, like the military example, where a process of deduction can solve the problem. When applying Bayesian to the political spectrum, it appears the problem becomes exacerbated over time as political situations are always in flux.
ReplyDeleteI agree. I believe Bayes is well suited for military problems since you can measure things such as capabilities and soldier experience whereas the solutions to political problems are unclear.
DeleteDevon-You mentioned the author lacks in his "how to explanation" of Bayesian Analysis. I think this is because Bayesian Analysis does not heavily rely on the analyst and a specific process. In other words, Bayesian Analysis is highly repetitive and it relies on big data to do the crunching. More recently, Bayesian Analysis seems to be done using complex algorithms to conduct the analysis. Any thoughts on this?
ReplyDeleteI agree. I think my expectations for this article were to receive a how-to conduct Bayes after the author introduced the article with some of the problems analysts see today. Even though it is a numbers game, I wanted to see how analysts landed on their probabilities, especially when the questions were so complex.
DeleteI think it is useless to seek for a technique that fits all intelligence questions ranging from complex to simple ones. Therefore, Bayes method also isn't appropriate for every intelligence questions. With this in mind, Devon, does author mention about that can any one who wants to apply this technique conduct it successfully or does that require expertise to some extent?
ReplyDeleteThe only expertise the author seems to mention in this article is that regarding the material or subject at hand and not the Bayesian method. I would hope that analysts would at least have been tested on their basic knowledge of the method so they know that their calculations are correct. I mean, I'm sure they've passed Math 101, but these are intelligence analysts and mathematicians so there is room for error regarding Bayes.
DeleteI agree. I think Bayes method is useful for intelligence questions that can be answered through the exploration of history data.
ReplyDeleteI agree that the example is a good way to show how Bayes can work. I think you kinda alluded to it, I do think something like the Middle East is usually too ambiguous to accurately use Bayes from my knowledge of it. I do like that they combined another method (Delphi) with Bayes, while it may not work in every scenario I think it would usually add nuance to an analytical piece.
ReplyDelete