In a report written for the CIA and declassified in 1994, Nicholas Schweitzer discusses the use of Bayesian analysis and how it can benefit traditional analysis.
Schweitzer begins his report by discussing how much information is generated on a yearly basis and how it is the analysts job to be the funnel for this information. Along with the ever increasing amount of information, the analyst also must deal with very complex situations that they must analyze, usually with models that fail to account for a large amount of complexities. The author believes that by using Bayesian analysis, some of the complexities of situation can be reduced.
Schweitzer discusses the use of Bayesian inference in the analysis of IMINT. Schweitzer gives the example of an analyst attempting to determine whether a a military unit is a motorized rifle battalion or an infantry regiment. In his example, the analyst finds that there are 10 tanks stationed with this unknown military unit. Using either expert opinion or historical observation, the analysts assigns a probability that the there is a 90% chance that the unit is a motorized rifle battalion and a 10% chance that it is an infantry regiment.
In discussing more complex applications, Schweitzer warns the reader that there is rarely objective probabilities of events and that historical observation is not very useful. However, if the analyst can overcome the difficulties of assigning subjective probabilities to events, than Bayesian analysis will allow him to "squeeze a little more information from the data we do receive". With this in mind though, he warns that analysts tend to attribute more precision to a number than they should.
The complex example that Schweitzer creates is four hypothetical scenarios, each involving the potential for war between Israel and Egypt and Syria. The scenarios are: No hostilities are planned by either country for 30 days, Syria, either alone or with other Arab nations, plans to attack Israel within 30 days, Israel is planning an attack with an Arab nation within 30 days, and the last one is that Egypt will disavow the disengagement treaty in the next 30 days. Among the analysts used, the initial probability that they assigned to continued peace was between 70% and 95%. After this, the analysts started to change their estimates based on new evidence that was presented.
The author concludes by discussing the applicability of Bayesian analysis and the types of questions it can be applied to. The questions must have mutually exclusive categories (war, no war), the question has to be expressed as specific hypothetical outcomes, there needs to be a rich flow of data that is related to the situation, and the question must resolve around an activity that produces signs and is not largely a chance or random event.
Critique
The biggest criticism that I see in this report is the lack of an assessment in the Middle East example. The author sets up the four scenarios and gives a preliminary probability of continued peace. He then begins introducing evidence into the question and assigns probabilities to the pieces of evidence and how they apply to a certain scenario. However, he stops there. He does not take the next step to show what the new probability is is each scenario playing out while taking into account the various pieces of evidence. he has an appendix of the formulas used, but he does not show the answer.
The author does a very good job of showing how difficult it can be to set up a question where Bayesian analysis can be applied. This includes is warning about subjectively assigned numbers and how analysts tend to put more belief in them than they should.
Overall, the paper was a good set up to Bayesian analysis and intelligence, it just felt like an important part was missing.
Source:
Schweitzer, N. Bayesian Analysis for Intelligence: Some Focus on the Middle East. Retrieved from: https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol20no2/html/v20i2a03p_0001.htm
Subscribe to:
Post Comments (Atom)
Posts
Ethan -- I agree with your assessment. It seems that the application of Bayesian Analysis to Intelligence related issues is particularly important. Though, it does seem to be slightly contradictory in the sense that after all of the propositions and explanations, there is no direct or actionable information that results from the use of Bayesian Analysis. I thought it was very interesting to see the application, especially from a work published by a US intelligence agency, but a more concrete answer would have made a more convincing argument.
ReplyDeleteBased on your summary and critique of the article I find it odd that the author went to the lengths to create a hypothetical scenario to analyze with bayesian methods, but didn't show how the new probabilities provided by bayesian methods would alter the existing estimate for each possible scenario. The whole point of using bayesian analysis according to my understanding of it is to bring in new pieces of evidence that change the probabilistic estimate, usually creating more reliable estimates overall. However, I do find some merit to the study you summarized and critiqued. The aspect of the study that I found relevant was bayesian analysis' ability to help with the complexity of issues brought to the intelligence analysts' attention. Being able to utilize more pieces of evidence in a complex scenario can allow the analyst to reduce the level of complexity of the problem, while simultaneously reducing their uncertainty of their probability estimate.
ReplyDeleteI thought this article applied Bayesian Analysis effectively to intelligence as well as to a real world example of a problem many analysts likely face each day. That said, I also agree with your critique. I think by expanding upon the study and showing the new probability the study would have been more well-rounded overall.
ReplyDeleteAlso, the author's belief that Bayesian Analysis can reduce complexities of a problem is an odd statement to me. I think Bayesian can possibly reduce the complexities of a problem in terms of how an analyst thinks about the problem rather than the problem overall.
Ethan, your article focuses on a different aspect of the Bayesian analysis than the article I summarized this week. According to your article Bayesian analysis can reduce the complexity of a situation where as my article addressed that often Bayesian analysis can only handle complex hypothesis. In your article, the author managed to provide examples of Bayesian analysis in use for varying degrees of complexity. Like you, I wish the authors had assigned probabilities to the second example and formulated an assessment. That would have solidified the idea that Bayesian analysis can handle complex situations. I also though it was interesting that article emphasized the use of probabilities during Bayesian analysis and objective probabilities are rare. This reiterates what we have learned in the methodology class. Everyone has biases.
ReplyDelete