Monday, April 15, 2013

Bayesian Analysis for Intelligence: Some Focus on the Middle East

In a report written for the CIA and declassified in 1994, Nicholas Schweitzer discusses the use of Bayesian analysis and how it can benefit traditional analysis.

Schweitzer begins his report by discussing how much information is generated on a yearly basis and how it is the analysts job to be the funnel for this information. Along with the ever increasing amount of information, the analyst also must deal with very complex situations that they must analyze, usually with models that fail to account for a large amount of complexities. The author believes that by using Bayesian analysis, some of the complexities of situation can be reduced.

Schweitzer discusses the use of Bayesian inference in the analysis of IMINT. Schweitzer gives the example of an analyst attempting to determine whether a a military unit is a motorized rifle battalion or an infantry regiment. In his example, the analyst finds that there are 10 tanks stationed with this unknown military unit. Using either expert opinion or historical observation, the analysts assigns a probability that the there is a 90% chance that the unit is a motorized rifle battalion and a 10% chance that it is an infantry regiment.

In discussing more complex applications, Schweitzer warns the reader that there is rarely objective probabilities of events and that historical observation is not very useful. However, if the analyst can overcome the difficulties of assigning subjective probabilities to events, than Bayesian analysis will allow him to "squeeze a little more information from the data we do receive". With this in mind though, he warns that analysts tend to attribute more precision to a number than they should.

The complex example that Schweitzer creates is four hypothetical scenarios, each involving the potential for war between Israel and Egypt and Syria. The scenarios are: No hostilities are planned by either country for 30 days, Syria, either alone or with other Arab nations, plans to attack Israel within 30 days, Israel is planning an attack with an Arab nation within 30 days, and the last one is that Egypt will disavow the disengagement treaty in the next 30 days. Among the analysts used, the initial probability that they assigned to continued peace was between 70% and 95%. After this, the analysts started to change their estimates based on new evidence that was presented.

The author concludes by discussing the applicability of Bayesian analysis and the types of questions it can be applied to. The questions must have mutually exclusive categories (war, no war), the question has to be expressed as specific hypothetical outcomes, there needs to be a rich flow of data that is related to the situation, and the question must resolve around an activity that produces signs and is not largely a chance or random event.

Critique

The biggest criticism that I see in this report is the lack of an assessment in the Middle East example. The author sets up the four scenarios and gives a preliminary probability of continued peace. He then begins introducing evidence into the question and assigns probabilities to the pieces of evidence and how they apply to a certain scenario. However, he stops there. He does not take the next step to show what the new probability is is each scenario playing out while taking into account the various pieces of evidence. he has an appendix of the formulas used, but he does not show the answer.

The author does a very good job of showing how difficult it can be to set up a question where Bayesian analysis can be applied. This includes is warning about subjectively assigned numbers and how analysts tend to put more belief in them than they should.

Overall, the paper was a good set up to Bayesian analysis and intelligence, it just felt like an important part was missing.

Source:

Schweitzer, N. Bayesian Analysis for Intelligence: Some Focus on the Middle East. Retrieved from: https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol20no2/html/v20i2a03p_0001.htm