Bayesian Analysis of Intelligence or Improved Advice to Decision-Makers

Introduction:

Although not the standard article, M. Elisabeth Pat-Cornell and David M. Blum’s ongoing research into the use of Bayesian analysis in intelligence problems is extremely relevant to the current subject matter. Their work builds on previous and ongoing research conducted by the National Center for Risk and Economic Analysis of Terrorism Events (CREATE).

(http://create.usc.edu/)

Summary:

According to the article, one of the main problems facing US national and homeland security is the response to very-near future threats. While longer term threats allow the time to build reports and plan courses of action, near term threats do not. As a result, analysts need to be able to judge the reliability of the new threat information in the context of all available intelligence in order to both minimize risk as well as responses to false threats. Researchers at CREATE have previously determined that Bayesian analysis is useful in such situations, as a way to gauge the credibility of potential threat scenarios. Furthermore, Bayesian analysis has been used in conjunction with various other analytical approaches, including probabilistic risk analysis, game theory, and Markov models.

Although the use of Bayesian analysis to measure threats is not new, it has not yet been adopted by the intelligence community, for several reasons:

1) the idea of the prior in intelligence has not been well defined;

2) academic research tends to assume a substantial amount of pre-processing by analysts to produce intelligence reports from raw intelligence feeds;

3) many Bayesian tools evaluate only a single hypothesis, ignoring multiple strategic interests;

4) crises imply a short but moving time horizon, which current models lack;

5) the process through which new intelligence data relating to a threat updates the prior belief about the threat has been considered trivial.

This new research seeks to remove these obstacles by incorporating a moving time-horizon into dynamic signaling games to better simulate crises, and also by creating a new model which will eliminate the intelligence community’s resistance to Bayesian techniques. The researchers then go through an in-depth research proposal, a case study, and the deliverables, of which the final results will be released in August 2012.

Further Readings:

Another research project on a similar topic is Bayesian Approach to Intelligence Analysis: (http://create.usc.edu/2011/03/bayesian_approach_to_intellige.html)

History of Bayesian analysis in risk assessment: http://www.usc.edu/dept/create/assets/001/50765.pdf

Probabilistic Modeling of Terrorist Threats: A Systems Analysis Approach to Setting Priorities Among Countermeasures: http://www.ingentaconnect.com/content/mors/mor/2002/00000007/00000004/art00004 (Purchase required)

Source:

http://create.usc.edu/2010/06/bayesian_analysis_of_intellige.html

Bayes' Theorem for Intelligence Analysis

Jack Zlotnick
CIA Historical Review Program

Author’s Note: Released by the CIA’s Historical Program in the early 1990’s, Jack Zlotnick wrote this piece in the 1970’s. At the time, the CIA was still in the process of testing Bayes’ Theorem. Due to the ongoing testing period (at that time), Zlotnick does not offer a position on the utility and validity of the Bayesian method with regards to intelligence. In fact, Zlotnick spends a considerable amount of time in the article discussing the ways the theorem should continue to be tested.

Summary
Due to the very nature of intelligence, analysts should be naturally interested in the Bayesian Theorem. Intelligence is probabilistic in nature. Intelligence analysts usually conduct their analysis based on incomplete evidence in which they must address probabilities (thus WOEP’s).

For intelligence applications, Bayes’ Theorem is represented by the equation R=PL. “R” is the revised estimate of the odds favoring one hypothesis over another competing hypothesis (the odds of a particular hypothesis occurring after new evidence is entered into the equation). “P” is the prior estimate on the hypotheses probabilities (the odds before considering the new evidence entered into the equation). The analyst must offer judgments about “L” or the likelihood ratio. This variable is the analyst’s evaluation of the “diagnosticity” of an item of evidence. For instance, if a foreign power mobilizes its troops, what are the chances that “X” will happen over “Y”.

The principle features of the Bayes Theorem distinguish it from conventional intelligence analysis in three ways. First, it forces analysts to quantify judgments that are not ordinarily expressed in numeric terms. Second, the analyst does not take the available evidence as given and draw conclusions. And third, the analyst makes his/her own judgments about the bits and pieces of evidence. He/she does not sum up the evidence as he/she would if he/she had to judge its meaning for a final conclusion. The mathematics does the summing up.

The author is skeptical that the complex tasks analysts are forced to consider can be reduced to numeric values. Bayes’ Theorem, however, may be useful for examining strategic warning by uncovering patterns of activity by foreign powers.