Tuesday, April 16, 2013

Bayesian Analysis of Intelligence or Improved Advice to Decision-Makers

Summary:
Decisions associated with protecting critical infrastructures are facilitated through collecting intelligence leading to the formulate courses of action to protect them from adversaries. Sometimes there are time restraints that prevent analysts from evaluating the full scope of a situation and evaluating every piece of data available. During a crisis, analysts must make assessments based on new pieces of information. Bayesian analysis allows decision makers to assess the credibility of potential threats. Researchers have often used Bayesian analysis along side other techniques such as probabilistic risk analysis and game theory to model threat scenarios in hopes of formulating effective responses by identifying vulnerabilities and risks.


According to the authors,’ Bayesian analysis is not very popular in the intelligence community to identify indicators and warnings. The following are the reasons why Bayesian analysis has not been be adopted by the intelligence community.


1.      Bayesian inferences in intelligence have not been defined because analysts are uncomfortable applying probabilistic distributions to events.

2.      It is assumed that analysts often pre-process raw intelligence to produce intelligence reports.

3.      A large number of Bayesian tools evaluate only one hypothesis and cannot be applied to situations where adversaries have more than one strategic interest.
4.      Current Bayesian models cannot handle the short time horizon during a crisis.
5.      There is a lack of need for a clear confidence threshold for decision makers.
6.      The way of updating the prior beliefs about a specific scenario utilizing new pieces of evidence is considered insignificant.   

With consideration of these reasons, the authors proposed a ways of improving the effectiveness of decision made during a crisis. First, they plan on incorporating the moving time horizon to the new model. Secondly, they plan on creating a model that is not hindered by above ideologies common to the intelligence community. The new model would include the following characteristics:

1.      Generalize the Bayesian approach of analyzing intelligence.

2.      Include signals intercepted knowingly and ones gathered through clandestine means.

3.      Recognize denial and deception.

4.      Evaluate the scenario using temporal elements.
5.      Play games that would allow misinterpretation of the data leading to signals directed at a third party.
6.      Identity and apply a threshold for decision making.
7.      Define prior beliefs based on available military and intelligence resources.
8.      Develop conditional probabilities of the existence or the absence of a threat based on new evidence.   

Critique:

Elisabeth Pat-Cornell and David Blum’s article provides a good introduction to the Bayesian analysis with regards to intelligence analysis. Although the authors proposed a new model to make Bayesian analysis more relevant to intelligence analysis, they did not sufficiently test their model by conduct an experiment. Therefore, no evidence is present to assess the effectiveness of their model. In addition, they failed to provide a good definition of the theory as well as the advantages and disadvantages of adopting Bayesian analysis for analyzing intelligence. Bayesian analysis allows for the use of prior knowledge to alongside or to update current knowledge of a scenario. However, Bayesian analysis is sometimes restricted to small sample sizes. Also, there is not valid method of choosing the priors. Each member of a team working to resolve a specific problem may come to the different conclusions depending on the prior they chose.      
 
Source:
Cornell, M., & Blum, D. (n.d.). Bayesian analysis of intelligence or improved advice to decision-makers. Retrieved from http://create.usc.edu/2010/06/bayesian_analysis_of_intellige.html

4 comments:

  1. This article takes a different stance in terms of Bayesian Analysis and intelligence. In my research I only found articles arguing in favor of the benefits that Bayesian Analysis could bring to analysts and the intelligence community overall. It is interesting to look at Bayesian analysis in this way. Additionally, I like the overall setup of the article and the list format of the issues presented, followed by the suggestions for a new model to improve Bayesian Analysis for the intelligence community.

    I agree with your critique that simply listing the components of the new model is not sufficient. Testing the new model in order to prove it will be more useful to the intelligence community is necessary and would further strengthen their argument of its lack of adoption into the intelligence community.

    ReplyDelete
  2. Some of the reasons the authors give for the failure of wide adoption of Bayesian analysis in the intelligence community were also found in my research, specifically the hypothesis limit. My research found that Bayesian analysis is limited to evaluating yes and no type questions. It fails to take into account more complex scenarios. The second criticism they address, the pre-processing of raw intelligence to create reports, sounds similar to the types of biases that we have discussed in previous classes, such as confirmation bias.

    ReplyDelete
  3. This article took a different approach to explaining Bayesian theory than others that I've seen. Although I'm not sure how effective it was in proving its point, I like the application to the intelligence community and the fact that there was an attempt to improve its effectiveness in that particular field. I do agree that there should have been a stronger attempt at demonstrating the effectiveness of their model because it seems to hold potential for the IC in decision-making during a crisis.

    ReplyDelete
  4. Your critique of the research is really good and I agree with you on the various points, particularly that as great as their model sounds, it is untested.

    One additional thing I think is missing from this research is the idea of in what OTHER circumstances those signals, as noted in their second characteristic, would occur which detract from the strength of that signal related to the probability.

    ReplyDelete