Friday, October 2, 2015

Bayes Theorem and Intelligence


Summary
Beliefs are based on probabilistic information in Bayes Theorem.  This method is used to measure incomplete knowledge and uncertainty. After observing new conditions, our initial beliefs are updated. At a glance, one can see that Bayes Rule identifies our initial beliefs as having a high margin of error. Analytic confidence or rigor increases as we observe more conditional events. According to the author, Bayes Theorem is “highly subjective and somewhat controversial compared to more objective probability theories in statistics.”

·         The author’s main argument is that “Intelligence analysis uses Bayes Theorem and is very subjective.” Below is a list of some of his chief arguments:

  • The prior belief is updated to a posterior belief after the observation of conditional events.
  • This is a variant of the Bayes Rule formula, where P is probability, C is the conditional event, and O is the observation, and ¬ is not.    
  •  p(C|O) = p(O|C)p(C)/p(O|C)p(C) + p(O|¬C)p(¬C)
  •  In English, the posterior belief is equal to the prior belief divided by the marginal probability.
  • Bayesian probability produces interesting results because it accounts for uncertainties created by False Positives and False Negatives.
Intelligence uses predictive analysis to predict a range of events in the face of extreme uncertainty. Yes, it is possible. It predicts the probability of events, but it cannot predict which events will occur.

In the case of law enforcement, suspects are carefully vetted several times and on occasion this broadens their field of suspects by the accumulation of evidence while increasing their certainty.

This is why Police carefully investigate suspects many times, and occasionally widen their field of suspects if they believe their initial investigation led them in the wrong direction. The following is an example breakdown a law enforcement application of Bayes:

There are 10,000 civilians. 1% of whom are insurgents pretending to be civilians. Police can investigate individuals and determine if they are an insurgent or civilian with 95% certainty.

Prior Probability is this: 0.01 (10,000) and 0.99(10,000). So
Group 1: 100 insurgents
Group 2: 9,900 Civilians
The Police investigate the entire population. This produces four groups:
Group 1: Insurgents – Positive test (0.95)
Group 2: Insurgents – False Negative test (0.05)
Group 3: Civilians – False Positive test (0.05)
Group 4: Civilians – Negative test (0.95)

How certain are the police that the men they captured are actually insurgents? The answer is 16%.
(0.95 x 0.01)/ (0.95 x 0.01) + (0.05 x 0.99) =
0.0095/0.0590 = 0.161

According to the author, the result is very counterintuitive due to the high level of uncertainty created by false negatives and false positives. For example:

“If police investigate the entire population, up to 6% of the population will test positive for being insurgents. But we know only 1% of them can be insurgents and the others are innocent. We also know some insurgents may have escaped detection. Thus the 16% certainty.”

Critique: Overall, I agree with the author’s position on Bayes Theorem and intelligence. He also seeks to address the wrong perception that intelligence analyst predict future events. The application of Bayes theory for detecting a black swan, for instance, is near impossible with the application of Bayes. This is due to Bayes reliance on scenarios which have already happen. Despite the fact Bayes will not predict a black swan, Bayes remains useful for forecasting.

Source Link:  https://netwar.wordpress.com/2007/07/28/bayes-theorem-and-intelligence/

3 comments:

  1. Based upon your critique, are you asserting that prior scenarios decrease or increase forecasting accuracy when using Bayes?

    ReplyDelete
  2. Overall, prior scenarios increase forecasting accuracy. This is, however, not always the case because it does not account for the alternative outcomes. To me, Bayes only accounts for the most likely outcomes and it is not useful when predicting surprise. Would I use Bayes to predict future IED emplacement along a route? Yes. Am I going to place all my bets on Bayes? No. However, my analytic confidence will be higher when I make a decision. On more qualitative problems, such as those presented in intelligence, I assert forecasting accuracy increases but not as much as it does when attempting to use Bayes for quantitative problem sets. What about you? Do you believe prior scenarios decrease or increase forecasting accuracy?

    ReplyDelete
  3. I felt that this article did a good job demonstrating some of the underlying fundementals of the mathematics involved with Bayes. Most articles I have seen either glaze over the math or become too bogged down in formuli and terminology. Overall I walked away with a better understanding of Bayes than I had prior to reading it. I preticularly liked the focus on how the process is dependent on prior occurrences and is not in itself looking forward.

    ReplyDelete