Summary
Beliefs are
based on probabilistic information in Bayes Theorem. This method is used to measure incomplete
knowledge and uncertainty. After observing new conditions, our initial beliefs
are updated. At a glance, one can see that Bayes Rule identifies our initial beliefs
as having a high margin of error. Analytic confidence or rigor increases as we
observe more conditional events. According to the author, Bayes Theorem is
“highly subjective and somewhat controversial compared to more objective
probability theories in statistics.”
·
The
author’s main argument is that “Intelligence analysis uses Bayes Theorem and is
very subjective.” Below is a list of some of his chief arguments:
- The prior belief is updated to a posterior belief after the observation of conditional events.
- This is a variant of the Bayes Rule formula, where P is probability, C is the conditional event, and O is the observation, and ¬ is not.
- p(C|O) = p(O|C)p(C)/p(O|C)p(C) + p(O|¬C)p(¬C)
- In English, the posterior belief is equal to the prior belief divided by the marginal probability.
- Bayesian probability produces interesting results because it accounts for uncertainties created by False Positives and False Negatives.
Intelligence
uses predictive analysis to predict a range of events in the face of extreme
uncertainty. Yes, it is possible. It predicts the probability of events, but it
cannot predict which events will occur.
In the case
of law enforcement, suspects are carefully vetted several times and on occasion
this broadens their field of suspects by the accumulation of evidence while increasing
their certainty.
This is why
Police carefully investigate suspects many times, and occasionally widen their
field of suspects if they believe their initial investigation led them in the
wrong direction. The following is an example breakdown a law enforcement
application of Bayes:
There are
10,000 civilians. 1% of whom are insurgents pretending to be civilians. Police
can investigate individuals and determine if they are an insurgent or civilian
with 95% certainty.
Prior
Probability is this: 0.01 (10,000) and 0.99(10,000). So
Group
1: 100 insurgents
Group
2: 9,900 Civilians
The
Police investigate the entire population. This produces four groups:
Group
1: Insurgents – Positive test (0.95)
Group
2: Insurgents – False Negative test (0.05)
Group
3: Civilians – False Positive test (0.05)
Group
4: Civilians – Negative test (0.95)
How certain
are the police that the men they captured are actually insurgents? The answer
is 16%.
(0.95 x
0.01)/ (0.95 x 0.01) + (0.05 x 0.99) =
0.0095/0.0590
= 0.161
According to
the author, the result is very counterintuitive due to the high level of
uncertainty created by false negatives and false positives. For example:
“If police
investigate the entire population, up to 6% of the population will test
positive for being insurgents. But we know only 1% of them can be insurgents
and the others are innocent. We also know some insurgents may have escaped
detection. Thus the 16% certainty.”
Critique:
Overall, I agree with the author’s position on Bayes Theorem and intelligence. He also seeks to
address the wrong perception that intelligence analyst predict future events.
The application of Bayes theory for detecting a black swan, for instance, is
near impossible with the application of Bayes. This is due to Bayes reliance on
scenarios which have already happen. Despite the fact Bayes will not predict a
black swan, Bayes remains useful for forecasting.
Source Link: https://netwar.wordpress.com/2007/07/28/bayes-theorem-and-intelligence/
Posts
Based upon your critique, are you asserting that prior scenarios decrease or increase forecasting accuracy when using Bayes?
ReplyDeleteOverall, prior scenarios increase forecasting accuracy. This is, however, not always the case because it does not account for the alternative outcomes. To me, Bayes only accounts for the most likely outcomes and it is not useful when predicting surprise. Would I use Bayes to predict future IED emplacement along a route? Yes. Am I going to place all my bets on Bayes? No. However, my analytic confidence will be higher when I make a decision. On more qualitative problems, such as those presented in intelligence, I assert forecasting accuracy increases but not as much as it does when attempting to use Bayes for quantitative problem sets. What about you? Do you believe prior scenarios decrease or increase forecasting accuracy?
ReplyDeleteI felt that this article did a good job demonstrating some of the underlying fundementals of the mathematics involved with Bayes. Most articles I have seen either glaze over the math or become too bogged down in formuli and terminology. Overall I walked away with a better understanding of Bayes than I had prior to reading it. I preticularly liked the focus on how the process is dependent on prior occurrences and is not in itself looking forward.
ReplyDelete