Summary
The
September 11th attacks should be viewed as one defense failure in a
series of foiled attempts. In risk
management, only defensive failures are broadcasted while successes often go
unnoticed to the public. The attacks
were successful due to a lack of information sharing. In regards to the little amount of
information that was shared, intelligence professionals were not able to
measure the value of particular pieces of information. In other words, they did not effectively find
signals in noise.
According
to Paté-Cornell, Bayesian models provide the efficient means to find signals in
voluminous amounts of noisy information.
Bayesian models calculate the probability of an event given a new signal
and a base-line probability or the priority prior to the new signal. A basic Bayesian model should contain a prior
probability estimate, a probability estimate given a new piece of evidence, and
a probability assessment of the relevance of the new piece of evidence.
Paté-Cornell
takes a Bayesian model for fusion intelligence one step further. She introduces various steps in the model in
order to account for the possibilities of a false positive and a false negative
given a new piece of evidence (or missed evidence). This approach is normally used in engineering
risk analysis, but it is logically applicable to intelligence work. The generalized formula is shown below.
Critique
Unfortunately
for Paté-Cornell, not all intelligence analysts are math fans. The extra requirements of estimating false
positives and false negatives, while logically sound, are difficult to do. Intelligence professions operate in a world
of unknown unknowns arguably more than any other profession. Quadrant Crunching™ provides a qualitative
and visual alternative for those who opt not to stress over numbers. With that being said, it is highly
commendable to incorporate uncertainty into the equation.
A
simpler, more user-friendly Bayesian model would be one given by Nate Silver in
his book, Signal and the Noise: Why So
Many Predictions Fail – but Some Don’t.
Silver’s Bayes contains the three, known necessary variables, but only
includes one unknown one dealing with the new signal. The way it is structured, the prior
probability or baseline is resilient in the face of new information.
“x”
is the prior probability, “y” is the probability of the signal resulting from a
particular event, and “z” is the
probability of the signal being unrelated to the same event considered in “y”. This formula is much simpler and, according
to Silver, can lead to “vast predictive insights.”
Source
Pate-Cornell,
E. (2002). Fusion of intelligence information: A Bayesian approach. Risk
Analysis, 22(3), 445–454.
Posts
Kyle,
ReplyDeleteI noticed that several articles reviewed this week make references to using Bayesian methods in teams, instead of by individuals like in the article I reviewed.
I can envision some mayhem in a group context when deciding how to account for false positives and false negatives as prescribed by Paté-Cornell.
Do you think that the more user-friendly model by Nate Silver is more conducive to team forecasts due to its relative simplicity?
Bayesian, no matter if it's a model by Paté-Cornell or Silver, can use group averages in its estimates. So each individual produces their own estimates, then all of the individual estimates are averaged in a separate Bayesian model. Aggregated estimates are offered all of the time.
ReplyDeleteRegardless of which model is used, I suspect that simultaneous group work on one Bayesian model would be difficult in too large of a group.