Bayes theorem of intelligence analysis

Summary
In his research on bayes probabilities, Zlotnich (1970) identifies bayes as a potentially useful tool for intelligence analysts. This reasoning lays in the nature of intelligence work.  As intelligence analysts, we work with uncertainties everyday. Bayes analysis understands this and uses probabilities to help identify the likelihood of events occurring based on past occurrences, such as an intelligence analyst would do.

In a very simplistic way, Zlotnich's bayes formula is R=PL.  R is the rivised estimate of the probability of an event occurring after new evidence has been taken into consideration. P is the previous estimate likelihood of the event occurring before the new piece of evidence. L is the estimate of the probobalitliy of the event occurring based only on this new piece of evidence.

Zlotnich identifies two main differences between intelligence analysis and bayes,
1) Bayes requires that analysts define a qualitative value to an estimate. Analysts are familiar with providing estimator words of probability, such as likely, unlikely, and almost certain. Using bayes, they must turn those words, into numbers.

2) Bayes allows the analyst to look and examine pieces of evidence individually, instead of creating an estimate based on the cumulative summary of the evidence. The math will provide the overall estimate of all the evidence combined. According to Zlotnich, his experiences and research suggest that analysts are often better at making estimates off one piece of evidence as opposed to having to summarize a conglomerate of evidence.

There are some problems with bayes, one being the presence of suspect evidence. Not all evidence comes from completely reliable and accurate sources. The analyst often has to judge the reliability of the information and account for that in his estimate. Unfortunetly, analysts will often weigh source reliability with their own opinion of the hypothesis. If information comes from a questionable source that is counter to what the analyst believes, the analyst may see the source as low credibility.

In addition, another issue faced by analysts using bayes is the passage of time and the erosion of evidence. As time passes, pieces of evidence often lose weight. A piece of evidence that was found a month ago is likely to not have the same weight as it did a month later.  Analysts must find a way to account for this in their analysis. One simple way is for the analyst to periodical go back and re-analyze pieces of evidence to see of they are still relevant or are out of date.

Critique

Zlotnich's article on the use of bayes, while dated, was informative and well written. Some of the issues he identified have been partially resolved today. For example, research has been done to help assign words of estimator probability with percentages (Kessleman). Also, to address source reliability, there has been research into the characteristics of a good source and actually being able to give a source a credibility ranking (Norman).  Zotnich provided an excellent outline of the benifits of bayes as well as many of the issues we as analysts face. Luckily, some of these issues had been identified and their are ways now around them.

Source

Zlotnich, Jack (1970). Bayes Theorem for intelligence analysis. CIA.