Tuesday, March 20, 2012

Decision Trees in Intelligence Analysis

Introduction:

Written during the Cold War, Edwin Sapp’s article on decision trees begins by noting the large amount of information available to intelligence analysts, the sophistication and seriousness of the weapons available, and ever-shortening time span in which effective decisions must be made. Sapp advocates the decision tree as a competent method for processing large amounts of data, while at the same time communicating degrees of certainty to an extent which language cannot.

Summary:

Edwin Greenlaw Sapp begins his article entitled Decision Trees by discussing the nature of intelligence analysis. He identifies four major categories that intelligence requirements fall into:

1) places (geographic locations, physical resources);

2) people (strengths and attitudes);

3) organizations (what people form and belong to); and

4) objects (things people make and possess; cities and weapons systems

Information is collected in these categories in order to make forecasts about futures states of affair. In Sapp’s view, Kent’s words of estimative probability are both vague and incomplete. He finds the decision tree more favorable, as it serves to “organize sizeable amounts of data…and communicate the degree of certainty relating to possible outcomes or the likelihood of the occurrence of specific events at some given time in the future.”

Sapp goes on to explain the technique used to create decision trees with two examples. His first is a simplistic question displaying the use of probabilities and subjective value of the outcomes to determine whether or not to hold a dinner party outside given a 40 percent chance of rain. The second example uses the Biblical story of the spies sent into Canaan. Sapp illustrates the intelligence cycle and how a decision tree would have aided this cycle by linking all the collection requirements and ensuring that the important questions were answered first.

According to Sapp there are two types of decision trees: deterministic and probabilistic. Deterministic decision trees deal with situations where no probability needs to be calculated in order to solve the problem on hand. The other use for decision trees is probabilistic, allowing for the forecasting of future events with a measured degree of certainty. Sapp also believes this type of decision tree allows the analyst to communicate more clearly to the decision maker, in that it provides solid numbers instead of seemingly vague language. The article goes on to address the difficulty of including all alternatives within a decision tree that is seeking to forecast the future. While one can never be certain that all possibilities are addressed within a probabilistic decision tree, Sapp notes that it is a simple process to return to the original model and update it should any unplanned alternatives arise.

Critique:

Can decision trees really handle the large amounts of data that Sapp claims?

How accurate can assigning probabilities be when such probabilities are related to intelligence problems?

Source:

Sapp, E. G. (1974, Winter). Decision Trees. Intelligence Studies, pp. 45-57 (declassified).

6 comments:

  1. In Sapp's discussion of probabilistic decision trees did he talk about how to assign probabilities to events?

    ReplyDelete
  2. Not in great enough detail. In his first example, assigning probability was easily done because there was a 40% chance of rain. For more complicated problems, though, all he says is "if it is possible to assign appropriate probabilities to the various branches, the result is both a decision-making tool and an effective vehicle for the communication of analysis." He doesn't expand on when it is possible, or how it would be done.

    ReplyDelete
  3. I think this is an interesting study but I agree with you that Sapp uses the decision tree method to simplify things. It would be very very difficult to address all alternatives and variables in any situation (ACH may be better suited for such a thing). In your opinion which tree makes for better analysis- the probabilistic or the deterministic? Which would you use and for what situation?

    ReplyDelete
  4. In my opinion, most intelligence analysis is probabilistic. Deterministic decision trees determine whether things are, or are not. Probabilistic decision trees list a variety of outcomes. From what I've seen they are much more business appropriate.

    ReplyDelete
  5. Could part of Sapp's belief that decision tree is a better approach be biased by the amount of information he had access to or available for analysis or making decision while he wrote that article?

    ReplyDelete
  6. Good analysis, but one serious flaw: you are overlaying the skill sets of four decades later to what was a primitive attempt to make a first application of a newly touted technique. Space limitations controlled the number of examples AND the application of probability modeling techniques (which also were in early stages in the first part of the 1970s).

    ReplyDelete