Summary of Findings: Decision Trees (3 out of 5 stars)

Definition:
A visual representation exploring all possible courses of action and the resulting consequences to aid in the decision making process. Decision trees are comprised of nodes (decisions/consequences), branches (links between nodes), and probabilities. The resulting form resembles a tree.

How to:

1. Begin from the top or the left-hand side as outcomes may flow from either left-to-right, or from top-to-bottom.
2. Define the problem/original decision visually represent this by using a rectangle (or box) around the decision to be made. This original decision is referred to as the "decision node."
3. Identify all possible courses of action that stem from that decision. The courses of action must be mutually exclusive and exhaustive. Each course of action should have a "branch" stemming out from the decision node.
4. Identify "chance nodes" (represented with circles) that represent the possible outcomes of the courses of action. Different outcomes should stem out from this chance node.
5. Sometimes branches emanating from decision and chance nodes can lead to other decision nodes - repeat steps 2 & 3 if this occurs, this will effectively “overgrow” the tree.
6. Indicate the associated probability (likelihood) that a particular outcome stemming from a chance node will occur. Probabilities are quantified with a value ranging from zero to 1. Therefore a probability of 0.6 would be the equivalent of a 60% chance. Use your experience and knowledge, as well as any conclusions from literature or other supporting data to assign a probability value.
7. The sum of the probabilities of all outcome branches stemming from a single chance node must equal 1.
8. When final consequences are identified, use a filled-in circle to represent that consequence.
9. Start “pruning” the decision tree by eliminating leaves and branches that are not (or are less) probabilistic.
   

Strengths:

• Structured, allowing for transparent recognition and interpretation of the constructed model
• Visual representation – good tool for presenters and audiences
• Provides an audit trail for the decision maker
• Applicable to multiple disciplines
• Produces quantifiable estimates
• More organized than a mind map
  

Weaknesses:

• High volume of quantitative data requires high level of mathematics capability
• Risky when choosing the "correct" variable to subdivide data
• Susceptible to "blind spots"
• Assigning probabilities may entail guesswork
• Susceptible to bias
  

Experience:
In order to get a feel for decision tree analysis, the group, while sitting in the bar across from the intel building, decided to conduct a decision tree analysis on whether or not we should do a decision tree experience exercise. This was the general consensus of decision tree analysis based on that exercise.

• It was difficult to assign probabilities without prior data that specifies what the likelihood is
  
• It is subjective and open to analysts' bias
• Can get messy if using pen and paper
• Open to wild cards and cognitive blindspots
• Does make the analyst think through the decisions and think outside of the box
• The visual representation makes the choices and consequences clear
• It does produce an estimation
  

Countering Terrorism: Integration of Practice and Theory

Countering Terrorism: Integration of Practice and Theory
An Invitational Conference
FBI Academy, Quantico, VA
Feb 28, 2002
Sponsored by the FBI Behavioral Science Unit

Summary:

The FBI's Behavioral Science Unit identifies decision tree techniques and data mining as "highly efficient methods for processing large volumes of data." However, they add that they need to be "tailored to the unique cultures in which they would be used," and that their use depends on a cooperative effort by those that design such methods with those who have to analyze them. The decision tree decision-making methodology is recognized by the FBI as one method to standardize responses to threats and to understand the seriousness of those threats.

After collaboration efforts to receive and organize incoming information have been established with a technical advisor and a decision tree has been created, using the decision tree can be broadly applied due to it's low technical skill demands. The FBI also adds that decision trees should serve only to report information to a decision maker, reinforcing the idea that decision trees should be suggestive - not to actually state the decision to be made.

Appendix 6 gives the most useful information regarding decision tree analysis. In this appendix, it defines decision trees as a tool to aid decision making. "The idea is to concretely identify the choice points and map the sequence of decisions from beginning to end."

Steps:

1. Started with a decision that must be made - the FBI uses the example of whether or not to arrest a suspect. Represent this decision with a square. This should be drawn on the left-most side of the paper/screen.
   
2. Using lines drawn outward and to the right, identify each possible solution. Write each solution on each line.
3. At the end of each line, the results are considered. Use a circle is drawn at the end of the line to identify if choices are available
   
4. If another decision is possible, draw a square with that decision listed.
   
5. If there is a final consequence, a solid dot is drawn with a filled-in circle at its end.

Evaluating the decisions:

The FBI identifies the procedure for choosing a decision as backward induction analysis. In order to do this first assign a number that represents the worth or utility of the final consequence (filled-in circle). Use a 0.0 to 1.0 scale to identify worth. Next, assign a sum to each event node (the circles) that represents the expected utility of the node - this is the weighted average utility for that event node. "Finally, each decision node is a assigned a number that is the maximum value of the nodes that branch out from it."

According to the Behavioral Science Unit the benefits of using decision tree analyses are: 1) that the possible choices are explicitly made; 2) the choices are evaluated by the importance of the outcome as well as quantified with the probability for that outcome; and, 3) displays communication flow.
The FBI, once again, states that "decision trees can be used to guide decisions, not make them. The final decision is left up to the operator."

Decision Tree Analysis: Drawing Some of the Uncertainty Out of Decision Making

Decision Tree Analysis: Drawing Some of the Uncertainty Out of Decision Making
by William E. Marsh, PhD

Summary:

This article was written primarily to describe the benefits and how-to's of using decision trees (which Marsh commonly refers to as 'decision analysis') to make decisions in the livestock business - particularly swine veterinary practice. Although Marsh's target audience is obviously for those who either own swines or practice medicine on them, his article draws out the basics information needed to conduct a decision tree analysis; and he does so in a easy-to-understand and practical manner. For the purposes of this blog post, and out of respect of my targeted audience, I will leave out all swine references, examples, and jokes.

Marsh essentially defines a decision tree as a visual representation that logically depicts a time-sequenced flow of events with the purpose of informing a decision maker with the probability of various outcomes. It is a structured approach to making decisions when uncertainty exists that helps us to quantify and "consider the effects of chance on the outcome of a given decision." Marsh bluntly states that, "In using decision analysis, it is important to understand that the objective is not to make a prediction...[but rather, it] uses probabilities...to provide a guide for what should be done."

Steps to conducting a decision tree analysis:

1. Define the problem - what is it that we are trying to make a decision about. This will be represented visually using a rectangle (or box) around the decision to be made. Marsh refers to this as the "decision node."
   
2. Identify a "mutually exclusive, exhaustive list of all possible courses of action to address the problem." Each course of action should have a "branch" stemming out from the decision node.
   
3. Create a "chance node" (represented with circles) that represent the possible outcomes of a course of action. Different outcomes should stem out from this chance node.
4. Sometimes branches emanating from decision and chance nodes can lead to other decision nodes - repeat steps 2 & 3 if this occurs.
5. Indicate the associated probability (likelihood) that a particular outcome stemming from a chance node will occur. Probabilities are quantified with a value ranging from zero to 1. Therefore a probability of 0.6 would be the equivalent of a 60% chance. Use your experience and knowledge, as well as any conclusions from literature or other supporting data to assign a probability value.
   
6. The sum of the probabilities of all outcome branches stemming from a single chance node must equal 1.

Marsh does not clearly identify any cons to conducting a decision tree analysis, however, it is quite obvious that he is a strong proponent of using this technique to strengthen the decisions he makes.

Decision Mapping Easier Than Argument Mapping?

timvangelder.com

Summary:
The explores the differences between decision mapping and argument mapping, two very similar techniques with a fundamental difference. Whereas argument mapping is applied to an argument, decision mapping is used to determine "choices between multiple possible actions." As an example of each, the author draws both a decision map and an argument map based on a recent New York Times article by Col. Muammar Qaddafi proposing a peace plan for the Middle East.

After applying both techniques to Col. Qaddafi's proposition (which was the creation of "Isratine" - a joint Isreali/Palestinian state), the author found that the decision map was easier to develop than the argument map. Part of this was due to the nature of the piece being analyzed : an article in someone else's words. "The translation from prose to decision map was much more straightforward than the translation from prose to argument map. In the latter case, there seemed to be far more discretion about how to do it, and hence a much higher level of effort and expertise was required to determine which of the approaches would be 'right' or best. "

The author expresses his surprise that the decision map was easier to develop, although he concedes that the subject of the exercise may have had alot to do with that. If decision mapping is, in fact, generally easier than argument mapping, he states that the following would be true as well:

• Decision mapping should find faster and wider uptake than argument mapping
• From a pedagogical or training perspective, decision mapping should be introduced first, with argument mapping treated as a more advanced subject.

In a comment to the blog, a reader notes that he, too, finds decision mapping to be easier to use than argument mapping. He points out "For some reason argument mapping seems to be so much more dependent on language and context and is perhaps more prone to making mistakes." He explains that it is possible to mis-represent the author of an argument when constructing the argument map due to the tendency to want to "close the gaps automatically even if the original text doesn’t necessarily support it. " He further states that the first step of the decision mapping process is the most crucial, i.e. "sizing up the situation appropriately." The author responds, stating that that first step is critical, although he has no advice as to how to make sure you go about it correctly.