Sunday, March 10, 2013

Use of Decision Trees for Intelligence Analysis


Summary:


Edwin Greenlaw Sapp’s four key categories of intelligence requirements include places, people, organizations, and objects.  Analysts collecting intelligence on these categories assist policy makers to formulate strategic and tactical decisions.  While collecting relevant intelligence to answer a requirement leads to better forecasting, the end results are always prone to chances of uncertainty.

According to Edwin Sapp, a modern day “information explosion” has caused delays and errors leading to ineffective decision making by the American Intelligence Community.  The amount of time the analysts and decision makers have to formulate an accurate decision is short due to the increased volume of information to be analyzed.  The Greek concept of ‘modeling’ may serves as an effective solution to managing these large volumes of data.  Analysts and decision makers may adopt this concept in order to manage intelligence which leads to more accurate and scientific decisions.   

An offshoot of the ‘modeling’ concept, a decision tree is a prototype of logic diagrams which graphically exhibit relationships and logical outcomes for a series of assessments.  Hence, decision trees serves as a method of organizing large volumes of data.  This method not only considers alternate outcomes to a certain scenario, it also indicates the degree of uncertainty with regards to adopting a certain outcome.   

Critique:


This article argues as to why decision trees can be an effective management tool in organizing intelligence.  However, the examples presented in the article failed to prove the effectiveness of decision trees for the use in the intelligence community. Given the simplicity of the first two examples, they did not successfully demonstrate the end product of managing a large volume of information.  

Sapp also failed to discuss the disadvantages of utilizing decision trees.  He failed to consider the outcomes of incomplete decision trees where certain nodes of the tree may lack information.  While complete decision trees can enhance the intelligence community’s accurate forecasting capabilities, incomplete decision trees can lead to inaccurate assessments and increase uncertainty.  In addition, drawing a decision tree for a large volume of data can be a lengthy and a complex process leading to difficulties in the interpretation.  Furthermore, decision trees are susceptible to even the smallest change in data.  Given the complexity of decision trees created from large volumes of intelligence, a small change in one node may compromise the entire tree.  In this case, decision trees are a better fit for managing a small volume of information.       

          
https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol18no4/html/v18i4a03p_0001.htm

Source:
CIA Historical Review Program. https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol18no4/html/v18i4a03p_0001.htm

2 comments:

  1. The author's point that decision trees express levels of uncertainty is important for the intelligence field, as we strive to improve accuracy and be explicit about what we don't know.

    I like your analysis of this article in that you are critical of the actual utility of decision trees in the intelligence field. I agree with quite a few of your concerns, particularly how drastically a decision tree will change with a single scenario change. However, I do not think that this is related to the method's ability to handle small/large amounts of data, rather its ability to accurately process static/changing data.

    In the article I pursued, decision trees were made to explore 19 variables in over 1000 car accidents, requiring substantial data. The reason it was successful was that the data remained the same throughout the process because the accidents already happened. Perhaps decision trees are only helpful in the intelligence field for actions that have already occurred, as opposed to predicting an outcome, in which case, their utility is certainly limited to specific cases or fields.

    ReplyDelete
  2. The comment by the author regarding the "information explosion" reminds me of our own issues throughout the year in deciding when to stop researching and when to start analyzing, especially during the country study. I am skeptical about the extent to which decision trees can actually be "complete". As you stated, missing information can lead to inaccurate assessments and I believe the intelligence community often deals with incomplete information.

    In the article I read, the author explicitly stated that the limitation to his study was that it only worked on smaller companies. It seems as though this may be an implicit limitation to decision trees in general, in that they don't handle large quantities of information well because the larger the tree gets, the more accuracy decreases. For the intelligence field, decision trees may be more useful as a brainstorming technique for small questions rather than as a management tool for organizing large amounts of information.

    ReplyDelete