Showing posts with label analysis. Show all posts
Showing posts with label analysis. Show all posts

Tuesday, March 20, 2012

Decision Trees in Intelligence Analysis

Introduction:

Written during the Cold War, Edwin Sapp’s article on decision trees begins by noting the large amount of information available to intelligence analysts, the sophistication and seriousness of the weapons available, and ever-shortening time span in which effective decisions must be made. Sapp advocates the decision tree as a competent method for processing large amounts of data, while at the same time communicating degrees of certainty to an extent which language cannot.

Summary:

Edwin Greenlaw Sapp begins his article entitled Decision Trees by discussing the nature of intelligence analysis. He identifies four major categories that intelligence requirements fall into:

1) places (geographic locations, physical resources);

2) people (strengths and attitudes);

3) organizations (what people form and belong to); and

4) objects (things people make and possess; cities and weapons systems

Information is collected in these categories in order to make forecasts about futures states of affair. In Sapp’s view, Kent’s words of estimative probability are both vague and incomplete. He finds the decision tree more favorable, as it serves to “organize sizeable amounts of data…and communicate the degree of certainty relating to possible outcomes or the likelihood of the occurrence of specific events at some given time in the future.”

Sapp goes on to explain the technique used to create decision trees with two examples. His first is a simplistic question displaying the use of probabilities and subjective value of the outcomes to determine whether or not to hold a dinner party outside given a 40 percent chance of rain. The second example uses the Biblical story of the spies sent into Canaan. Sapp illustrates the intelligence cycle and how a decision tree would have aided this cycle by linking all the collection requirements and ensuring that the important questions were answered first.

According to Sapp there are two types of decision trees: deterministic and probabilistic. Deterministic decision trees deal with situations where no probability needs to be calculated in order to solve the problem on hand. The other use for decision trees is probabilistic, allowing for the forecasting of future events with a measured degree of certainty. Sapp also believes this type of decision tree allows the analyst to communicate more clearly to the decision maker, in that it provides solid numbers instead of seemingly vague language. The article goes on to address the difficulty of including all alternatives within a decision tree that is seeking to forecast the future. While one can never be certain that all possibilities are addressed within a probabilistic decision tree, Sapp notes that it is a simple process to return to the original model and update it should any unplanned alternatives arise.

Critique:

Can decision trees really handle the large amounts of data that Sapp claims?

How accurate can assigning probabilities be when such probabilities are related to intelligence problems?

Source:

Sapp, E. G. (1974, Winter). Decision Trees. Intelligence Studies, pp. 45-57 (declassified).

Tuesday, May 5, 2009

Using Search Engine Optimization For Intelligence Analysis

SEO Analysis Now - A Site For Using SEO in Intelligence Analysis
Rated: 4 Stars out of 5

As a final requirement of this course, I had to research and report findings on an analytical technique of my choosing. In addition, I had to test-out and apply the technique in order to reveal its true strengths and shortcomings; as well as come up with a how-to guide to use the technique. For my project, I chose to conduct a Search Engine Optimization (SEO) Analysis on the Websites of two popular coffee shops, Starbucks (which is well established) and Caribou Coffee (which is slowly gaining popularity) and reflect on how this process can be applied to Competitive Intelligence (CI), to Law Enforcement Intelligence (LEI), or to the National Security sector. I chose SEO not only because it is an emerging analytical process and can be useful to intelligence analysts, but also because I found it quite intriguing and fun.

SEO provides a way to gain insight into a Website’s audience.

If we know who an audience is (age, gender, ethnicity, education, affluence, etc), how they behave (what other Websites, or types of Websites, they visit), from where they log on, when they visit, and what other sites direct their traffic (as well as what their general interests are), then we can make assessments and predictions about how to either promote their behavior (steering them toward a particular site – useful for CI & marketing purposes) or how to counter that behavior (keeping them away from sites – useful for LEI & N’tl Sec purposes).

For more detailed information about how SEO can be used for Intel analysis, please check out my project:

SEO Analysis Now - A Site For Using SEO in Intelligence Analysis


(The image below is a geographical search index comparison of online users searching for Caribou Coffee [left] and Starbucks [right]. Images provided by Google Insights for Search)

Wednesday, April 22, 2009

Summary Of Findings: Game Theory (4 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 22 APR 2009 regarding Game Theory specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Game theory is a method based on applied mathematics and economic theory. It can be useful when attempting to analyze (and ultimately predict) the strategic interactions between two or more actors and the way in which their actions influence future decisions. Game theory assumes that all actors are rational, and can be influenced by various individuals and factors. Games typically involve five common elements: players, strategies, rules, outcomes, and payoffs.

Strengths:

-assumes rational actors
-assumes actors will adjust their actions based on the actions of other actors
-not clearly differentiated from role-playing, simulations, and/or decision trees
-very mathematically based (can be intimidating)
-difficult to quantify options, strategies, and motivations
-may not be a valid method to produce an accurate estimation (see Game Theory, Simulated Interaction, and Unaided Judgment For Forecasting Decisions in Conflict: Further Evidence)
--In real world applications, identifying all of the key players and outcomes can be difficult

Weaknesses:

-Visual step-by-step trail to a conclusion/estimate
-Ability to quantify variables in play
-Emphasis on mathematics and scientifc method
-Applicable to multiple fields (economics, conflict, etc)
-90% rate of success according to BDM

How-To:

Game Theory varies in complexity and in application, however, each application has the following in common:

*Establish the players and the complexity of the game being played, so as to understand the rules which govern the players and the game.
*Identify the possible outcomes for the choices the players can make (although this is particularly difficult as not all decisions can be predicted)
*Establish measurable values for predicted outcomes.
*Eliminate dominated strategies and employ dominate strategies. Repeat this step until a clear, singular strategy emerges or equilibrium is reached between the players.
*Employ selected strategy.

Experience:

As a class, we visited www.gametheory.net and played the repeatable version of Prisoner's Delemma under the "Interactive Materials" tab. Each student played the game at their personal computers. Our objective as we played against the five "personalities" was to identify the particular strategies employed by the computer (in addition to scoring the most utility points). Some of the strategies employed by the computers personalities included "tit-for-tat" and "tit-for-two tats."

Sunday, April 12, 2009

The "Red Team": Forging A Well-Conceived Contigency Plan

By Col. Timothy G. Malone and Maj. Reagan E. Schaupp
Aerospace Power Journal
Summer, 2002


Author's Note: Due to the article's length, only the most relevant information for our class was summarized.


Summary
Throughout the article, the authors provide two separate definitions for red teaming. Although the definitions are similar, they are also different (which is somewhat puzzling). I synthesized the two definitions and came up with this: a red team is a group of subject-matter experts with appropriate backgrounds that provide an independent review of processes and products using devil’s advocacy and knowledgeable role-playing of the enemy. The red team assesses planning decisions, assumptions, and courses of action from the perspective of friendly and enemy organizations.

The authors acknowledge that the concept of red teaming is far from new. Government, military, and civilian circles have all used red teaming in a variety of contexts. In government circles, it normally is associated with assessing the vulnerabilities of systems or structures, especially within the warfare arena. In the business world, red teaming usually refers to a peer review of a concept or proposal.

If conducted effectively, red teaming can produce more complete analysis at all phases and deliver a better plan of operations for the decision-maker. Effective red teams can pinpoint planning shortfalls, deviations from doctrine, reveal overlooked opportunities, and extrapolate unanticipated strategic implications. Additionally, red teaming can also determine whether the required task is understood or whether further guidance is needed.

Because a red team will conduct a comprehensive review planning products and processes, the selection of team members is critical. Red team members must have credibility, which comes only with expertise and experience. If some red team members blatantly fall short of this prerequisite, their counterparts will be skeptical of any insights they claim to have about the operation. The timing of red teaming events can play a crucial role in planning success. Ideally, the commander should form a red team as early in the planning effort as possible.
Reblog this post [with Zemanta]

Thursday, April 9, 2009

The Role And Status Of DoD Red Teaming Activites

Defense Science Board Task Force
September 2003
Sections I-III,VI


Summary
Red teams can be a powerful tool to understand risks and increase options. Their purpose is to reduce an enterprise’s risks and increase its opportunities. Red teaming can be used at all three levels of an enterprise: strategic, operational, and tactical. The Defense Science Board, however, found that the use of red teams within the Department of Defense is mixed at best.

Despite this mixed record, the Defense Science Board concluded that the use of red teams is especially important given today’s climate. Adversaries are tough targets for intelligence (compared to the Cold War). Red teaming can both complement and inform intelligence collection and analysis. Aggressive red teams challenge emerging operational concepts in order to discover weaknesses before real adversaries do. Red team also tempers the complacency that often follows success (referenced to the time period following Desert Storm).

Red teams come in many varieties and there are different views about what constitutes a red team. The Defense Science Board defined the term broadly, including not only playing the adversary, but also playing devil’s advocate and related roles. While differing in some respects, these activities all have in common the challenging of an organization’s norms. A red team is comprised of individuals selected for their special subject matter and expertise, perspective, imagination, or critical analysis. The red team itself is only one element in a red teaming process. Elements of the process include who the red team reports to, how the red team interacts with the management of the enterprise, and how the enterprise considers the use of the red team’s products.

Although red teaming is important, it is not easy and rarely done well. Typical causes for red team failure include the read team not taking their tasking seriously, the red team loses its independence, and the red team becomes removed from the decision making process. Conversely, attributes of an effective red team include an environment that values internal criticism, “top cover” or the support of upper level management to raise issues that may be unpopular, and proper staffing on the red team.
Reblog this post [with Zemanta]