Showing posts with label intelligence. Show all posts
Showing posts with label intelligence. Show all posts

Tuesday, April 16, 2013

Fusion of Intelligence Information: A Bayesian Approach

Elizabeth Paté-Cornell presents a classical probabilistic Bayesian model that she believes can be utilized by the intelligence community to aid in the fusion of intelligence information. The awareness of the need for such fusion is apparent in the wake of September 11, 2001, which the author suggests the probability of impending attacks can be found through a Bayesian analysis. The author's two major arguments for the use of the Bayesian model in the IC, particularly related to terrorist attacks, is that it allows for the computation of the posterior probability of an event given the probability of the event prior to observing signals, and the quality of the signals based on the probabilities of false positives and false negatives.

Summary:
The author begins by discussing the problems associated with a fusion of information within the US intelligence community, namely difficulties in ensuring internal communications and the difficulties in merging the content of multiple signals, some more sharp than others, some dependent or independent of others. This research claims that Bayesian analysis can be applied to help solve the difficulties associated with the latter and is explained in terms of identifying the probability of an impending terrorist attack. It should be noted the author does not claim the model will better detect impending terrorist attacks, rather that it can increase the probability that an attack plan is foiled through guiding "clear thinking at a time when the amount of information is large and confusing and intuitions can be seriously misleading" (Paté-Cornell, 2002, p. 454).

The elements of Paté-Cornell's Bayesian model can be explained through the following notations:
 
Namely, the event of interest throughout the article is an impending terrorist attack. Through the model, the author presents formula that addresses both the prior probability of the event occurring before reading signals, such as intercepted telephone conversations, as well as the quality of the signals. The formula, as appears in the following figure, considers what alternatives to the event of interest could occur in conjunction with the signal, a very important thing to consider in the intelligence field.


Additionally, the formulas the author presents address the chances the signals observed are false positives, or that some signals has been missed (false negatives), and how these affect the probability of a future terrorist attack. The probability of false positives can be calculated by considering the prior probability of the impending attack without considering the signals, in conjunction with the rate at which the signal occurs during normal sensor operation when the event does not occur. She explains that her definition of false positives and its application in Bayesian analysis is most useful to the intelligence community because of its consideration of the prior probability of the event, especially considering how drastically the prior probability has increased post-September 11.

Estimating the prior probability of an impending attack can be considered as a combination of the intention of the enemy to attack, the effective planning of that attack (ie. the ability of the perpetrators to coordinate a plan and avoid detection), and the successful implementation of the plan on a given day (ie. the ability of the perpetrators to carry out the plan and avoid target's safeguards). The author argues that the identification of these probabilities alone is of use to the intelligence community given the chance to reduce the probability of an attack attempt through various measures hitting these areas (ie. cutting flow of funds or increasing security).

Critique:
The research applies a Bayesian model by using hypothetical numerical illustrations for the interpretation and fusion of intelligence information and could be strengthened through the use of real-life numerical examples, though sensitive in nature. Additionally, the author switches between examples for multiple formulas, sometimes relating it back to the overarching theme of terrorist attacks, while other times relying on the unrelated example of testing chemicals for poison. This back-and-forth detracts from the overall readability of the research and does not add to the application of the model to the intelligence community. The author uses good examples of potential signals used intelligence but does not carry them throughout the research.

The author further admits some limitations of the research. First, the assumption that both the event and the signals are black and white, either they occur or they do not occur, which is not always the case, particularly in the intelligence community. Further, the research assumes that the likelihood of false signals, whether positive or negative, remains the same throughout time, also unlikely in the intelligence field. Finally, many of the sources of data for such a model are difficult to accurately quantify, including the frequency of past observations, reliability data for sensors or links, or expert opinions. For instance, how can we accurately, and quantitatively, determine the reliability of human intelligence?

Overall, the research is very interesting and provides insight into the intelligence community and process. Admittedly, the approach only helps solve the second half of the information fusion, not aiding in the means of internal communication among the intelligence community, however, any reduction in uncertainty, particularly through objective means, helps the success rate of thwarting plans of terrorists attacks, or other such problems addressed by the intelligence community.

Source: 
Paté-Cornell, E. (2002). Fusion of Intelligence Information: A Bayesian Approach. Risk Analysis: An International Journal, 22(3), 445-454.

Tuesday, May 5, 2009

Using Search Engine Optimization For Intelligence Analysis

SEO Analysis Now - A Site For Using SEO in Intelligence Analysis
Rated: 4 Stars out of 5

As a final requirement of this course, I had to research and report findings on an analytical technique of my choosing. In addition, I had to test-out and apply the technique in order to reveal its true strengths and shortcomings; as well as come up with a how-to guide to use the technique. For my project, I chose to conduct a Search Engine Optimization (SEO) Analysis on the Websites of two popular coffee shops, Starbucks (which is well established) and Caribou Coffee (which is slowly gaining popularity) and reflect on how this process can be applied to Competitive Intelligence (CI), to Law Enforcement Intelligence (LEI), or to the National Security sector. I chose SEO not only because it is an emerging analytical process and can be useful to intelligence analysts, but also because I found it quite intriguing and fun.

SEO provides a way to gain insight into a Website’s audience.

If we know who an audience is (age, gender, ethnicity, education, affluence, etc), how they behave (what other Websites, or types of Websites, they visit), from where they log on, when they visit, and what other sites direct their traffic (as well as what their general interests are), then we can make assessments and predictions about how to either promote their behavior (steering them toward a particular site – useful for CI & marketing purposes) or how to counter that behavior (keeping them away from sites – useful for LEI & N’tl Sec purposes).

For more detailed information about how SEO can be used for Intel analysis, please check out my project:

SEO Analysis Now - A Site For Using SEO in Intelligence Analysis


(The image below is a geographical search index comparison of online users searching for Caribou Coffee [left] and Starbucks [right]. Images provided by Google Insights for Search)

Friday, May 1, 2009

Bayesian Statistics In The Real World Of Intelligence Analysis: Lessons Learned

Bayesian Statistics In The Real World Of Intelligence Analysis: Lessons Learned
By Kristan Wheaton, Jennifer Lee, & Hemangini Deshmukh
Journal of Strategic Studies, vol. 2 n.1
February 2009

Summary:
In this article, Kris Wheaton, in collaboration with Jennifer Lee and Hemangini Deshmukh, agree that alternative methods, ones that are more structured, should be applied to the intelligence process in order to improve intelligence analysis. Recognizing the potential that Bayesian Statistics can bring to the field of intelligence, the author questions the ease in which entry-level intelligence analysts can apply and use this advanced statistical method.

Following the model of an experiment conducted by Gerd Gigenrenzer to test the accuracy of a diagnosis by two groups of doctors (with one group using traditional statistic formulations [5% or .05] and the other using natural frequencies [5 times out of 100]), the author tested 67 Senior Intelligence Studies students at Mercyhurst College. The findings of Wheaton’s experiment were extremely similar to that of Gigenrenzer’s, showing that groups who receive natural frequencies have a much higher rate of being accurate when using Bayesian statistics (79% versus 18% accuracy). This accuracy is attributed to the power of ‘framing’ questions. The author concludes, “natural frequencies are an effective method for encouraging Bayesian reasoning.”

In addition to the experiment, the article covers a brief how-to and overview of Bayesian Analysis. In short, Bayesian statistics is particularly useful due to its ability to take-in-account probabilities of one event affecting another, allowing the analyst to rationally update a prior assessment in light of new evidence. This process also helps in the reduction of two very common cognitive biases, the vividness and recency biases. See article for relevant examples of how the Bayes Theorem can be applied to intelligence-related issues.

The Bayes Theorem is illustrated below:


Bayesian Analysis For Intelligence: Some Focus on the Middle East

Bayesian Analysis For Intelligence: Some Focus on the Middle East
By Nicholas Schweitzer
Approved For Release 1994
CIA Historical Review Program
02 July 96

Summary:
Nicholas Schweitzer suggests that advanced analytical methods, such as Bayesian analysis, should be used to aid analysts in an age where information flows continue to rise. In an effort to test Bayesian Analysis as a tool for intelligence analysts, he used the technique among a group of intelligence analysts to assess complex political-military problems. The Middle East was chosen as a discussion point because of the level regional complexities.

Schweitzer defines Bayesian Analysis as “a tool of statistical inference used to deduce the probabilities of various hypothetical causes from the observation of a real event. It also provides a convenient method for recalculating those probabilities in the light of a continuing flow of new events…the ‘rule of Bayes’ states that the probability of an underlying cause (hypothesis) equals its previous probability multiplied by the probability that the observed event was caused by that hypothesis.”

How to:

Because of limitations, the Bayesian technique can only be applied where certain criteria are met. First, the question to be answered must lend itself to formulation in mutually exclusive categories (i.e. war vs. no war); the insertion of overlapping possibilities reduces accuracy of the Bayesian technique. Second, the question must be expressed as a specific set of hypothetical outcomes. Third, there should be a fairly rich flow of data that is at least peripherally related to the question. Lastly, the question must revolve around the type of activity that produces preliminary signs and is not largely a chance or random event. If this criteria is met then:

1. Assign numeric probabilities to hypotheses. The sum of the values must equal .1 or 100%. Because the examination of political/military affairs and events do not automatically yield quantified results, the possible outcomes (hypotheses) have to be quantified. Schweitzer asserts that implementing a Delphi method is the best solution to quantify possible outcomes. He suggests the following procedure to do this:
  • Use analysts who are experts on the subject matter (preferably ones who are working on the situation with you)
  • Establish a periodic routine for reporting
  • On the first day of the period, each of a number of participating analysts submits the items of evidence they have seen since the last round.
  • Submissions should be in the form of 1-2 sentences summarizing the item, along with the date, source, & classification.
  • The inclusion of relevant items and exclusion of irrelevant items is up to the discretion of the analyst.
  • A coordinator consolidates the items, resolving differences of wording, emphasis, and meaning, and returns the complete list of items to the participants.
  • On the following day, the analysts (working individually) evaluate the items and return the numerical assessments
  • *the use of a group of analysts, as opposed to a single expert, is highly recommended*
2. Assess and quantify the evidence that supports/negates the hypotheses.
3. Calculate the new probabilities according to the rule of Bayes:

E is an event, an “item” of intelligence
H is a hypothesis, a hypothetical cause of events
Hi is one of a set of n mutually exclusive hypotheses
P(Hi) is the starting, or “prior” probability of a hypothesis
P(E/Hi) is the probability of an event given Hi, of an event occurring, given a particular underlying cause
P(Hi/E) is the probability of a hypothesis given E, the “revised” probability of a hypothesis, given that a particular event has occurred.

Strengths (please see article for further explanation):
  • Allows for the weighting of evidence
  • Provides transparency in intelligence assessments
  • Forces the consideration of alternative possibilities
  • Quantifies analysis instead of using words of estimative probability
  • Displays the trend toward an outcome quicker than the analyst can typically realize it on their own
  • Incorporating the Delphi method adds credibility to the assessment when presented to managers and decision makers.
Weaknesses (please see article for further explanation):
  • Limited applicability
  • Data problems – can exist in deciding which information is relevant and should be included, as well as what weight values should be given to evidence.
  • Source reliability – what is the best practice to account for this
  • “Negative evidence”- the absence of any positive evidence may in itself be highly indicative of something
  • Problems over time – problems in using this method in a project continuing over many months
  • Problem with numbers – cannot use the probability of ‘zero’ (doesn’t work mathematically or analytically) therefore extremely low probabilities must be indicated by a very small number. Also, some people have difficulty thinking in, and assigning, probabilities.
  • Subject to bias and manipulation – this is one of the reasons for which the author suggests using a group of experts/analysts to assign probabilities.

Wednesday, April 29, 2009

Summary Of Findings: Gap Analysis (3 Out Of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 29 APR 2009 regarding Gap Analysis specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:
Traditionally, "gap analysis" is a method used to conduct an internal operational analysis, whereas the gap analysis identifies the "gap" between a current state and a desired endstate within a company or agency. From an intelligence analysis perspective, "gap analysis" can be used as a tool to identify the likely pathway or pathways a target may take to arrive at a given endstate from a known position. Thus, "gap analysis" does not necessarily provide an estimate, but rather provides the analyst with a list of possible actions a target may likely take. Gap analysis as an analytic technique bears a striking resemblance to several other methods, such as Indicators & Warnings and Decision Trees.

Strengths:
1) Identify the target
2) Characterize the current status of the target as well as the target's goals
3) Identify what you want to know about the target
4) List the pieces of information that you have
5) Use a systematic approach to infer what the target is likely to do in order for the target to reach its goals

Weaknesses:
* No structured method to conduct the analysis
* May not leave you with a clear estimate
* Open to bias and other cognitive downfalls (satisficing, mirror imaging, etc.)
* Overlaps with the process of other methods and modifiers (i.e. decision treees, I&W, Brainstorming, SWOT, etc.)
* Susceptible to deception
* Danger of pitfalls

How-To:
1) Identify the target
2) Characterize the current status of the target as well as the target's goals
3) Identify what you want to know about the target
4) List the pieces of information that you have
5) Use a systematic approach to infer what the target is likely to do in order for the target to reach its goals

Experience:
For the first application, the group tried to determine what thesis topic Mary, a fictional first year graduate, would write about. Professor Wheaton acted in place of Mary, and we role played in questions and answer format. The group determined the gaps needing filled would be what intelligence track she was most interested in (national security, law enforcement, or competitive), what area or topic in her previous classes interested her the most, the choice of her primary reader, and the reader's academic interests. Upon filling these gaps, the group ascertained a plausible topic for Mary's thesis.

For the second application, the group discussed Russia's long held ambitions for a warm water port, preferably on the Mediterranean. The first part of the discussion centered on hostilities between Georgia and Russia, and actions Russia could take against Georgia to maintain their sphere of influence in the Black Sea. Additionally, the group discussed what actions may be necessary in their diplomatic relations with other nations bordering the sea. After discussing how Russia could potentially gain a Black Sea port by bringing Georgia into their sphere of influence, the group discussed how Russia could proceed toward gaining access to Mediterranean ports. The group determined that Turkey would be central in future Russian objectives for ports in the Mediterranean. The group put together lists of things the Russians could do that they are not doing now that would indicate their goals of extending their influence in Georgia and Turkey.

These applications illustrated the following:
* Helped in thinking through the steps that lead to a decision
* Allowed for open discussion and debate, helping the critical thinking process
* At some points it felt like stabbing in the dark, but the estimates later became more clear as the group discussed the options
* Eliminated peripheral influences not directly related to the topic, such as administrative processes.

Thursday, April 23, 2009

Gap Analysis: As Is or To Be, What is the Question?

Gap Analysis: As Is or To Be, What is the Question?
By Dorothy Ball,
Four Thought Group

Summary:
This article describes how to use gap analysis in Business Process Improvement (BPI) strategies, relating the method to Health Care organizations. In this summary, I have attempted to relate the gap analysis process, as the author uses it, to the intelligence field - identifying and overcoming intelligence gaps.

Dorothy Ball, a senior policy and business consultant at Four Thought Group, Inc., states that gap analysis is an effective solution for businesses and organizations that have a service orientation (which includes for-profits, non-profits, and government agencies). To paraphrase, gap analysis is a method for an organization to improve performance or make gains by identifying the potential needs for that organization to move from where they are (or what intelligence you have) to where they want to be (or what intelligence is still needed). Ball describes gap analysis as a method to develop a roadmap that gets you from “where you are now (As Is) to where you want to be (To Be).”

How to:
  1. Identify what your organization looks like now (or what intelligence/information you currently have available) and what you want your organization to look like (or what intelligence/information you desire). Needs for improvement (more intelligence) is often indicated by changes such as policy, resources, environment (or events that spark inquiry). Changes may be event-driven, or ongoing.
  2. Understand the Business Process (or collection process). Ball describes the business process as “a collection of related, structured activities, or chain of business functions, activities and tasks, that produce each specific service or product… Each business process consists of inputs, method, and outputs. The inputs are required before the method can be put into practice to achieve the outcome. When the method is applied to the inputs then certain outputs will be created.”
  3. Use comparative analysis techniques to identify what is needed to get you from where you are (the Intel you have) to where you want to be (the Intel desired). Examination of the current process, and the modeling of the new process, may be necessary in order to discover the interrelationships/interconnectedness of how the current process can get you the desired results.
  4. Create a plan for implementation.

Wednesday, April 22, 2009

Summary Of Findings: Game Theory (4 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 22 APR 2009 regarding Game Theory specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Game theory is a method based on applied mathematics and economic theory. It can be useful when attempting to analyze (and ultimately predict) the strategic interactions between two or more actors and the way in which their actions influence future decisions. Game theory assumes that all actors are rational, and can be influenced by various individuals and factors. Games typically involve five common elements: players, strategies, rules, outcomes, and payoffs.

Strengths:

-assumes rational actors
-assumes actors will adjust their actions based on the actions of other actors
-not clearly differentiated from role-playing, simulations, and/or decision trees
-very mathematically based (can be intimidating)
-difficult to quantify options, strategies, and motivations
-may not be a valid method to produce an accurate estimation (see Game Theory, Simulated Interaction, and Unaided Judgment For Forecasting Decisions in Conflict: Further Evidence)
--In real world applications, identifying all of the key players and outcomes can be difficult

Weaknesses:

-Visual step-by-step trail to a conclusion/estimate
-Ability to quantify variables in play
-Emphasis on mathematics and scientifc method
-Applicable to multiple fields (economics, conflict, etc)
-90% rate of success according to BDM

How-To:

Game Theory varies in complexity and in application, however, each application has the following in common:

*Establish the players and the complexity of the game being played, so as to understand the rules which govern the players and the game.
*Identify the possible outcomes for the choices the players can make (although this is particularly difficult as not all decisions can be predicted)
*Establish measurable values for predicted outcomes.
*Eliminate dominated strategies and employ dominate strategies. Repeat this step until a clear, singular strategy emerges or equilibrium is reached between the players.
*Employ selected strategy.

Experience:

As a class, we visited www.gametheory.net and played the repeatable version of Prisoner's Delemma under the "Interactive Materials" tab. Each student played the game at their personal computers. Our objective as we played against the five "personalities" was to identify the particular strategies employed by the computer (in addition to scoring the most utility points). Some of the strategies employed by the computers personalities included "tit-for-tat" and "tit-for-two tats."

Wednesday, April 15, 2009

Summary Of Findings: Red Teaming (2 Stars Out Of 5)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 15 APR 2009 regarding Red Teaming specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:
Red Teaming is an analytical modifier that can be used in two distinct ways:

First, in an objective sense, it is used to challenge emerging operational concepts in order to discover weaknesses with an organization's procedures and reactions.
Second, in a subjective sense, red teaming is used to generate options for adversaries that may be overlooked due to biases or heuristics.
When red teaming is used in the first manner, the effectiveness of red teaming is usually easier to monitor and evaluate. When using the second approach, it is more difficult to measure the effectiveness of red teaming, as the effectiveness is subject to forces outside of the method itself.

Strengths:
*Forces "thinking outside the box"
*Challenges "groupthink"; may reduce bias
*Can provide invaluable near-real world training (as close to real world as possible)
*Can identify previously unknown threats and gaps in security
*Can provide a diverse outlook on a problem
*Can identify new uses for innovations

Weaknesses:
*No textbook way to conduct red teaming
*Can verge on science fiction
*Red Team can be marginalized
*Results can be ignored by SME if non-SME conduct the exercise
*Red Team may pursue path of least resistance and not take the excercise seriously
*Participants must fully understand culture of the Red Team they are playing to make the most of the excercise
*Requires leadership committed to making changes based on the recommendations of the Red Team
*Memebers of the Red Team may be of a poor quality and thus negate the usefullness of the exercise
*May be used to politicize intelligence ( see The Power of Nightmares)
*No particular guidance on how many team members are needed to function optimally
*Does not always allow creativity
*Excercise can be stifled by rules and parameters set up to "prove a point"
*Can be subject to "groupthink" within the Red Team

How To:
**A hodgpodge of instructions exist on how to conduct red teaming exercises. The "How To" is largely dependent upon the type or form of red teaming being executed.

1)Populate team (can be composed of subject experts or outside consultants chosen for other unique skills/qualities)
2)Receive tasking or objective from management
3)Assume the role assigned with particular consideration of the limitations imposed on the team (cultural, technological, equipment, etc.)
4)Provide the red team with the necessary independance and credibility to challenge existing norms and ideas and suggest "outside the box" ideas.
5) Execute tasking or exercise.
6)Disseminate and if necessary advocate the key findings to the person excercising control over both the red and blue teams.

Experience:
In order to gain a better understanding of the process of red teaming, the group played a conflict simulation game, "Strike Force One". The game is a simple computer-based simluation used to recreate a combat situation in a key area of West Germany during the Cold War. One player, in this case the computer (AI), advances as the Soviet Army, and the human user takes the role of US Army forces defending key towns on the grid. Movement takes place on a board of "hexes" and the result of the combat is determined by simulated die rolls, as any common combat board game. The group "red teamed" the anticipated movements of the enemy forces, but was ultimately defeated by the Soviets.
Here are some of our experiences:
*Team members dissented over the most likely movements of advesaries - illustrating the multitude of options available for these adversaries.
*Relatively easy to anticipate the adversary's general movements, but more difficult to anticipate which specific move he would choose; but combat results are not entirely predictable.
*Team still had the US Army's interests in mind; not able to fully think like the adversary.

Thursday, April 9, 2009

The Role And Status Of DoD Red Teaming Activites

Defense Science Board Task Force
September 2003
Sections I-III,VI


Summary
Red teams can be a powerful tool to understand risks and increase options. Their purpose is to reduce an enterprise’s risks and increase its opportunities. Red teaming can be used at all three levels of an enterprise: strategic, operational, and tactical. The Defense Science Board, however, found that the use of red teams within the Department of Defense is mixed at best.

Despite this mixed record, the Defense Science Board concluded that the use of red teams is especially important given today’s climate. Adversaries are tough targets for intelligence (compared to the Cold War). Red teaming can both complement and inform intelligence collection and analysis. Aggressive red teams challenge emerging operational concepts in order to discover weaknesses before real adversaries do. Red team also tempers the complacency that often follows success (referenced to the time period following Desert Storm).

Red teams come in many varieties and there are different views about what constitutes a red team. The Defense Science Board defined the term broadly, including not only playing the adversary, but also playing devil’s advocate and related roles. While differing in some respects, these activities all have in common the challenging of an organization’s norms. A red team is comprised of individuals selected for their special subject matter and expertise, perspective, imagination, or critical analysis. The red team itself is only one element in a red teaming process. Elements of the process include who the red team reports to, how the red team interacts with the management of the enterprise, and how the enterprise considers the use of the red team’s products.

Although red teaming is important, it is not easy and rarely done well. Typical causes for red team failure include the read team not taking their tasking seriously, the red team loses its independence, and the red team becomes removed from the decision making process. Conversely, attributes of an effective red team include an environment that values internal criticism, “top cover” or the support of upper level management to raise issues that may be unpopular, and proper staffing on the red team.
Reblog this post [with Zemanta]

Wednesday, April 1, 2009

Summary Findings: Argument Mapping (3 Stars Out Of 5)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 1 APR 2009 regarding Argument Mapping
specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.


Definition:
Argument mapping (AM) is an analytic modifier that can be used to examine the logic behind the development of a particular conclusion and/or hypothesis. The product of AM is a visual representation (typically a box-and-line diagram) of the reasons that support and oppose the claim. Constructing a visual depiction of a complex argument reduces the level of abstraction in evaluating a decision.

Strengths:
Provides Audit Trail
Breaks down prose arguments and visualizes arguments for decision makers.
Shows strengths and weaknesses of argument
Improves critical thinking
Reduces cognitive bias and blindspots

Weaknesses:
Not estimative in nature.
The method does not explicitly seek out contradictory evidence.
The language specific to argument mapping is not uniform from scholar to scholar.
Dependent on the quality of presentation of original argument
The technique does not consider the impact of deliberately deceptive information

How-To:
* Must locate a central claim for a position (this is the conclusion of an argument)
--conclusion indicators include: therefore, thus, so, hence
* extrapolate true and logistical reasons and objections which support or refute the central claim.
--rewrite statements as individual sentences
--Reasons must answer the question: "How do we know that [insert claim] is true/warranted?"
* include all premises for reasons - leave nothing to be implied
--premise indicators include: since, because, for, given that
* With objections, list any possible refutations underneath the objection
* Form a 'warrant' - a statement that justifies the step from a reason to a claim
--warrants reveals an arguer's core implicit beliefs
--used to scrutinize the soundness of an argument
--used to accept or reject controversial arguments
*Use arrows to indicate which premises or co-premises support claims; or premises refute oppositions.
*indicate premises that need to be combined in order to support a conclusion, and premises that are each seperate reasons to believe a conclusion
*Use a color-coding scheme (if possible) to help visualize the arguments
--Supporting claims - green
--opposing claims - red
--nuetral claims - gray or no color

Experience:
We attempted to argument map a one page NY times editorial. The comments below were generated as a result of that exercise.

Argument mapping is difficult to synthesize from prose that is not written in a BLUF (Bottom Line Up Front) format.

The ability to effectively argument map is too dependent on quality of the articles given.

People will disagree on structure of argument and levels of relevance of data.

Argument mapping is not appropriate to use to develop an analytic estimate, rather it can be best implemented as a modifier to validate an already determined estimate.

Argument mapping would be an excellent modifier to test the validity of a conclusion/forecast. Would be an excellent supplement to ACH - suggest conducting an ACH to find a claim that is most likely, then test that claim for validity using Argument Mapping. This will help to reduce cognitive blindspots and biases, as well as test the reasoning behind the conclusion.

Extrapolating the main claim of the argument was easier once all of the premises supporting it were laid out.

The original format of the 'argument" really matters when it come to the ease or difficulty of applying argument mapping to the text.

Thursday, March 26, 2009

Argument Mapping (Efebia)

Summary
Argument mapping is a logical sequencing method that employs box-and-line diagrams to "map" a course of possible decisions for a given "argument" and the ramifications of each option. The purpose of the map is to provide a visual depiction of the relationships between the overall contention and the logical (evidential) claims supporting or opposing it. Upon examination of the supporting and opposing claims, an argument map should serve as a useful tool for use in the decision making process, by reducing the complexity of a dilemma, conflict, or argument. Since argument mapping distinguishes itself from other box-and-line methods by utilizing different shapes, colors, and positioning, it appeals to the strong visual comprehension abilities of humans, allowing for improved processing and understanding, ultimately simplifying the complexity of an argument.

Basic Construction of an Argument Map
The components of an argument map are listed below with a brief explanation. Those pieces of the map are identified in the example below (click on the map to link to the source and to access a higher resolution map).
  • The main claim: referred to in this article as the "position" or "contention" -is the hypothesis to be logically examined. The argument map's purpose is to help the decision maker accept or reject this hypothesis.
  • Reason: a positive claim, or one that directly supports the main claim. Reasons not only support the main position, they may also support another reason.
  • Objection: a negative claim, or one that opposes the overall main claim.
  • Rebuttal: opposes an objection directly above it; or, "objection against an objection".
Strengths of Argument Mapping
According to the article, by using colors, shapes, etc. in argument mapping, the mind can better process the complexity and abstract nature of a difficult argument. Additionally, the logical structure of argument mapping is easier to present than traditional prose, which requires the reader to deconstruct the argument for himself, demanding a large investment in time and cognition.

The article also cites "extensive research" conducted by the University of Melbourne to gauge the effectiveness of argument mapping in the context of critical thinking. The study compared the achieved critical thinking abilities of students who utilized the Reason!able (Rationale) argument mapping tool versus those who used traditional prose. The study determined that, in a 12-week course, students who used argument mapping gained 12 IQ points.

Author's Comment:
Since this article may have commercial implications (as it names a particular brand of software), it is obviously a strong advocate for argument mapping, and does not approach the method with much objectivity. While there may be some obvious disadvantages to using argument mapping in certain situations, none are provided here. Moreover, the only other method this article discussed was traditional prose, which is not an analytic method by itself, but a medium of production or dissemination. Despite a mere mention of conceptual modeling and flow charts, there was no substantive, formal comparative discussion on them.

Wednesday, March 25, 2009

Summary Findings: Dialectic And The Socratic Method (3 Stars Out Of 5)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 25 MAR 2009 regarding Dialectics generally and the Socratic Method specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Definition:

The Dialectic Method is an analytic technique designed to force the participants to re-examine their internal beliefs, biases, and conclusions through an open and directed dialogue.

The Dialectic Method uses questioning techniques with the intention of creating a better understanding of a problem or concept. In the realm of intelligence analysis, it should be used as an analytic modifier; i.e. a technique to reassess the validity of the analytic process, not as a forecasting method.

Strengths:

--The primary strength is the ability to identify and challenge initial assumptions about a target, and in effect, it reduces prejudice and bias.
--The dialectic method is also useful throughout the intelligence cycle from requirements, estimative conclusions, and feedback.
--Using dialectic demands the analyst think critically about the certainty of the analysis generated.

Weaknesses:

-- It does not provide an analytical forecast by itself.
-- The questioner needs to be highly skilled in managing the process.
-- As a cautionary note, thinkers caught in their own illogical concepts may become irritated or even angered by such an approach.
-- The approach can be time consuming, and should not be used under time constraints.

How-To:

-- The first step is to provide an initial, well-formulated question with group-wide understanding of the hypothesis at-hand.

-- After the initial hypothesis is presented, the group undetakes an opposing line of questioning to disect the hypothesis and its sub-components.

-- Use the discussion to synthesize arguements for and against the initial hypothesis to determine its truth and validity.

Experience:

We applied the Socratic Method specifically and the principles of dialectics generally to a variety of realistic intelligence situations. We explored how a formal questioning approach that assumes an antithesis, for example, could perhaps have impacted the estimate regarding the presence of WMD's in pre-war Iraq. Even if a Socratic approach to questioning the conclusions of that estimate would not have changed the overall finding, the group generally agreed that it would probably have altered the final level of confidence.

We also applied the method to the requirement phase where the decisionmaker who wants "everything" could be seen as establishing a thesis while the intelligence professional who knows that the decisionmaker doesn't need everything essentially establishes an antithesis. The ultimate intelligence requirement could then be seen as the synthesis of the two positions.

Wednesday, March 11, 2009

Welcome!

Welcome to the blog for the Spring 2009 Advanced Analytic Techniques class at Mercyhurst College. Each week we will be taking a hard look at a particular technique used by analysts in the law enforcement, business or national security intelligence communities.

Students in this class are required to identify and summarize two "resources" on each technique before Tuesday of each week. Resources include primary and secondary sources of any kind on the particular technique we are studying. This includes, books, articles, case studies, websites, company profiles, etc.

The summaries, however, will focus on four main questions. Based on what is contained in the selected resource:
  • What does this technique do (or purport to do)?
  • What are its strengths?
  • What are its weaknesses?
  • What step-by-step actions should you take to use this method?

Obviously, not all of these questions will be answered in each of the resources the students find and summarize. However, the collected set of resources that all of the students find should, ideally, create a consistent series of answers to the four questions above.

Ideally. That is the operative word here. Anyone who has worked with a variety of intelligence analysis techniques knows that consistency is not necessarily their hallmark. My expectation is that we will learn as much (if not more) from those techniques that are poorly defined as we will from those whose definitions are clear, concise and consistent.

Let the learning begin!