Summary of Findings (White Team): Role Playing (4 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 29 April 2010 regarding Role Playing specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Role-playing has been used and continues to be employed as a tool to facilitate training in vocational and interpersonal skills and as a method of active learning in educational and commercial settings. Though there is no single type of role-played exercise, the term is best applied where the teaching and learning experience rests with a form of `as-if’ experimentation.

The methodology is a recognized and successful tool in behavioral assessment, which encourages the application of the technique. In crisis negotiation role playing, which was done in today's exercise, participants used communication skills and reasoning to resolve crisis situations. Results from Role Playing are qualitative results that allow a detailed view of the particular scenario and possible outcomes. The White Team recognizes this technique as a 'method' as opposed to a 'modifier'.

Strengths:

• Allows for development of ideas and thoughts in a controlled environment.
• Provides an opportunity for active learning in a safe, low-risk environment.
• Allows for the application to multiple different types of situations or persons.
• Can be easily conducted as a low cost exercise.
• Simulates the emotions of an actual event which engages participants in the process.

Weaknesses:

• Participants may not take the exercise seriously.
• Participants may act as a caricature of the person they area assigned, rather than respond as the person.
• Participants may have a use different decision-making processes due to their social upbringing.
• A false confidence in their understanding of the role they are playing may result in participant overconfidence in their predictive ability of the person's action.
• The situation in which the participants are placed may be missing critical elements of reality present in emergency situations.

How To:

• Begin with a historical background and basic understanding of the topics / issues that is the subject of the role play
• Discuss the problem to be portrayed
• Determine the teams/groups
• Define the roles of the members of the group
• Make sure that the participants are committed to the process.
• Act out the situation, responding spontaneously.
• Evaluate the actions of each player.
• Repeat, taking different courses of action if needed.
• Discuss results and feedback

Application:

Students were presented with a scenario where there was a sinking ship and only 10 people could fit on the life boat. Each student picked a role to play out of a hat and introduced themselves to the group. After all the roles were announced each made their case as to why they should stay on the boat. Some participants voluntarily requested to remain on the sinking ship where others pleaded their individual cases to secure a spot on the life boat. Through this discussion it allowed different conceptual models to emerge that influenced the way decisions were made. We also came to the understanding that in order to role play it is important that all participants take their part seriously as it influences the outcome of the exercise. It is also important to note that value assumptions,mirror imaging and cultural biases are likely to manifest and it is vital to the success of the exercise to mitigate those factors.

Using the Delphi Technique to Achieve Consensus

How it is leading us away from representative government to an illusion of citizen participation.

This article explained the Delphi Technique and how it is used as a methodology in the education system. It was critical of the technique and explains the consequences and motivations of those who are using it.

The Delphi Technique and consensus building are both founded in the same principle - the Hegelian dialectic of thesis, antithesis, and synthesis, with synthesis becoming the new thesis. The goal is a continual evolution to "oneness of mind" (consensus means solidarity of belief) -the collective mind, the wholistic society, the wholistic earth, etc. In thesis and antithesis, opinions or views are presented on a subject to establish views and opposing views. In synthesis, opposites are brought together to form the new thesis. All participants in the process are then to accept ownership of the new thesis and support it, changing their views to align with the new thesis. Through a continual process of evolution, "oneness of mind" will supposedly occur.
In group settings, the Delphi Technique is an unethical method of achieving consensus on controversial topics. It requires well-trained professionals, known as "facilitators" or "change agents," who deliberately escalate tension among group members, pitting one faction against another to make a preordained viewpoint appear "sensible," while making opposing views appear ridiculous.

The facilitators or change agents encourage each person in a group to express concerns about the programs, projects, or policies in question. They listen attentively, elicit input from group members, form "task forces," urge participants to make lists, and in going through these motions, learn about each member of a group. They are trained to identify the "leaders," the "loud mouths," the "weak or non-committal members," and those who are apt to change sides frequently during an argument.

Suddenly, the amiable facilitators become professional agitators and "devil's advocates." Using the "divide and conquer" principle, they manipulate one opinion against another, making those who are out of step appear "ridiculous, unknowledgeable, inarticulate, or dogmatic." They attempt to anger certain participants, thereby accelerating tensions. The facilitators are well trained in psychological manipulation. They are able to predict the reactions of each member in a group. Individuals in opposition to the desired policy or program will be shut out.

The Delphi Technique works. It is very effective with parents, teachers, school children, and community groups. The "targets" rarely, if ever, realize that they are being manipulated. If they do suspect what is happening, they do not know how to end the process. The facilitator seeks to polarize the group in order to become an accepted member of the group and of the process. The desired idea is then placed on the table and individual opinions are sought during discussion. Soon, associates from the divided group begin to adopt the idea as if it were their own, and they pressure the entire group to accept their proposition.

How the Delphi Technique Works
First, a facilitator is hired. While his job is supposedly neutral and non-judgmental, the opposite is actually true. The facilitator is there to direct the meeting to a preset conclusion. The facilitator begins by working the crowd to establish a good-guy-bad-guy scenario. Anyone disagreeing with the facilitator must be made to appear as the bad guy, with the facilitator appearing as the good guy.

Next, the attendees are broken up into smaller groups of seven or eight people. Each group has its own facilitator. The group facilitators steer participants to discuss preset issues, employing the same tactics as the lead facilitator. Why hold such meetings at all if the outcomes are already established? The answer is because it is imperative for the acceptance of the School-to-Work agenda, or the environmental agenda, or whatever the agenda, that ordinary people assume ownership of the preset outcomes. If people believe an idea is theirs, they'll support it. If they believe an idea is being forced on them, they'll resist.


How to Diffuse the Delphi Technique

Three steps can diffuse the Delphi Technique as facilitators attempt to steer a meeting in a specific direction.

Always be charming, courteous, and pleasant. Smile. Moderate your voice so as not to come across as belligerent or aggressive.

Stay focused. If possible, jot down your thoughts or questions. When facilitators are asked questions they don't want to answer, they often digress from the issue that was raised and try instead to put the questioner on the defensive. Do not fall for this tactic. Courteously bring the facilitator back to your original question. If he rephrases it so that it becomes an accusatory statement (a popular tactic), simply say, "That is not what I asked. What I asked was . . ." and repeat your question.

Be persistent. If putting you on the defensive doesn't work, facilitators often resort to long monologues that drag on for several minutes. During that time, the group usually forgets the question that was asked, which is the intent. Let the facilitator finish. Then with polite persistence state: "But you didn't answer my question. My question was . . ." and repeat your question.

Never become angry under any circumstances. Anger directed at the facilitator will immediately make the facilitator the victim. This defeats the purpose. The goal of facilitators is to make the majority of the group members like them, and to alienate anyone who might pose a threat to the realization of their agenda. People with firm, fixed beliefs, who are not afraid to stand up for what they believe in, are obvious threats.

At a meeting, have two or three people who know the Delphi Technique dispersed through the crowd so that, when the facilitator digresses from a question, they can stand up and politely say: "But you didn't answer that lady/gentleman's question."

Establish a plan of action before a meeting. Everyone on your team should know his part. Later, analyze what went right, what went wrong and why, and what needs to happen the next time. Never strategize during a meeting.

link: http://www.eagleforum.org/educate/1998/nov98/focus.html

This process sounds very political in this type of setting and that personal agendas, or vendettas may come into play if a facilitator is not in control of the situation. The illusion of a representative participation is obvious in this type of situation.

Hypergame Analysis, Part 1

In situations of competition and conflict, no single player can dictate the outcome. What occurs depends on the strategy each player pursues. In turn, the strategy each player pursues depends on the strategy each player believes his or her opponent will pursue, and so on. Analysts often use game theory to model such situations.

In 1977, Peter Bennett introduced hypergame analysis, an elegant and useful extension to game theory. Unlike standard game theoretic models, Bennett’s concept permits players to perceive different games. This feature better approximates real-world conditions and, in particular, allows analysts to model situations involving manipulation, stratagem, and deception more directly.

In hypergame terms, a situation in which both players correctly perceive the same game is designated a level-zero hypergame. A situation in which both players believe they are playing the same game while at least one player misperceives the game is designated a level-one hypergame. A situation in which at least one player perceives the other player’s (assumed) misperceptions is designated a level-two hypergame.

Example:
A scammer offers a great deal to a mark on the street: “My friend’s business has failed,” says the scammer, “and I’ve got a van full of DVD players I need to sell quickly at a great price.” The mark hesitates. Maybe they’re stolen, he thinks. He decides to take a look anyway. The scammer opens the back of a van containing stacks of boxes. He opens one to reveal an off-brand but slick-looking portable DVD player. “This is yours for $20,” he tells the mark, who weighs the opportunity. The stuff’s obviously boxed, the mark tells himself; maybe it’s not stolen after all. He ignores his initial misgivings, hands over a twenty, and walks away with a mint-in-the box DVD player, or so he believes. When he gets to his car, he eagerly opens the box and discovers a brick. He drives back to the scene of the crime, but the scammer is gone. This situation is easily described using the hypergame framework. The mark assumes the two are playing the same game. In this game, the scammer’s options are {(sell a stolen player) (sell a legitimate player)} while the mark’s options are {(buy a player) (walk away)}. The mark’s challenge, then, is to decide whether the players are stolen. If the mark doesn’t care either way, then the choice is easy: buy a player. The scammer is playing a different game. For the sake of this example, let’s assume the scammer keeps a couple of real DVD players handy in case he suspects the mark might blow the con. In the scammer’s game, then, the mark’s options are {(buy a player) (walk away) (blow the con)} and the con’s options are {(sell a broken player) (sell a working player)}. If all goes well for the scammer, the mark never suspects (1) the scammer is playing a different game and (2) the scammer is playing a higher-order game–that is, the scammer is not only playing a different game but is aware of the mark’s misperceptions. This yields an advantage to the scammer. As long as the mark doesn’t suspect that most of the boxes contain bricks, he believes his choice is simply an ethical one: should I buy possibly stolen merchandise ? The concept of higher perspectives is sometimes referred to as expectation.5 Expectation is arguably as critical to the hypergame approach as is the more basic concept of different games.

Weaknesses: Hypergames of level three and higher are possible but challenging because they require increasingly convoluted mental recursions (I think he thinks I think he thinks, and so on). Hypergame analysis as described in the existing literature can be fairly complex. It is not something an interested analyst or red teamer will typically pick up in a day. When it is used, it is usually delegated to a specialist, who must then translate the outcome back into terms a decision maker can absorb.

Strengths: As the example illustrates, the player who correctly perceives a level-two hypergame enjoys a clear decision advantage over a player who believes the two sides are playing the same game. This situation does not necessarily arise by chance, and a clever player will aim to create and exploit such conditions. As a result, the benefit of hypergame modeling to a red teamer or decision maker rests not strictly in describing a situation but also in modeling a situation explicitly in order to gain a position of advantage.
Awareness of the hypergame construct encourages a player to avoid granting his or her opponent a position of advantage.

Conclusions: In any game-like contest, a player should always remember to ask “what do I perceive, and what does my opponent perceive?” To be avoided, for example, are states in which you, a player, believe you and your opponent are playing the same game when your opponent is actually playing a level-two hypergame. To be sought are states in which these roles are reversed.

Summary Of Findings: Bayesian Analysis (4 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 6 MAY 2009 regarding Bayesian Analysis specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:
Bayesian analysis is a method that uses Bayesian statistics to assess the likelihood of an event happening in light of new evidence. It generates an estimate and the use of Bayesian statistics in Intelligence analysis allows for the uncertainty of the traditional intelligence data set to be understood in a scientifically valid manner.

Strengths:

*can limit analyst biases by reducing the weight of evidence simply because it is new or vivid

*forces the analyst to resassess evidence and consider alternative possibilites

*adheres to rigid mathematical formulas

*provides a numerical likelihood

*provides audit trail and ability to reproduce results

Weaknesses:

*Probabilities are based largely on subjectivity

*Susceptible to biases

*Highly complex problems require heavy computations

*Can be mathematically complex

*Not always useful as a stand alone method (works well in tandem with methods like Delphi); may require SMEs for determining probability distributions

*Some reliance on ambiguous validities

*"Negative evidence"--absence of positive evidence

How-To:

This method loosely follows the guidance suggested by his line of research into the use of natural frequencies in teaching and explaining Bayes to beginners.

1.) Create a 2x2 matrix. Label the quadrants with the respective information that creates true positive, false negative, false positive, and true negative quadrants.

2.) Take the given information, the base line (for example, 100 out of 1,000) with the new information (for example, a new document that is 90% credible saying that war is immiment) which means that your true positive and your false negative must equal 100 and the false positive and true negative must equal 900.

3.) To calculate the true positive quandrant, take 90% of the 100 from the base line (which equals 90).

4.) To calculate the false negative quadrant, take the numerator of the base line (100) and subtract the true positive quadrant (90), creating an answer of 10.

5.) To calculate the true negative quadrant, take 90% of your non-war cases (900), equalling 810.

6.) To calculate the false positives, subtract the sum of the three quadrants known from the total number of cases (1,000), which equals 90.

7.) To calculate the new probablitiy, divide what the numerator of the base line (100) from the new total of positive caes (90+90=180), which equals 55.5%

The 55.5% means that there is a 55% probability that countries X and Y are likely to go to war.

Experience:

To understand the basic mathematical principles behind Bayes, the class worked through some sample problems. One of the problems was based on a medical test with an 80% accuracy rate for a cancer with 2% affliction rate in the general population. The class applied this to a sample population of 1000 cases. We established a matrix and assessed the true positive, false positive, false negative, and true negative quantities (16, 116, 4, and 784 respectively). We plugged these numbers into the appropriate matrix fields. We then divided the number of actual cases of cancer (20 or 2% of 1000) into the number of positive tests (132--the 16 true positives and 116 false postives). The result was 15% rate of those who have the cancer from the positive tests, a rather stark difference from the 2% base rate! This problem actually reflects the number of breast cancer rates from a medical treatment from around a two decades ago!

Note: see the matrix for a synopsis of another of the problems we worked through (a peice of evidence emerging suggesting a cause for war).

The class also used a Bayesian application to assess the likelihood we would contract swine flu. We started with the initial hypothesis that we would contract swine flu or we would not contract swine flu, and assigned an initial probability to each hypothesis (the latter >5%) We then added weighted evidence which influenced the base rate of the hypothesis. After all the evidence was entered, the class assessed the likelihood of contracting swine flu.

What is Bayesian Analysis?

http://www.bayesian.org/bayesexp/bayesexp.html

This article is a summary of the International Society for Bayesian Analysis's (ISBA) definition of the basics of the Bayesian theorem and analysis. The according to the ISBA website, the organization "was founded in 1992 to promote the development and application of Bayesian analysis useful in the solution of theoretical and applied problems in science, industry and government. By sponsoring and organizing meetings, publishing the electronic journal of Bayesian statistics Bayesian Analysis, and other activities ISBA provides a focal point for those interested in Bayesian analysis and its applications".

Summary
Bayesian analysis, a statistical tool for handling probability distributions, got its start in the mid 18th century. It was not, however, until the 1980s when modern computers were able to handle the complex computations that made Bayesian implementation difficult, that Bayesian analysis gained more widespread acceptance. Since then, its use has increased in popularity, being used in many different applications--from healthcare, to weather, to criminal justice. Despite the many nuanced manifestations of Bayesian analysis, it serves a common application: to analyze the probability of unknown and uncertain occurences.



How To:

The left side of the equation expresses the known quantities--"parameters"--as a probability of the current data--"prior distribution". 'y' represents the new data that enters into the calculation. Thus, the "'likelihood,' [is] proportional to the distribution of the observed data given the model parameters.

On the right, the equation's new probablity distribution (posterior distribution) is read: "posterior is proportional to the prior times the likelihood".

Strengths:

• Many diverse applications
• "Philosophical consistency"
• Lacks problems that are associated with other 'frequentist' methods
• Produce clear answers, products
• Reformulates for each variable

Weaknesses:

• Subjective nature of prior probabilities-"your prior information is different from mine".
• More complex problems require more powerful computational tools

Summary Of Findings: Gap Analysis (3 Out Of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 29 APR 2009 regarding Gap Analysis specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:
Traditionally, "gap analysis" is a method used to conduct an internal operational analysis, whereas the gap analysis identifies the "gap" between a current state and a desired endstate within a company or agency. From an intelligence analysis perspective, "gap analysis" can be used as a tool to identify the likely pathway or pathways a target may take to arrive at a given endstate from a known position. Thus, "gap analysis" does not necessarily provide an estimate, but rather provides the analyst with a list of possible actions a target may likely take. Gap analysis as an analytic technique bears a striking resemblance to several other methods, such as Indicators & Warnings and Decision Trees.

Strengths:

1) Identify the target

2) Characterize the current status of the target as well as the target's goals

3) Identify what you want to know about the target

4) List the pieces of information that you have

5) Use a systematic approach to infer what the target is likely to do in order for the target to reach its goals

Weaknesses:

* No structured method to conduct the analysis

* May not leave you with a clear estimate

* Open to bias and other cognitive downfalls (satisficing, mirror imaging, etc.)

* Overlaps with the process of other methods and modifiers (i.e. decision treees, I&W, Brainstorming, SWOT, etc.)

* Susceptible to deception

* Danger of pitfalls

How-To:

1) Identify the target

2) Characterize the current status of the target as well as the target's goals

3) Identify what you want to know about the target

4) List the pieces of information that you have

5) Use a systematic approach to infer what the target is likely to do in order for the target to reach its goals

Experience:

For the first application, the group tried to determine what thesis topic Mary, a fictional first year graduate, would write about. Professor Wheaton acted in place of Mary, and we role played in questions and answer format. The group determined the gaps needing filled would be what intelligence track she was most interested in (national security, law enforcement, or competitive), what area or topic in her previous classes interested her the most, the choice of her primary reader, and the reader's academic interests. Upon filling these gaps, the group ascertained a plausible topic for Mary's thesis.

For the second application, the group discussed Russia's long held ambitions for a warm water port, preferably on the Mediterranean. The first part of the discussion centered on hostilities between Georgia and Russia, and actions Russia could take against Georgia to maintain their sphere of influence in the Black Sea. Additionally, the group discussed what actions may be necessary in their diplomatic relations with other nations bordering the sea. After discussing how Russia could potentially gain a Black Sea port by bringing Georgia into their sphere of influence, the group discussed how Russia could proceed toward gaining access to Mediterranean ports. The group determined that Turkey would be central in future Russian objectives for ports in the Mediterranean. The group put together lists of things the Russians could do that they are not doing now that would indicate their goals of extending their influence in Georgia and Turkey.

These applications illustrated the following:

* Helped in thinking through the steps that lead to a decision

* Allowed for open discussion and debate, helping the critical thinking process

* At some points it felt like stabbing in the dark, but the estimates later became more clear as the group discussed the options

* Eliminated peripheral influences not directly related to the topic, such as administrative processes.

Summary Of Findings: Game Theory (4 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 22 APR 2009 regarding Game Theory specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Game theory is a method based on applied mathematics and economic theory. It can be useful when attempting to analyze (and ultimately predict) the strategic interactions between two or more actors and the way in which their actions influence future decisions. Game theory assumes that all actors are rational, and can be influenced by various individuals and factors. Games typically involve five common elements: players, strategies, rules, outcomes, and payoffs.

Strengths:

-assumes rational actors

-assumes actors will adjust their actions based on the actions of other actors

-not clearly differentiated from role-playing, simulations, and/or decision trees

-very mathematically based (can be intimidating)

-difficult to quantify options, strategies, and motivations

-may not be a valid method to produce an accurate estimation (see Game Theory, Simulated Interaction, and Unaided Judgment For Forecasting Decisions in Conflict: Further Evidence)

--In real world applications, identifying all of the key players and outcomes can be difficult

Weaknesses:

-Visual step-by-step trail to a conclusion/estimate

-Ability to quantify variables in play

-Emphasis on mathematics and scientifc method

-Applicable to multiple fields (economics, conflict, etc)

-90% rate of success according to BDM

How-To:

Game Theory varies in complexity and in application, however, each application has the following in common:

*Establish the players and the complexity of the game being played, so as to understand the rules which govern the players and the game.

*Identify the possible outcomes for the choices the players can make (although this is particularly difficult as not all decisions can be predicted)

*Establish measurable values for predicted outcomes.

*Eliminate dominated strategies and employ dominate strategies. Repeat this step until a clear, singular strategy emerges or equilibrium is reached between the players.

*Employ selected strategy.

Experience:

As a class, we visited www.gametheory.net and played the repeatable version of Prisoner's Delemma under the "Interactive Materials" tab. Each student played the game at their personal computers. Our objective as we played against the five "personalities" was to identify the particular strategies employed by the computer (in addition to scoring the most utility points). Some of the strategies employed by the computers personalities included "tit-for-tat" and "tit-for-two tats."

Game theory, simulated interaction, and unaided judgement for forecasting decisions in conflicts: Further evidence

Kesten C. Green

Summary
Green's article, published in the International Journal of Forcasting (2005), is a validity study that compares the accuracy of game theory, simulated interactions (role playing), and unaided judgment as methods in forecasting decisions in conflicts. It provides further evidence to support his earlier research, which indicated that expert game theorists' forecasts had less accuracy than those of novice (student) role players.

It is important to note here that these methods were tested in experiments involving specific conflict situations only. Not all game theorists recommend game theory for generating forecasts in specific situations.

Green's 2002 study reflected the accuracy of the three methods for forecasting conflict decisions. Participants in the study were asked to select the most likely decision (in 5 conflict situations) from a list. The results of the study did not bode well for the game theory experts (participants, % accuracy):

• University Students (using unaided judgment), 27%
• Game Theory Experts, 28%
• University Students (using role playing/simulated interactions), 61%

Green's 2002 study drew some attention from his peers and game theorists, who commented that perhaps some conflict situations are more appropriate for use with game theory than others. The 2005 study utilized the same basic process, and was administered to participants representing the aforementioned three methods. It was directed at three (actual) conflicts, unrelated to the five used in the 2002 study:

1. Personal Grievance. An employee of a New Zealand student association felt there was a disparity between her work and the level of pay she was receiving. Upon the administration's evaluation of her situation, it was discovered that she was being compensated above the top-level of pay in her salary bracket. While her manager did not consider cutting her salary, she did arrange for a mediator to set up a meeting between the parties, as it was obvious she would not be elligible for a pay increase in the near future. Participants were to choose from several decisions.
2. Telco Takeover. This situation presented the participants with the conflict stemming from a corporate takeover battle between Alltel and CenturyTel. In 2001, executives from the much smaller CenturyTel company presented an offer that involved the sale of their mobile phone division to Alltel, which Alltel declined. Soon after, Alltel returned to CenturyTel with an offer to buy all of the company at 40% more than the share price. The board of CenturyTel fought to prevent an Alltel takeover. Participants were to forecast the execution of the deal and predict the basis of terms.
3. Water Dispute. This conflict raged over Syria's and Iraq's claims to access of the Euphrates River in 1975. The flow of water into Iraq had been slowed following Syria's construction of a dam across the Euphrates to create a reservoir in Syria. Both sides mobilized for war, and Saudi Arabia called both parties together for mediation. Participants in the study were to forecast whether Syria would increase water flow from the dam or whether Iraq would declare war and bomb Syria's dam.

Since the findings of this research were to be compared with those of the 2002 study, Green took steps to evenly match the conflict situations with game theorists' and other decision makers' interests, i.e. types of organizations involved, nature of the disputes, and familiarity of situations.

Results
The findings of this follow-up study are entirely consistent with the 2002 study. Game theory experts and unaided novices had a much lower forecast accuracy than novices who implemented role playing (simulated interactions) in predicting the eight conflict decisions.

Summary Of Findings: Red Teaming (2 Stars Out Of 5)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 15 APR 2009 regarding Red Teaming specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:
Red Teaming is an analytical modifier that can be used in two distinct ways:
First, in an objective sense, it is used to challenge emerging operational concepts in order to discover weaknesses with an organization's procedures and reactions.
Second, in a subjective sense, red teaming is used to generate options for adversaries that may be overlooked due to biases or heuristics.
When red teaming is used in the first manner, the effectiveness of red teaming is usually easier to monitor and evaluate. When using the second approach, it is more difficult to measure the effectiveness of red teaming, as the effectiveness is subject to forces outside of the method itself.

Strengths:
*Forces "thinking outside the box"

*Challenges "groupthink"; may reduce bias

*Can provide invaluable near-real world training (as close to real world as possible)

*Can identify previously unknown threats and gaps in security

*Can provide a diverse outlook on a problem

*Can identify new uses for innovations

Weaknesses:
*No textbook way to conduct red teaming

*Can verge on science fiction

*Red Team can be marginalized

*Results can be ignored by SME if non-SME conduct the exercise

*Red Team may pursue path of least resistance and not take the excercise seriously

*Participants must fully understand culture of the Red Team they are playing to make the most of the excercise

*Requires leadership committed to making changes based on the recommendations of the Red Team

*Memebers of the Red Team may be of a poor quality and thus negate the usefullness of the exercise

*May be used to politicize intelligence ( see The Power of Nightmares)

*No particular guidance on how many team members are needed to function optimally

*Does not always allow creativity

*Excercise can be stifled by rules and parameters set up to "prove a point"

*Can be subject to "groupthink" within the Red Team

How To:
**A hodgpodge of instructions exist on how to conduct red teaming exercises. The "How To" is largely dependent upon the type or form of red teaming being executed.

1)Populate team (can be composed of subject experts or outside consultants chosen for other unique skills/qualities)
2)Receive tasking or objective from management
3)Assume the role assigned with particular consideration of the limitations imposed on the team (cultural, technological, equipment, etc.)
4)Provide the red team with the necessary independance and credibility to challenge existing norms and ideas and suggest "outside the box" ideas.
5) Execute tasking or exercise.
6)Disseminate and if necessary advocate the key findings to the person excercising control over both the red and blue teams.

Experience:
In order to gain a better understanding of the process of red teaming, the group played a conflict simulation game, "Strike Force One". The game is a simple computer-based simluation used to recreate a combat situation in a key area of West Germany during the Cold War. One player, in this case the computer (AI), advances as the Soviet Army, and the human user takes the role of US Army forces defending key towns on the grid. Movement takes place on a board of "hexes" and the result of the combat is determined by simulated die rolls, as any common combat board game. The group "red teamed" the anticipated movements of the enemy forces, but was ultimately defeated by the Soviets.

Here are some of our experiences:

*Team members dissented over the most likely movements of advesaries - illustrating the multitude of options available for these adversaries.

*Relatively easy to anticipate the adversary's general movements, but more difficult to anticipate which specific move he would choose; but combat results are not entirely predictable.

*Team still had the US Army's interests in mind; not able to fully think like the adversary.

Summary Findings: Argument Mapping (3 Stars Out Of 5)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 1 APR 2009 regarding Argument Mapping
specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Definition:

Argument mapping (AM) is an analytic modifier that can be used to examine the logic behind the development of a particular conclusion and/or hypothesis. The product of AM is a visual representation (typically a box-and-line diagram) of the reasons that support and oppose the claim. Constructing a visual depiction of a complex argument reduces the level of abstraction in evaluating a decision.

Strengths:

Provides Audit Trail

Breaks down prose arguments and visualizes arguments for decision makers.

Shows strengths and weaknesses of argument

Improves critical thinking

Reduces cognitive bias and blindspots

Weaknesses:

Not estimative in nature.

The method does not explicitly seek out contradictory evidence.

The language specific to argument mapping is not uniform from scholar to scholar.

Dependent on the quality of presentation of original argument

The technique does not consider the impact of deliberately deceptive information

How-To:

* Must locate a central claim for a position (this is the conclusion of an argument)

--conclusion indicators include: therefore, thus, so, hence

* extrapolate true and logistical reasons and objections which support or refute the central claim.

--rewrite statements as individual sentences

--Reasons must answer the question: "How do we know that [insert claim] is true/warranted?"

* include all premises for reasons - leave nothing to be implied

--premise indicators include: since, because, for, given that

* With objections, list any possible refutations underneath the objection

* Form a 'warrant' - a statement that justifies the step from a reason to a claim

--warrants reveals an arguer's core implicit beliefs

--used to scrutinize the soundness of an argument

--used to accept or reject controversial arguments

*Use arrows to indicate which premises or co-premises support claims; or premises refute oppositions.

*indicate premises that need to be combined in order to support a conclusion, and premises that are each seperate reasons to believe a conclusion

*Use a color-coding scheme (if possible) to help visualize the arguments

--Supporting claims - green

--opposing claims - red

--nuetral claims - gray or no color

Experience:
We attempted to argument map a one page NY times editorial. The comments below were generated as a result of that exercise.

Argument mapping is difficult to synthesize from prose that is not written in a BLUF (Bottom Line Up Front) format.

The ability to effectively argument map is too dependent on quality of the articles given.

People will disagree on structure of argument and levels of relevance of data.

Argument mapping is not appropriate to use to develop an analytic estimate, rather it can be best implemented as a modifier to validate an already determined estimate.

Argument mapping would be an excellent modifier to test the validity of a conclusion/forecast. Would be an excellent supplement to ACH - suggest conducting an ACH to find a claim that is most likely, then test that claim for validity using Argument Mapping. This will help to reduce cognitive blindspots and biases, as well as test the reasoning behind the conclusion.

Extrapolating the main claim of the argument was easier once all of the premises supporting it were laid out.

The original format of the 'argument" really matters when it come to the ease or difficulty of applying argument mapping to the text.

Argument Mapping (Efebia)

Summary
Argument mapping is a logical sequencing method that employs box-and-line diagrams to "map" a course of possible decisions for a given "argument" and the ramifications of each option. The purpose of the map is to provide a visual depiction of the relationships between the overall contention and the logical (evidential) claims supporting or opposing it. Upon examination of the supporting and opposing claims, an argument map should serve as a useful tool for use in the decision making process, by reducing the complexity of a dilemma, conflict, or argument. Since argument mapping distinguishes itself from other box-and-line methods by utilizing different shapes, colors, and positioning, it appeals to the strong visual comprehension abilities of humans, allowing for improved processing and understanding, ultimately simplifying the complexity of an argument.

Basic Construction of an Argument Map
The components of an argument map are listed below with a brief explanation. Those pieces of the map are identified in the example below (click on the map to link to the source and to access a higher resolution map).

• The main claim: referred to in this article as the "position" or "contention" -is the hypothesis to be logically examined. The argument map's purpose is to help the decision maker accept or reject this hypothesis.
  
• Reason: a positive claim, or one that directly supports the main claim. Reasons not only support the main position, they may also support another reason.
• Objection: a negative claim, or one that opposes the overall main claim.
• Rebuttal: opposes an objection directly above it; or, "objection against an objection".

Strengths of Argument Mapping
According to the article, by using colors, shapes, etc. in argument mapping, the mind can better process the complexity and abstract nature of a difficult argument. Additionally, the logical structure of argument mapping is easier to present than traditional prose, which requires the reader to deconstruct the argument for himself, demanding a large investment in time and cognition.

The article also cites "extensive research" conducted by the University of Melbourne to gauge the effectiveness of argument mapping in the context of critical thinking. The study compared the achieved critical thinking abilities of students who utilized the Reason!able (Rationale) argument mapping tool versus those who used traditional prose. The study determined that, in a 12-week course, students who used argument mapping gained 12 IQ points.

Author's Comment:
Since this article may have commercial implications (as it names a particular brand of software), it is obviously a strong advocate for argument mapping, and does not approach the method with much objectivity. While there may be some obvious disadvantages to using argument mapping in certain situations, none are provided here. Moreover, the only other method this article discussed was traditional prose, which is not an analytic method by itself, but a medium of production or dissemination. Despite a mere mention of conceptual modeling and flow charts, there was no substantive, formal comparative discussion on them.

Argument Mapping and Analysis of Competing Hypotheses (AM vs ACH)

Summary

This article presents a comparative assessment of argument mapping (AM) and Analysis of Competing Hypotheses (ACH) as analytic diagramming techniques. While the paper focuses primarily on the differing techniques in diagramming a decision making pathway, the author considers both as "complementary analytical frameworks".

Comparing the Techniques

Both techniques involve analyzing a given proposition (hypotheses in ACH; conclusions in AM), evaluating the evidence (evidence in ACH; reasons and objections in AM), and diagramming a visual representation of the relationships (matrices in ACH; box-and-line diagrams in AM). In ACH, the letters C, I, and N represent positive (consistent), negative (inconsistent), or nuetral relationships, respectively. In AM, arrows indicate a relationship between the reasons supporting or opposing the conclusion; color indicates whether the evidence is a reason (green) or an objection (red) for the argument. When a piece of evidence is neither a reason nor an objection (neutral or irrelevant), the creator of the AM may choose either to eliminate it from the map or to make it a different color (typically gray).

Multiple Propositions
Due to the manner in which each of these methods handles multiple propositions, ACH gets the advantage over AM. ACH incorporates multiple propositions simply by adding another column in the matrix. However, adding any additional propositions to an AM may result in a cluttered diagram with crossed lines and complicated pathways. To resolve this issue, it may be more appropriate to duplicate pieces of evidence, or even create an additional map, making it less efficient.

Multi-tiered Evidence
The paper uses an example from Psychology of Intelligence Analysis to demonstrate a situation in which AM may be a more efficient method. The chosen proposition is "Iraq will not retaliate forUS bombing of its intelligence headquarters," and the piece of evidence in question is "Saddam [has made a] public statement of intent not to retaliate". Often, as cited in this example, an additional piece of evidence may be necessary to establish a piece of evidence. When using an ACH, the analyst may need to construct another matrix to validate evidence. The AM technique allows for the addition another level of evidence within the same map. The paper refers to the multi-tiered evidence as "granularity". Granularity allows evidence of differing levels of abstraction to be present on the same map; "the higher the level on map, the more general or abstract the reason or objection".

Assumptions
AM also allows the user to add assumptions or warrants alongside the pieces of evidence. Incorporating an assumption into an ACH would force the user either to add a justification with the conclusion in its cell in the matrix, or combine multiple pieces of evidence together.

Moral choice and dialectic failure.

Myers, Robert J.. Society, Jul/Aug94, Vol. 31 Issue 5, p37-42, 6p; (AN 9407251855)

Summary
Myers calls dialectics a "logical term, used today for abstract disputation, devoid of any practical value". Plato contributes its invention to Zeno of Elea, intended to resolve dilemmas through a series of questions and answers. Myers cites its use within the Socratic Dialogue to identify shortcomings in conventional wisdom, and its later use by Hegel to show the reality of history as only that of ideas--"every idea, as it is affirmed by its truth, brings with it the idea that is its negation." Marx extended this concept into the debate of economic systems. The major historical uses of the dialectic incorporate three important components, which Myers calls the trinity of dialectics:

1. Thesis
2. Antithesis
3. Synthesis

Myers published this paper in 1994, when the fall of the Soviet Union (and, to him, Communism in general) was still a new phenomenon. Thus, his main point of contention was the Hegelian dialectic versus Marxist dialectic. Myers is highly critical of the dialectic explanation of history and its use to forecast the future. Instead, he advocates for an empirical explanation. He flippantly closes the paper, suggesting that "it (dialectics) has come from nowhere and will return to nowhere, allowing the human mind and associated intelligence to continue the search for truth in the abstract and to sort out empirically the good from the bad, the better from the worse, and to construct an ethical scale, weighed with democratic values". It is his contention that the dialectic was "simply a mental construct" and it has reached the end of its importance in "democratic" thought.

Dialectics – a commentary to Singer: “Global business and the dialectic”

Sørensen, Asger. Human Systems Management, 2002, Vol. 21 Issue 4, p267, 3p; (AN 9330655).

Summary

Sørensen uses this commentary on dialectics to build upon a case previously presented by Alan Singer, which considers the use of philosophy, particularly dialectic reasoning, in developing business and political strategy. He believes that the use of dialectics to examine dilemmas, tensions, and contradictions has not been widely used, largely due to its "guilt by association" with marxists and totalitarians.

He begins by providing an overview of the development of dialectics in the realm of philosophy. The main focus of the historical overview is to examine the development and opposing viewpoints of dialectics, from Plato (ideas), Aristotle (deduction), Kant, Hegel, Marx, and Engels (dialectical materialism).

Sørensen further points out that Singer sided with Hegel and Engels, who advocated its application to nature and culture. While Singer endorsed its use as a scheme of thought, he did not fully commit to its necessity in examining tensions.

This examination of dialectics closes with a brief mention of its implications in the realm of business and political strategy. By nature, dialectics is concerned with the truth of reality in its entirety, and is therefore at odds with basing political decisions on the wants and needs of the individual. He adds to this by hypothesizing that "introducing dialectics into business strategy might signal a shift in focus from the market to the organization, i.e., from coping with universal competition outside the firm to handling internal affairs, just as it is relevant in an economy dominated by the monopolies of multinational corporations."