Advanced Analytic Techniques

Learning While You Dream

2010-05-06T11:15:00.003-04:00

This article from the New York Times focuses on a recent study published in Current Biology. the Beth Israel Deaconess Medical Center recently conducted a study focusing on the role of sleep and dreams on learning and problem solving.

Study:
The study focuses on 99 volunteers who trained for an hour on a three-dimensional virtual maze. At the end of the hour, half of the participants were kept awake for 90 minutes but allowed to read or relax. The other half were allowed to sleep for 90 minutes but were awakened periodically and asked to describe their thoughts or dreams. After the 90 minutes were up, all participants were asked to tackle the puzzle again.

Results:
For participants who did not sleep, their second try at the puzzle yielded the same or worse results than the first hour. For participants allowed to sleep, the study registered marginal improvement, though not statistically significant results. However, the 4 participants who reported dreaming about the maze cut their completion time in half, and their final scores were ten times better than those who had slept and not dreamed. All 4 participants who dreamed were recorded as struggling with the maze during the first hour.

Conclusion:
Researchers included several reasons they speculated that those who dreamed improved their scores.

When topics grip people at an emotional level, the brain is forced to continue the process even when the person is asleep.
The brain is tuned to find associations you don't notice when awake.

Researchers also made recommendations based on their study.

If a person wants to improve the likelihood they will dream about a topic, the best way is to become emotionally invested in it.
Researchers also recommend future studies on ways to format information that will increase the ability to induce dreams.

An Intentional Modeling Process to Teach Professional Behavior: Students' Clinical Observations of Preceptors

2010-05-06T10:54:00.003-04:00

Introduction:
In this article, the author describes an innovative approach to role modeling called the Students' Clinical Observations of Preceptors (SCOOP) used to teach medical students professionalism and communication skills during their clinical training. This method is innovative because it reverses the traditional direction of structured observations.

Summary:
The author argues that on a personal and collegial level, professionalism is reflected when physicians practice integrity, honesty, accountability, respect for the expertise of others, self-reflection motivated by a drive toward self-improvement, and awareness of their own limitations. Although medical school report some formal training in professionalism such as isolated didactic presentations during preclinical courses, to courses integrated throughout the entire medical school curriculum, the "informal" and "hidden"curriculum may exert more influence than the formal curriculum in the moral and professional development
of students and residents.

Role modeling is an effective way to teach the "intangibles"; however, it must incorporate two key components: intentional modeling of important learning goals, and focused observation
on the part of the learner. Students' Clinical Observations of Preceptors (SCOOP) provides an observation framework for students that reverses the roles of preceptors and learners in Structured Clinical Observations (SCO).

SCOOP is performed as follows:
The preceptor informs the student that there will be no questions about the content or clinical reasoning reflected in the interaction allowing the student to better focus onthe process, skills, and behaviors being intentionally modeled.

The student is given a check list of clinical skills and behaviors such as such as medical interview skills, relationship skills, and approaches to the physical examination to focus their observations. This framework also helps preceptors place their behaviors in a meaningful context for students.

After the observation commences, the preceptor will ask the student a series of questions to open discussion regarding their observations.

This process is beneficial to the preceptors because it improves their self monitoring and is beneficial to the students because they learn directly from faculty members who deal with uncertainty, deliver difficult information, or recognize areas for professional
improvement.

Conclusion:
This type of learning (role-modeling) can be applied to the field of intelligence because new analysts can benefit from observing the way "mentors" deal with uncertainty, deliver difficult information, or recognize areas for professional improvement. What the article did not mention; however, is outlining the way physicians think or how they diagnose. In order for this to be more effective for analysts, it would be important for mentors to explain their methods and thought processes via decision trees or an alternative tool in addition to displaying professional behavior.

Source:
Jones W, Hanson J, Longacre J. An Intentional Modeling Process to Teach Professional Behavior: Students' Clinical Observations of Preceptors. Teaching & Learning in Medicine [serial online]. Summer2004 2004;16(3):264-269. Available from: Academic Search Complete, Ipswich, MA. Accessed May 6, 2010.

Police Recruit Training: Facilitating Learning Between the Academy & Field Training

2010-05-04T10:34:00.003-04:00

Introduction
This article, coming from the FBI Law Enforcement Bulletin, discusses how new police recruits learn in two very different environments. "Police recruit training occurs across multiple learning activities; it begins in the classroom and ends with hands-on field experience." According to the author there is plenty of literature that focuses on each individual activity assuming that knowledge will transfer between settings, however this is not necessarily the case. There is a growing realization within the law enforcement (LE) community that organizations need to hone in on how learning extends across activities (i.e. - from the classroom to the field).

This study was conducted at a regional police academy in Michigan that is the training headquarters for a medium-sized department with over 200 sworn officers. The author selected 10 cadets quasi-randomly and gathered data through observation and structured interviews. Furthermore, in the field the author rode along with the new recruit and his/her mentor in order to record, transcribe and code the ensuing dialog to see what the recruit had learned in the academy.

The Academy
In this study, the academy work consisted of primarily classroom lecture (a traditional lecture format), however there were other portions of the programs such as firearms proficiency, defensive tactics, first aid, driving strategies and police scenarios. The author notes that the traditional lecture format is the norm across the police academy world and that it follows a behaviorist line of teaching. Furthermore, "learning in the academy depends on lecture and less than 3% of basic training is spent on alternative forms of instruction".

Many leading scholars note that this pen and paper form of learning is outdated and that perhaps a "problem-based approach" would be a better method. This method focuses on using critical thinking skills to remedy real-world problems. According to the author this method stresses an "andragogical approach" which assumes learners are self-directed. Many of the recruits studied, drew upon previous experience in police work to supplement their learning.

Another possible method of teaching that is similar to the andragogical approach is the "constructivist approach". This approach is learner centered, involves reciprocal learning and peer collaboration to teach in an effective manner. In the constructivist approach, the teacher acts as a facilitator who guides individuals/groups in problem-solving and critical-thinking exercises using real-world scenarios.

Field Training
The author notes that learning in the field is much different than in the classroom. In most programs, probationary officers train for 16 weeks, rotating through 3 phases with many different field training officers. This form of learning closely resembles an apprenticeship where the new officer gradually transitions into the role of a full-time officer. This allows the new officer to bridge the knowledge gap between the academy and what actually happens on the streets. The author notes that one finding of his is that the training officers generally take the new officer's knowledge for granted and some training officers even dismissed the significance of the academy as a training tool.

Conclusion
The author stresses the need for LE to overcome the divide between the academy and the field. One way to do that is to use a learner-centered, constructivist approach which will "ensure that recruits engage in learning with which they identify".

Source
For any further information on the study, please see "Police Recruit Training: Facilitating Learning Between the Academy & Field Training" in the FBI Law Enforcement Bulletin, written by Steven Hundersmarck accessed through Academic Search Complete.

How Do People Learn? What Organizations Should Do To Create Better Training Programs Based on Modern Learning Theory

2010-05-04T10:32:00.002-04:00

This article, written by Chris Christensen, is a guide that he suggests corporations should follow (regardless of their industry) in order to provide proper training to their employees that will lead to learning. Based on modern learning studies, Christensen discusses ten (out of fourteen) key principles that he believes every corporate training program (CTP) should adopt.

1. Change the Learner's Mental Models

In this step, the trainer is simply trying to open the learner's mind to new ideas; the learner's mind needs to be "unfrozen, then changed, and finally the new mental model must then be refrozen."

2. Training is Only One Part of a Behavior Change Program

CTP are only a means to an end, therefore training employees itself does not lead to changed behavior. Other elements necessary include clearly defined objectives, rewards for making desired changes, and accountability for displaying the wanted behavior.

3. People Can't Overcome Their Weaknesses

CTP should not focus solely on identifying individuals' weaknesses and training them to overcome those weaknesses. Instead, identifying strengths and training individuals to build upon those to compensate for weaknesses would be more effective.

4. Apply Learning Immediately

Christensen suggests adopting a "Just-In-Time (JIT)" training strategy that allows trainees who learn new skills to apply them to their daily tasks immediately. He argues that if CTP does not allow trainees to apply these new skills immediately it could lower morale and lead to negative views of the employer.

5. Training Must Improve the Business

Donald Kirkpatrick developed a four-level scale to measure the effectiveness of a CTP
Level 1 - Did the trainee feel there was value in the training?
Level 2 - Was knowledge or skill acquired by the trainee?
Level 3 - Did an improvement in the trainee's behavior result from the training?
Level 4 - Did the training increase the quality of the business' performance?
Most corporations have some mechanism for achieving Level 1, but Levels 2-4 are mostly conducted at an academic level, but should be adopted by corporations.

6. Pay Attention to Short Attention Spans

In designing a CTP, corporations should take care as to not overload their trainees. That is, do not schedule an eight-hour training session to be completed in only one day. Most of the trainees will have stopped paying attention by the end of the day and effort could be wasted. Instead, although not as cost-effective, corporations should break these sessions down into, say, four two-hour sessions which would be more effective.

7. Employ the Same Technological Tools and Practices in Training that are Used in Work

The idea here is quite simple: train the employees in the EXACT same environment that they conduct their day-to-day tasks. For example, if the employees undergoing training normally teleconference every day, then conduct the training via teleconference, and so forth.

8. Employ "Different Strokes for Different Folks"

Not everybody learns the same way, some people are visual learners and others may be auditory or kinesthetic learners. In addition, some people may learn in multiple ways and switch between styles (visual to auditory, etc.) Therefore, CTP trainers should employ these three types of learning styles into their teaching methods which would ensure that each student is learning.

9. Training Must Entertain

CTP should include a variety of techniques and material that keep the students engaged. After all, if a student is not interested in the class, it's quite likely that very little or no learning took place.

10. Trainers Must Be Experts

In the corporate world, most trainees expect trainers to be experts in the subject matter in addition to being a good instructor. That is, a trainer should have experience applying whatever it is he or she may be teaching to others.

CONCLUSION
In reading this article, it became quite apparent that Christensen believes that learners in the corporate setting have a much more difficult time acquiring new information than students in an academic setting. I believe that the first step, opening the minds, is the most important that any learner/student must accomplish if he or she is going to be learn anything at all.

Implicit Learning: A New Frontier in Cognitive Psychology

2010-05-04T09:30:00.002-04:00

Dr. Darlene Howard, Davis Family Distinguished Professor of Psychology at Georgetown University, investigates how the human mind absorbs subtle patterns and learns about the world indirectly in the article Implicit Learning: A New Frontier in Cognitive Psychology.

Summary:

Howard and her colleagues in the Cognitive Aging Lab seek to unlock the secrets of implicit learning. Borrowing observation techniques from other areas of research and developing their own behavioral measurements, Howard and her collaborators use carefully designed testing situations to monitor the outcomes of the indirect absorption of patterns and information, allowing researchers to effectively quantify implicit learning

Questions Howard was looking to answer:
Two major questions for this population are :

1) Whether implicit learning declines with age and

2) How implicit learning can be facilitated at all ages.

Imagine a person in their seventies selling their home of 30 years and moving to a retirement home. In the familiar home, they know from experience how to operate the oven and how to get from their bedroom to the bathroom. How quickly can they adjust in the new retirement home? What is the best way for them to effectively familiarize themselves with their new environment? Howard’s research can help reveal the underlying mechanisms of such learning to determine not only how the learning takes place but whether there may be more effective ways of teaching people essential tasks.

Initially, Howard’s research and other studies suggested that during healthy aging declarative, conscious memory declined, while implicit learning was spared. Consider a comparison between a young college student and an elderly person, both moving to a new home: The elderly person is likely to have a hard time memorizing her new phone number while the college student can remember it easily. This is because the task of declarative learning—explicitly memorizing the phone number—is often more difficult for the elderly.

However, both the college student and the elderly person, after a few days of routine, would likely have little trouble going to the correct cabinet to retrieve a water glass without giving it much thought. At least this would be so if implicit learning didn’t decline with aging.

Conclusion:

Howard’s more recent work indicates that while some forms of implicit learning are spared during aging, others are not. Her research has shown that there are multiple forms of implicit learning relying on different brain regions, which may be affected differently by aging. Howard and her collaborators are working to understand how aging affects these different learning systems.

The long-range implications of her research could lead to methods of teaching and learning that effectively help aging persons impaired by aging-induced neurological changes. “We want to understand which implicit learning systems are spared with aging and which aren’t,” Howard says of her intention. “We are working to understand how these are related to the changes in the brain that occur in the course of aging. And most important, we want eventually to figure out what can be done to help people of all ages adapt to changing environments, because this is an essential component of successful aging.”

Game design and learning: a conjectural analysis of how massively multiple online role-playing games (MMORPGs) foster intrinsic motivation

2010-05-04T05:48:00.003-04:00

In this scholarly article published in 2007, the author - Michele Dickey provides an analysis of how the structure of massively multiple online role-playing games (MMORPGs) might influence the design of interactive learning and game based learning environments.

The author primarily looks at how "character design" and "narrative environment" - the two key elements of MMORPGs can support intrinsic motivation which might foster learning.

The assumption behind interactive learning environments is that, learning happens by interacting with information, tools and materials as well as by collaborating with other learners. The author states that computer / video games fundamentally have these features and MMORPGs (such as World of Warcraft, ToonTown) more so; because MMORPGs are persistent, networked, interactive narrative environments where players collaborate, strategize, plan and interact with objects, resources and other players.

Character Design:

By having the freedom to choose a character and determine its specific traits and characters, the author argues (by referencing several research studies) that it is essentially a form of role playing in a simulated environment. To quote a 1995 study on Multi-User Dungeons - predecessor to MMORPGs "virtual environments allow users to experiment in a safe, non-threatening environment and to expand, explore and reflect on different aspects of themselves".

The players also have an emotional connection to their characters as they invest significant amounts of time and make critical choices in developing their character (by the addition of skills, capabilities or adornments) which is in a sense, the player taking on a particular role.

Narrative environment:

The narrative environment in a MMORPG fosters exploration. Players continually explore various regions to find quest givers, characters, objects and tools related to completing small quests which can help them advance in the game. Players often make critical choices about the most economical way to complete a variety of small quests. The author goes on to list the various types of small quests ( such as "Collection quests", "Bounty quests") and how they aid in acquiring knowledge and learning.

To illustrate:

Research has identified and characterized four different types of knowledge:

- Declarative: Knowledge of facts, data and principles

- Procedural: Knowledge of how to perform a task / action / process

- Strategic: Awareness of applying knowledge, principles and experiences to various situations

- Metacognitive: Reflection and regulation of one's thinking during an activity

The author explains that a bounty quest for instance - which requires the player / players to defeat a character / characters are often challenging and requires players to analyze their character's strengths and weaknesses and to balance those against the environmental factors they may encounter - helps in gaining "strategic knowledge".

In a bounty quest the player continuously applies the knowledge gained from exploration / interaction or experience from past quests to succeed in this new situation thus enhancing strategic knowledge.

The author goes on to explain in detail the various types of knowledge and their equivalent quest types in her article which, can be accessed here.

Comments:

This article is well researched and does make a solid case for using MMORPGs as a tool for learning. The way the author has classified the different types of quests and mapped them to knowledge domains is particularly interesting and does corroborate personal experiences with such games.

Apart from all this, MMORPGs as a tool for learning would certainly be more fun.

Teaching Smart People How to Learn

2010-05-04T05:44:00.003-04:00

In this article, the author Chris Argyris puts it simply, "because many professionals are almost always successful at what they do, they rarely experience failure. And because they have rarely failed, they have never learned how to learn from failure." He basically says that when these individuals are wrong, they become defensive, screen out criticism, and blame anyone and everyone but themselves. In short, their ability to learn shuts down precisely at the moment they need it the most.

Getting people to learn is largely a matter of motivation. When people have the right attitudes and commitment, learning automatically follows. So companies focus on creating new organizational structures—compensation programs, performance reviews, corporate cultures, and the like—that are designed to create motivated and committed employees.

For 15 years, the author of this article studied professionals and their learning habits. He determined that as long as efforts at learning and change focused on external organizational factors - such as job redesign, compensation programs, performance review, and leadership training—the professionals were enthusiastically learning. However, as soon as the focus turned to the professional's own performance, their motivation and commitment to excellence was gone. Argyris determined that this occurred because the professionals were threatened by the prospect of critically examining their own role in the organization, making them feel vulnerable. This resulted in defensive reasoning.

To counter defensive reasoning, Argyris suggests the professionals need to learn how to reason productively. People can be taught how to recognize the reasoning they use when they design and implement their actions. They can begin to identify the inconsistencies between their perceived and actual theories of action. Once they have an understanding of what they do and how they do it, the professionals will learn more efficiently and effectively.

Source: http://pds8.egloos.com/pds/200805/20/87/chris_argyris_learning.pdf

The effects of active learning on students' memories for course content

2010-05-04T01:08:00.003-04:00

This article published in Active Learning in Higher Education determines active learning techniques and their relation to the course content retained by students in a higher education setting. Two studies performed by the author identified that students cited memory of activities which actively engaged them and forced them to reflect upon concepts.

Study 1
A group of 250 undergraduate students enrolled in three different courses at private Midwestern liberal arts college were provided with a survey on the last day of class. Participants were enrolled in introductory courses. The participants were instructed to write down ten things they remembered from the course. The survey instructed the participants to freely report anything that they remembered. The survey was left anonymous. The participant responses were coded for their level of understanding and the frequencies were summarized. Results showed that the videos and in class active exercises were most cited in the participant free responses.

Study 2
A group of 64 undergraduate students enrolled in three different courses at private Midwestern liberal arts college were provided with a survey on the last day of class. The participants were enrolled in advanced courses. The participants were instructed to write down ten things they remembered from the course. The survey instructed the participants to freely report anything that they remembered. The survey was not anonymous. The participant responses were coded for their level of understanding and the frequencies were summarized. Results showed that the videos and in class active exercises were most cited in the participant free responses just like those found in the introductory courses.

Source
The effects of active learning on students' memories for course content
Cherney, Isabelle D.
Active Learning in Higher Education, Jul 2008; vol. 9: pp. 152-171

Cultural Orientations and Collaborative Learning

2010-05-04T01:03:00.006-04:00

Summary:
Collaborative learning is learning in which student input is sought during instruction. The use of class discussions and group projects is an example of this type of learning. Recent literature suggests that collaborative learning has advantages over a more traditional lecture presentation. The authors of this article explored whether collaborative learning was superior to traditional methods. They also wanted to know if collaborative learning is equally effective across students from all cultural learning orientations and across all learning styles. The authors defined cultural orientation as being either collectivist, or focused on the group, or individualistic with an emphais on the needs of the individual.

Method:
The authors gave a survey to 100 students enrolled in an introductory communication theory course during the end of the fall and spring terms at a university in Western NY. The courses were taught using a mix of 75% traditional lectures and 25% collaborative learning. The survey assessed student's cultural orientations, learning styles, and their reported level of satisfaction with the collaborative learning exercises. Experimentors also compared student grades on tests from the sections taught via collaboration to those from sections taught in a lecture format.

Results:
The results indicated that collaborative learning was more effective than lecture methods for students regardless of learning style or cultural orientation. However, the more individualistic students reported lower satisfaction levels with collaboration than their more collectivist peers. As lower student satisfaction resulted in lower group grades, it is important to try to improve class members' satisfaction during the learning process. The authors suggest ways to make collaborative learning more palatable to individulatistic participants by grouping students based on those that share the same cultural orientation, giving clear guidlines, and by assigning individual grades to each member of a group.

Source:

http://www.findarticles.com/p/articles/mi_hb3325/is_3_10/ai_n29308482

Learning to Learn

2010-05-04T00:40:00.004-04:00

This paper, written by professors Karl R. Wirth and Dexter Perkins, focuses on significant learning and its applications to learning in higher education and , more specifically, its applications to the field of geology.

In addition to being able to recall information and ideas, one also needs to be able to apply one's knowledge or skills to new situations; this is application. This includes learning to engage in new kinds of thinking (critical, creative, practical) as well as certain skills (communication, playing an instrument, etc.). The learning categories here are: foundational knowlege, application, integration, human dimension, caring, and learning how to learn.

Effective Learning and Learning Styles
The best learning occurs when students are engaged in active learning. Cooperative learning is an important component of active learning. The key step to effective learning is identifying your learning style. After that, you can improve learning by translating material from other modes into a mode that best fits you. Since there are many different ways of modeling learning, it is important to be aware of your own learning style preferences so that you can make the necessary adjustments to maximize your learning.

Strengths

Aids in becoming an intentional learner - an integrative thinker who sees connections in seemingly disparate information to inform his/her decisions.
Aids in becoming a self-directed learner - someone who is highly motivated, independent, and strives towards self-direction and autonomy
Active learning improves student knowledge retention, conceptual understanding, engagement, and attitudes about learning.
Cooperative learning improves individual achievement, metacognitive thought, willingness to assume difficult tasks, persistence, motivation, and transfer of learning to new situations.
Learning improves brain function

Challenges

Becoming/being an intentional learner takes deliberate effort and continual reflection.

More Effective Learning - A Simple Technique

2010-05-03T22:26:00.003-04:00

Learning

Summary:
Steve Gillman, in an article on Buzzle.com (Intelligent Life on the Web), writes how current methods of learning maybe aren't the most effective ways to learn. The first method to improving learning is by teaching. Gillman writes that by teaching others, we grasp a better understanding of something ourselves.

Strengths:
Helps improve your learning of a topic
Weaknesses:
It may be tough to find people willing to listen/learn
It takes time to learn about a topic, and then teach it to others

Gillman then discusses how "vividly imagining" yourself teaching can improve your learning even more. By first teaching a friend about a topic you are trying to learn, you can gain a better understanding of the topic. If you then vividly imagine yourself teaching the topic you are learning about, you will become better at learning the topic.
Strengths:
Your vivid teaching lessons go faster than real ones and have less interruptions.

Link:
http://www.buzzle.com/articles/more-effective-learning-simple-technique.html

Behavioral and Psychosocial Considerations in Intelligence Analysis: A Preliminary review of Literature on Critical Thinking Skills

2010-05-03T18:22:00.002-04:00

This article is a review of published literature in the fields of psychology, education, and intelligence analysis to look for any information related to the link between critical thinking skills and an improvement in intelligence analysis. From the review, the 2nd LT Chin Ki Tam found that as of October 2008 there was very little information related to the link between critical thinking and the improvement of intelligence analysis. The problem begins, similar to intelligence itself, with a lack of one true definition of critical thinking.

Methods
In response to the tasking from the Air Combat Command that came of a January 2008 meeting at Langley Air Force Base, the 711 Human Performance Wing Continuous Learning Branch looked into existing literature in psychology, education, and intelligence. The goal was to combine the information to provide a picture of whether the teaching of critical thinking truly improves intelligence analysis.

Results
The results of this literature review show that there is no consensus on the definition of critical thinking nor is there a consensus on what exactly is a “critical thinking skill” and which skills a person should utilize to conduct critical thinking.
The results of a Delphi study conducted in Feb. 1988 to November 1989 show that six skills are necessary for critical thinking. They are: Interpretation, Analysis, Evaluation, Inference, Explanation, and Self-Regulation.
Beyond all the literature related to what exactly critical thinking is, only a few people have written about how critical thinking and intelligence analysis correlate. One of these authors is Dick Heuer in his book, Psychology of Intelligence Analysis. He writes about cognitive bias. He defines these biases as: selectivity bias, availability bias, absence of evidence bias, confirmation bias, overconfidence bias, oversensitivity to consistency bias, and discredited evidence bias as the biases that lead to flawed intelligence analysis.
In addition to all the information related to what critical thinking is and how it relates to intelligence analysis there is another problem of how people learn. The article talks about how people have different learning styles and that critical thinking in one domain in no necessarily transferable to another domain. Without tackling the problems associated with learning it is difficult to apply such thinking techniques to intelligence analysis.

Correlation and Current Intelligence Curriculum
Research has yet to be conducted to show whether or not critical thinking truly improves intelligence analysis however different agencies within the US Government have begun to institute programs designed to help with the development of critical thinking skills. The CIA, NSA, and DIA have all begun a training program for analysts designed to improve critical thinking skills. The most novel of which is DIA working to utilize gaming as a way to improve intelligence analysis however they only invested $2.6 million for the development of three games. This drastically limits the ability to be truly high quality learning tools when one considers that the average XBOX game costs between $3 million - $5 million to produce from start to finish and the average XBOX 360 or PS3 game costs around $10 million to produce. (Edwards, 2006)

Discussion
Although the intelligence community has invested heavily in the improvement of cognitive thinking skills, there is no evidence that directly links the improvement in these skills to the improvement of intelligence analysis. It would be a great help in the future if studies were designed to show whether there is an improvement. These studies would go a long way in justifying more or less money being put into the budgets of such training programs for future analysts.

Source:
Edwards, R. (2006, May 5). The economics of game publishing. Retrieved May 3, 2010, from IGN website: http://games.ign.com/articles/708/708972p1.html

Learning Techniques

2010-05-03T17:45:00.003-04:00

A lot of work has been done over the past few decades about how people learn. This document suggests a list of techniques that may make learning more effective.

Principles

You can learn anything if you have a goal that requires it.
Learning how to learn is the core skill.
Anyone can learn faster by structuring the information.
Intelligence is not fixed.
Knowledge and skills overcome obstacles.
Everything to which you were paying attention, either consciously or unconsciously , will be remembered permanently.

There are a number of stages to learn:

The Right States of Mind

There are six aspects for this:

Find a personal reason to want to learn this materials.
Translate these reasons into motivation
Find a way to make the materials relevant to you
Have positive expectation
Have a calm mind

A Variety of Ways of Inputs

Play to your strengths in term of how you process information.
Make a general outline of what you are learning.
Ask what do I already know about this.
Break the materials into small chucks.
Summarise the material out loud

Exploring from different Angles

There are 7 type of intelligences:

Linguistic intelligence-Describe the materials out lout.
Logical-mathematical intelligence-Use a flowchart or diagram.
Spatial intelligence-Make an image of the material.
Musicale intelligence-Play background intelligence as you learn.
Interpersonal intelligence-Teach someone else.
Intrapersonal intelligence-ruminate on the material.
Bodily intelligence-Use index cards.

Memorising

Decide to remember, take regular breaks, Review notes regularly, etc.

Showing you Know

Demonstrating to yourself that you really do understand and remember increases your learning.

Reviewing and Reflecting in the Process

After every learning session, review the process you follow.

Implicit sequence learning in a search task

2010-05-03T11:18:00.001-04:00

The purpose of this study was to see how the selection process is affected by task demands; that everyday skills rely on implicit sequence learning where learning happens unintentionally and the effects are beyond the learners’ ability to report on exactly what they learned. The researchers’ intent was to investigate whether learning about a sequence of stimuli appearing as part of a complex scene can be different from responding to a single target.

Method

Participants were presented with eight digits distributed pseudo-randomly over 16 possible locations arranged on an invisible 4X4 matrix. The targets followed a sequence that learners could exploit to improve performance. The researchers compared sequence learning under this seemingly demanding task with two different control conditions in which the perceptual load was lower. In a first control group, perceptual load was decreased by removing the distracters, but the target appeared at the same variable locations, in order to analyze whether the spatial uncertainty could produce any effect independent from perceptual load.

In the second control group, the researchers removed both factors by presenting the targets alone at the center of the screen. Because the search conditions were expected to hinder the achievement of explicit learning, they arranged a known sequence rather than a probabilistic procedure.

Participants

A total of 90 students participated in the experiment and a total of 30 participants were assigned to each condition: central, variable location, and search.

The stimuli consisted of a set of colored digits over a grey background. Target stimuli were even numbers presented in random colors. In the central condition, the target was presented alone at the center of the screen. In the search condition, the target appeared on each trial at one of the 16 locations defined by an invisible 4X4matrix. The location of the target was decided pseudo-randomly on each trial, with the only constraint that all 16 possible locations should be sampled before any of them was repeated. The distracters were seven instances of the same odd number, randomly chosen for each trial from the numbers 1, 3, 5, 7. The seven distracters plus the target stimulus were colored and located pseudo-randomly, so that two of them were drawn in each possible color and two items were located at each one of the matrix quadrants. At the end of the experimental blocks, participants performed a cued generation task, designed to measure their ability to predict the more likely successor.

Results

The study showed that implicit sequence learning does not depend on the resources demanded by the selection task. The results indicate that implicit sequence learning is resistant to selection demands, and that presenting the targets at variable locations can be useful to control for the acquisition of explicit learning.

The full results of the experiment can be accessed through Academic Search Complete: Implicit sequence learning in a search task by Luis Jime´nez and Gustavo A. Va´zquez. The Quarterly Journal of Experimental Psychology, 2008.

Inductive Learning: Effective Media Training that Isn’t Boring

2010-05-02T21:13:00.002-04:00

Found at: http://www.mindspace.net/about_us/in_the_news/PRNews-06162009.pdf

Inductive Learning: Effective Media Training that Isn’t Boring
By: Brent Diggins
PR Account Executive | Mindspace

Inductive learning, the process of learning through first-hand experience, is essential to producing an ingrained media training session that is fun for participants. Compared with deductive learning,learning through instructor “telling,” inductive learning has been proven in many academic studies to be a more effective teaching method, and is especially effective in short-term learning situations.Perhaps the best attribute of inductive learning is that the participation can be astoundingly fun,and everyone will appreciate that.

Benefits:

1) Practical—Today’s fast paced people
don’t want to be lectured to. They want to
actively participate and be hands-on. As
people are participating in a variety of simulated
exercises, they will be able to walk out
of the room and right into an interview.

2) Shared Learning—Unlike deductive
learning, an inductive media relations
program gives participants the opportunity
to learn from watching, critiquing and
analyzing others. Watching video of good,
bad and mixed-bag interviews under the
tutelage of a media trainer gives participants
the perspective of an interview they would
not have otherwise.

3) Fun—Actively participating in inductive
learning options like role playing, mock press
conferences and journalist roles makes media
training fun for participants. Further, putting
together an inductive program is creative,
fun and without boundaries.

Weaknesses:

1.)Can be time consuming
2.)Can potentially lead the audience astray from the topic if not properly monitored by an administrator.

Inductive Learning Techniques:
The best way to showcase a solid inductive learning-based
media training program is to combine your end-goals and media training rule-of-thumbs with your creativity. Here are three great inductive learning techniques and methods:

1) Role Play and Scenarios
2) Watch Video and Critique
3) Hold a Mock Pres Conference

Comments:
This article was written by a Public Relations Account Executive with a teacher or manager in mind. It gave techniques that one could use if there was an audience that needed to be kept engaged in the subject they were being presented, so that the audience could learn more effectively.

Green Team Summary of Findings: Role-playing (4 out of 5 stars)

2010-04-29T13:55:00.004-04:00

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 29 April 2010 regarding Role-playing specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description

Role-play is used as a training method to acquire knowledge, attitudes, and skills in a range of disciplines and with learners of different ages. Role-play in particular is a type of simulation that focuses attention on the interaction of people with one another and emphasizes the functions performed by different people under various circumstances. Role-play can be fully scripted or partially scripted. Players can rotate through roles or new information can be added into the role-play.

Role-playing has been used and continues to be employed as a tool to facilitate training in vocational and interpersonal skills and as a method of active learning in educational and commercial settings. Though there is no single type of role-played exercise, the term is best applied where the teaching and learning experience rests with a form of `as-if’ experimentation. Role playing, is most often used in a classroom and requires physical involvement on the part of students.

Strengths:

Can identify critical issues in predeveloped plans
Enables active critical thinking to solve encountered problems
Can be tailored to fit a specific scale
Engages participants effectively
Allowed for development of critical skills in a controlled environment
Enables critical feedback from supervisor
Provides an engaging learning environment.
An opportunity for active learning in a safe, low-risk environment
Develop a range of transferable skills, qualities and competencies
Rapidly and effectively improve the communication skills of trainees
Participants have a better overview of problems and can foresee what possible solutions exist and why. Furthermore, participants become more involved in the training
Allows participants to experiment with various strategies without real consequences, allowing them to express their opinion freely
Enhances self-confidence of participants
The fictitious nature of the interaction reduces the psychological stress of a situation which allows shyer people to participate more fully than they might feel free to do in real life.
Gives students a better understanding of the decision-making process
Can be applied to various situations**
Good way to investigate the consequences of decisions or designs with little risk or cost
Captures cultural aspects well in designs and decisions
The activity simulates the emotions of an actual event which engages participants in the process.
Encourages creative solutions
Increases insight into another person's point of view
Analyzes problems from various perspectives
Implements brainstorming methodology in simulations of real cases
Develops team-work, co-operation and creative problem solving in groups

Weaknesses:

Effectiveness can be hampered by individuals not taking it seriously
The absence of training in role-play disciplines can lead to 'under-play,’ where the participant’s inability hinders role development, or to 'over-play,’ where the participant exaggerates or caricatures their part
Inability of some role-players to effectively participate due to anxiety or dislike of a given role
Participants may become apathetic to role play exercises over time
Puts individuals on the spot, which might lead participants to resist
Increased possibility of risk to personnel in some scenarios
Role play exercises are challenging to devise
May require a large amount of resources
Can be difficult to manage or facilitate
Emotionally and intellectually demanding
Does not allow for much detail in the information being studied or all of the variables involved
Lack of realism in roles, setting, or task
Cannot account for all real world issues

How To:

There are many ways in which to conduct a role-plaing exercise depending on the desired outcome. However, from the research several consistent trends emerge. They are as follows:

1.) Begin with a historical background and basic understanding of the concepts/topics at hand.

2)Make sure that the participants are committed to the process.

3.) Determine the teams/groups

4.) Define the roles of the members of the group

5.) Actively role play responding spontaneously

6.) Discuss results and give feedback

7.) If needed, repeat taking different courses of action

Personal Application:

Each member of the class was randomly assigned a role ranging from a four year old child to a 75 year old all with very different "roles" such a doctor, a professor, a mechanic, a zooligist, and an athlete. The context was that the group was on a sinking ship and a lifeboat was available with eight spots. However, there were twelve people so each person role-played to decide who should be saved and who should not. During the fifteen minutes of the exercise, each person particpated in deciding the fate of the members and of themselves by the role they were given.

Summary of Findings (White Team): Role Playing (4 out of 5 Stars)

2010-04-29T13:44:00.003-04:00

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 29 April 2010 regarding Role Playing specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

The methodology is a recognized and successful tool in behavioral assessment, which encourages the application of the technique. In crisis negotiation role playing, which was done in today's exercise, participants used communication skills and reasoning to resolve crisis situations. Results from Role Playing are qualitative results that allow a detailed view of the particular scenario and possible outcomes. The White Team recognizes this technique as a 'method' as opposed to a 'modifier'.

Strengths:

Allows for development of ideas and thoughts in a controlled environment.
Provides an opportunity for active learning in a safe, low-risk environment.
Allows for the application to multiple different types of situations or persons.
Can be easily conducted as a low cost exercise.
Simulates the emotions of an actual event which engages participants in the process.

Weaknesses:

Participants may not take the exercise seriously.
Participants may act as a caricature of the person they area assigned, rather than respond as the person.
Participants may have a use different decision-making processes due to their social upbringing.
A false confidence in their understanding of the role they are playing may result in participant overconfidence in their predictive ability of the person's action.
The situation in which the participants are placed may be missing critical elements of reality present in emergency situations.

How To:

Begin with a historical background and basic understanding of the topics / issues that is the subject of the role play
Discuss the problem to be portrayed
Determine the teams/groups
Define the roles of the members of the group
Make sure that the participants are committed to the process.
Act out the situation, responding spontaneously.
Evaluate the actions of each player.
Repeat, taking different courses of action if needed.
Discuss results and feedback

Application:

Students were presented with a scenario where there was a sinking ship and only 10 people could fit on the life boat. Each student picked a role to play out of a hat and introduced themselves to the group. After all the roles were announced each made their case as to why they should stay on the boat. Some participants voluntarily requested to remain on the sinking ship where others pleaded their individual cases to secure a spot on the life boat. Through this discussion it allowed different conceptual models to emerge that influenced the way decisions were made. We also came to the understanding that in order to role play it is important that all participants take their part seriously as it influences the outcome of the exercise. It is also important to note that value assumptions,mirror imaging and cultural biases are likely to manifest and it is vital to the success of the exercise to mitigate those factors.

Role-Play Training for Negotiators in Diverse Environments

2010-04-27T10:02:00.003-04:00

Introduction

The author of this article opens up the discussion by citing facts of the ever increasing stakes for the hostage negotiator in today's complex world. In the wake of the Columbine shootings and the DC sniper events, hostage negotiators must be ever vigilant and better trained. Furthermore, the author states that "By using locations previously unfamiliar to team members and tactical units, commanders may decrease the anxiety associated with a particular environment while also role-playing in a realistic setting".

One Agency's Experience

The majority of the article focuses on one police department in Suffolk County, New York (located on Long Island). The Suffolk County Police Department (SCPD) "is training in a variety of locations and venues to understand and respond more quickly and effectively to a hostage or barricade incident". The SCPD notes that two of the most important factors for carrying out a successful hostage role-play are the planning involved and the chosen location.

In regards to planning, the SCPD says that by consulting with officials at the proposed site, Hostage Negotiation Team (HNT) supervisors can design and implement realistic role-play scenarios based on the unique threat environment. "Role-play scripts and scenarios are designed to provide the best and most realistic training within the limitations of time and personnel available."

Lastly in terms of location, the HNT has sought out sites that have the potential to be hostage situation sites. Some of the sites that the team has trained at include a bank slated for demolition, a local high school (during the summer months), the Suffolk County jail and correctional center, the medical center at the local VA, a university dormitory and a municipal park. All of the role-plays at each of these locations gave the negotiators an idea of how such events could transpire and what the difficulties would be in dealing with each specific situation.

Source
For any further information and for the results of the study, please see "Role Play Training for Negotiators in Diverse Environments" in the FBI Law Enforcement Bulletin, written by James R Maher accessed through Academic Search Complete.

Role-Play For Medical Students Learning About Communication: Guidlines For Maximising Benefits

2010-04-27T04:58:00.003-04:00

Debra Nestel and Tanya Tierney, the authors of this article, state that role-play is widely used as an educational method for learning about communication in medical education. Despite the wide use of role-play as a teaching method, there is little published evidence about its effectiveness. This paper explores students' experiences in role-play with the intent of producing guidelines for maximizing the benefits of this learning method.

Method
As part of their communication class, first-year undergraduate medical students participated in a role-play session. Both before and after the session, students were asked to complete questionnaires that asked about their experiences with role-play. Immediately following the role-playing exercise, the students answered similar questions in relation to the role-play activity they had just completed. Their answers were then analyzed.

Results
284 students completed the questionnaires. The results indicated that 63 (22.2%) had prior unhelpful experiences with role-play. However, most students (274; 96.5%) found this experience helpful. The students reported that this activity was helpful because it produced opportunities for observation, rehearsal and discussion, realistic roles and alignment of roles with other aspects of the curriculum. The students also reported that some of the unhelpful aspects of the exercise were factors that contributed to a lack of realism.

Conclusion
This study concluded that role-play was valued by students and that it helped them acquire better communication skills. Finally, the authors' proposed guidelines for effective role-play include adequate preparation, alignment of roles and tasks with level of practice, structured feedback guidelines, and acknowledgment of the importance of social interactions for learning.

Source: http://www.biomedcentral.com/content/pdf/1472-6920-7-3.pdf

Joint Civilian/National Guard Mass Casualty Exercise Provides Model for Preparedness Training

2010-04-27T01:04:00.004-04:00

Following criticism of civilian and military response to hurricanes Katrina and Rita, members of the New York Air National Guard and the Central New York Medical Reserve Corps developed plans for a full scale exercise to determine their roles in a mass casualty situation. The Exercise was developed between October 2005 and March 2006. The event was held over two days, with the first day providing a series of partial drills for each section of the groups activities like triage. On the second day a full scale exercise including 350 individuals from over 25 agencies took place. The simulated mass casualty drill involved 32 victims contaminated in a hydrogen fluoride explosion. Following the drill three evaluation sessions were given; one by first responders, one by medical personnel, and a third by military personnel. The exercise was able to identify critical issues in joint military/civilian operations, and provide possible solutions to a number of the identified problems.

Strengths:
-Can identify critical issues in predeveloped plans
-Enables active critical thinking to solve encountered problems
-Can be tailored to fit a specific scale
-Engages participants effectively

Weaknesses:
-Effectiveness can be hampered by personnel not taking it seriously
-Increased possibility of risk to personnel in some scenarios
-May require a large amount of resources
-Requires extensive preparation

Source:
Grant, W., & Secreti, L. (2007). Joint Civilian/National Guard Mass Casualty Exercise Provides Model for Preparedness Training. Military Medicine, 172(8), 806-811. Retrieved from Military & Government Collection database.

The Korean War: A Role-Play to Remember

2010-04-27T00:13:00.003-04:00

Introduction

This article is written from a teacher's point of view with an educational context in which the class participates in a role play of the Korean War. The author first discusses the importance of the Korean War and talks about how it is one of the most overlooked American wars to date. She then points out the benefits of using role playing and elaborates on the technique. The author says that "role play generates student interest in the topic and allows students to physically, as well as mentally participate in their own learning".

The Role-Play Itself

Prior to engaging in the role-play, the author recommends stimulating a discussion on the concept of "war". In preparation for the lesson, the teacher should cut out and mount the script cards (found in the appendix of the article), create the role-play roles (U.N., U.S., North Korea, South Korea, etc.) and using masking tape outline the Korean peninsula on the floor of the classroom. The role-play parts that the students will be re-enacting include some historical figures such as Stalin, a KGB agent, France, General MacArthur and the Yalu River to name a few examples. The author recommends that the day of the activity, you as the teacher administer a pre-test to see what the students know about the Korean War. To actually accomplish the role-play the students should follow the directions on the cards but should not break any of the following rules:

1) Do not trade roles; roles are distributed randomly.
2) Attach your role to your chest so it can be seen.
3) When called to particpate read the card that you are given.
4) Read loudly enough so everyone can hear.

Following the role-play the author recommends administering a post-test to see what the students have learned. Furthermore, she advises to conduct a class discussion to see what they thought about the role-play experience and to see what they learned!

Source

For any further information and for the results of the study, please see "The Korean War: A Role-Play to Remember" by Marjori M Krebs accessed through Academic Search Complete.

DECISIONS, DECISIONS! A Genetics Role-playing Activity

2010-04-27T00:00:00.003-04:00

Role Playing

Summary:
Sharon Nelson of Waunakee High School in Wisconsin published a role playing scenario she used for her high school students. The scenario is about the "3 P's of science", as Nelson writes; "problem posing, problem solving, and peer persuasion." She briefly discusses the purpose of the role play, and the procedure in which to do it.
In the procedure she says "students will be assigned to work in groups of 2-5, at the instructor's discretion. Students will then choose a scenario for role playing. Their job is to research any and all information pertinent to their situation. Students may divide the roles in any way they see fit (one may choose to assume the role of parent, another the role of doctor, etc.) Once they have the necessary information, they must make some decisions regarding their situation, based on the bioethical issues involved. In addition to a 5-10 minute oral presentation, students must, with their group members, turn in one written report."

She then lists the possible scenarios the students can choose from. All the scenarios are relatively short and to the point. An example of one is...
You have hemophilia; you and your spouse would like to have children. You are referred to a genetic counselor.
Income: You just lost your job due to missing so many days of work for hospital stays. Wife's income as teacher's aide: $18,000.
Insurance: none

Weaknesses:
Very lengthy on preparation

Link
http://www.accessexcellence.org/AE/ATG/data/released/0350-SharonNelson/index.php

Role Playing: Applications in Hostage and Crisis Negotiation Skills Training

2010-04-26T23:53:00.002-04:00

This article provides an overview of various applications of the role playing are used by the FBI as part of their crisis negotiation training. The article states that role playing is a recognized and successful tool in behavioral assessment, which encourages the application of the technique. In crisis negotiation role playing, participants develop critical active listening skills commonly used to resolve crisis situations. The article highlights a number of role playing tests and activities performed by the FBI. Additionally it highlights a possible future trend in digital applications to crisis negotiation. Validation for the method is limited to studies indicating increased abilities among subject who had undergone extensive role playing.

Types of Role Playing Activities:
-Table top
-Functional
-Partial/Full scale

Strengths:
-Allowed for development of critical skills in a controlled environment
-Enabled critical feedback from supervisor
-provides an engaging learning environment.
-Identifies potential weaknesses

Limitations:
-Cannot account for all real world issues
-Requires a capable confederate force
-Limited field validation (in crisis negotiation)
-Require extensive resources for full scale exercise

Source:
Role Playing: Applications in Hostage and Crisis Negotiation Skills Training
Van Hasselt, Vincent B., Romano, Stephen J., Vecchi, Gregory M.
Behav Modif, Mar 2008; vol. 32: pp. 248-263

Pre-managerial Business Education: a role for role-plays?

2010-04-26T13:10:00.002-04:00

This article looks at the value of role playing into an undergraduate business curriculum. The author contends that those with limited or zero management experience can benefit greatly from a unique perspective on management challenges and from simulating aspects of interaction and decision-making in organizational settings.

Role playing defined
Role-playing has been used and continues to be employed as a tool to facilitate
training in vocational and interpersonal skills and as a method of active learning in
educational and commercial settings. Though there is no single type of role-played
exercise, the term is best applied where the teaching and learning experience rests
with a form of `as-if’ experimentation.

Strengths
• A opportunity for active learning in a safe, low-risk environment
• Can be applied to many different vocations
• Advance subject-based learning
• Develop a range of transferable skills, qualities and competencies

Weaknesses
• Emotionally and intellectually demanding
• The absence of training in role-play disciplines can lead to `under-play’, where the participant’s inability hinders role development, or to `over-play’, where the participant exaggerates or caricatures their part
• Inability of some role-players to effectively participate due to anxiety or dislike of a given role
• Can be difficult to manage or facilitate

How-To (as applied to the business curriculum for this study)
1.) Begin with a historical background and basic understanding of the business concepts/topics at hand.
2.) Determine the teams/groups
3.) Define the roles of the members of the group
4.) Establish the topic to role play
5.) Actively role play
6.) Discuss results and give feedback

This article was accessed through Academic Search Complete through Mercyhurst College libraries: "Pre-managerial Business Education: a role for role-plays?" By Simon A. Mercado

Effectiveness Of Role Playing, Case Studies, and Simulation Games in Teaching Agricultural Economics

2010-04-26T11:55:00.001-04:00

This study assessed the impact of role playing, case studies, and simulations for teaching at the undergraduate level. Its purpose was to show how effective the tools were at improving the student’s level of understanding for certain concepts.

Objectives and Methodology

The educational impact model was used in a qualitative evaluation of the content and the level of effectiveness for role playing, case studies, and simulations. The study was conducted during two classes with one professor teaching the same course. One course was taught using only lecture and the other class was taught using the three different aids. The study also used an average of student test scores as an indicator of the level of effectiveness and student performance. Both classes were given the same test.

Role Playing

Role playing, as most often used in a classroom, requires physical involvement on the part of students. Two or more people "act out" the part of individuals in a hypothetical situation. For the purpose of the class, the following situation was used:

Students were presented with the subject of “price determination; a role-playing exercise patterned around the open-outcry market typical of commodity futures markets.”

1.) Sixteen students participated in three successive auctions, some acting as farmers and others acting as wholesalers.

2.) Information concerning quantities to be bought or sold and profit/cost levels per pound was provided on index cards given to each participant by the instructor.

3.) During each three-minute auction the students had to negotiate transaction prices with one another.

4.) After the third auction, the three sets of resulting prices were analyzed by the entire class.

5.) After the instructor told the class which set of prices were generated in an equilibrium, surplus, and shortage situation the students could see that the theory presented in their textbooks did, in fact, describe what happened in their exercise.

Results for the Effectiveness of Role Playing

The results for the experiment concerning use of a role-playing exercise provide some insight into the impact. Role playing had only a small direct effect on exam scores; only a 3 percent increase on scores. However, students made written comments on how role playing raised the student’s level of interest. 73 percent of the participants noted that the exercise was one of the most favorable parts of the course.

Strengths

· Gives students a better understanding of the decision-making process

· Can be applied to various situations

Weakness

· Does not allow for much detail in the information being studied or all of the variables involved

Comments

For results of the whole study including case studies and simulations, please see “Effectiveness Of Role Playing, Case Studies, and Simulation Games in Teaching Agricultural Economics” by Steven Blank accessed through Academic Search Complete.

Investigating the Use of Role Play Training to Improve the Communication Skills of IS Professionals: Some Empirical Evidence

2010-04-25T19:10:00.002-04:00

Mark Frolick, the author of this article from The Journal of Computer Information Systems, looks at the effectiveness of the role play exercise for the communication skill improvement of Information Systems (IS) professionals. Role play is an active learning technique that creates a training situation in which the interpersonal interactions and communication flow characteristics of the real world can be accurately reflected.

The study consisted of 93 role play exercises aiming to improve the two dimensions of communication skills, content and process related skills, which were conducted among 92 graduate students enrolled in systems analysis and design courses between 1998 and 2000. It looked at two questions: How fast can role play exercises generate measurable results? and Does the incremental improvement in communication skills between role play exercises decrease over time? Each group was given 5 minutes to prepare a scenario, which was then acted out. Fellow students acted as judges and rated the participants in terms of effective communication in both content and process. Each student participated in 3 role-playing exercises about 4 weeks apart.

Results
The analysis of the data suggests role play is a viable training method that can yield measurable results of communication skill improvement. The content and process related skills of the participants improved consistently over the three role play exercises on average. Incremental improvement decreases as more role play exercises were conducted, which is consistent with learning curve theory.

Strengths

rapidly and effectively improve the communication skills of trainees
participants become more involved in the training
participants have a better overview of problems and can foresee what possible solutions exist and why
allows participants to experiment with various strategies without real consequences, allowing them to express their opinion freely
enhance self-confidence of participants

Weaknesses

participants may become apathetic to role play exercises over time
role play exercises are challenging to devise
puts individuals on the spot, which might lead participants to resist

Source:

http://www.allbusiness.com/human-resources/workforce-management/1126451-1.html

Role-play for Medical Students Learning About Communication: Guidelines for Maximizing Benefits

2010-04-25T18:08:00.002-04:00

This article, from BioMed Central Medical Education, evalutes the use of role-playing as an educaitonal method for medical students. Authors Debra Nestel and Tayna Tierney focus on role-playing by 284 first-year medical students at Imperial College's communications program, which divides students into groups of 30 to learn specific skills associated with medical interviewing. Students take the role of interviewer, patient, or observer and have 5 minutes preparation, 5 minutes in role-play and 10 minutes in feedback. Before the sessions, students completed questionnaires about their prior experience with role-play, and afterwards, they completed evaluations about whether or not they thought role-play was helpful based on their new experience. Responses were both closed and open-ended.

According to the authors' research, role-play is used as a training method to acquire knowledge, attitudes, and skils in a range of disciplines and with learners of different ages. Role-play in particular is a type of simulaiotn that focuses attention on the interaction of people with one another and emphasizes the functions performed by different people under various circumstances. Role-play can be fully scripted or partially scripted. Players can rotate through roles or new information can be added into the role-play.

Results
Of the 284 students who completed forms, 199 had prior experience with role-play. 221 students reported role-play to be valuable for learning while the remainder reported it to be not valuable. After the role-play, 274 students reported that role-play had been helpful for leaning. Unhelpful experiences focused on emptional responses that impede learning, such as embarrasment, intimidation, or anxiety. Overall, role-play was reported to be an effective means of learning communication skills, desptite prior negative experiences with role-play.

Strengths

students gained insight into their own and others' behavior
increased understanding of certain issues
increased communication skills
players get immediate feedback

Weaknesses

players not taking role-play seriously
lack of realism in roles, setting, or tasks

Source:

http://www.biomedcentral.com/content/pdf/1472-6920-7-3.pdf

On The Move With A Magic Thing: Role Playing In Concept Design Of Mobile Services And Devices

2010-04-25T04:43:00.004-04:00

This study, conducted in 2000, evaluated the success of using role playing to design a mobile network for the campus of Helsinki University of Technology to investigate the nomadic Internet use of the future. The design team hoped to overcome three main difficulties of their task through role playing. The design team was planning for the use of a technology that did not exist at the time, so it was hard to know how it would be used. They needed a design that allowed for mobility, and they had to incorporate their users' lifestyles and cultures into their plan for the technology.

The designers used two methods of role playing. One of these is called participatory design technique, in which the designers asked a sample of their target audience to enact daily situations imagining using a technology in a variety of campus settings and while playing either themselves or different roles. They utilized props to make the situation seem more realistic. In contrast, the second type of role playing, referred to as Situated and Participative Enactment of Scenarios (SPES) had people carry a mock up of the anticipated product in their hand with them through their daily activities and imagine how they might use the product. A trained observer recorded the participant's thoughts. The study revealed the following strengths of role playing in general.

Strengths:

Engages users in a common cause
Good way to investigate the consequences of decisions or designs with little risk or cost
Captures cultural aspects well in designs and decisions

Results:

The study also made it clear that the participatory design technique in which people acted out fictional situations was better suited to designing services involving group interaction and dynamic incidents. In contrast, SPES was more useful for considering less dynamic activities such as listening to music, and it helped give designers a more detailed scenario because the recorder noted the context of a user's actions.

Source:

http://www.cs.hut.fi/~pmrg/publications/GO-PROD/2000/Lacucci_Kuutti_Ranta_OMMT.pdf

The Social Framework of the Role-Playing Process

2010-04-25T01:31:00.007-04:00

This article, written by Markus Montola, looks at the process of role-playing that takes place in different games, and the inconsistencies between the definitions of role-playing. In this article, Montola defines the role-playing mindset as a method of game playing, which can be optionally combined with various game systems. Role-play typically has no inherent endogenous goals at all. The rules of role-playing only provide the structure for the activity, but give no end condition or an objective.

Montola states that role-play, a social activity, always has three elements present: an imaginary game world, a power structure, and personified player characters. According to the article, the power structure of role-playing activities about imaginary people acting out in an imaginary environment is what differentiates it from children's play. Montola proposes three general rules that should always be followed in any type of role-playing:

1) Role-playing is an interactive process of defining and re-defining the state, properties and contents of an imaginary game world.
2) The power to define the game world is allocated to participants of the game. The participants recognize the existence of this power hierarchy.
3) Player-participants define the game world through personified character constructs, conforming to the state, properties and contents of the game world.

This article examines several forms of role-playing, primarily taking examples from tabletop role-playing, live-action role-playing, and virtual role-playing.

Source: http://journalofroleplaying.org/

Problem Solving and Conflict Resolution Role Playing

2010-04-25T01:29:00.005-04:00

The article describes the use of role playing to come up with constructive solutions to conflict and problems in a classroom setting. The technique entails assigning people to act out a conflict scenario in order to come up with the best solution to a problem. Role playing is often used in education to improve children's interpersonal skills and ability to handle difficult social situations. It is versatile and has applications for solving problems as simple as playground bullying to those as complex as international tensions.

Strengths:

The activity simulates the emotions of an actual event which engages participants in the process.
The fictitious nature of the interaction reduces the psychological stress of a situation which allows shyer people to participate more fully than they might feel free to do in real life.
Encourages creative solutions.
Increases insight into another person's point of view.

Weaknesses:

It is not a fruitful exercise unless the participants take it seriously.

How to:

Discuss the problem to be portrayed.
Make sure that the participants are committed to the process.
Assign roles.
Act out the situation, responding spontaneously.
Evaluate the actions of each player.
Repeat, taking different courses of action if needed.

Source:

http://www.classroomsthatwork.com/pdf/Roll%20Playing.pdf

Team Role Play

2010-04-24T13:16:00.004-04:00

Role Playing

Summary:
The author of this article on Team Role Playing, Jeffrey Baumgartner, writes that role playing "is when a group of people act out roles for a particular scenario." He goes on to describe a simple role playing scenario about a salesman and a customer. In essence, the two sales trainees play the two roles. Baumgartner mentions that other people may watch the scenario in order to better understand also.

The article continues with Baumgartner's scenario of salesman and customer being put to groups of trainees. There are specific rules that are laid out before each group and eventually each group must make a decision about how to best "play out" the scenario. The author mentions that when a team member is trying to help the "actor" that it should not be discouraged because it could possibly lead to better solutions. He sums up his discussion about his scenario by showing how each scenario can become more and more complex by adding more specifics, forcing the "actors" to think more about how to reach the best outcome for their team.

Strengths:

Analyzes problems from various perspectives
Implements brainstorming methodology in simulations of real cases
Tries various solutions in a case scenario
dDevelops team-work, co-operation and creative problem solving in groups
Exercises creative techniques in a risk-free environment

Link:
http://www.jpb.com/creative/roleplay.php

Green Team Summary of Findings: Delphi Analysis (3 out of 5 stars)

2010-04-22T13:49:00.001-04:00

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 22 April 2010 regarding Delphi Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

DESCRIPTION

The Delphi method is based on a structured process for collecting and distilling knowledge from a group of experts by means of a series of questionnaires combined with controlled opinion feed back. It was developed in the early 1950s, and mainly applied to technology forecasting, but also to many types of policy analysis.

Delphi focuses on researching the future or things about which little is known. It relies on the use of expert opinion, utilizes remote group processes, adopts an iterative research process, and creates a consensus of opinion.

There are three versions of the Delphi Technique:

Numeric – aims to specify a single or minimum range of numeric estimates through the use of summary statistics.

Policy - on the exploration, generation and definition of several alternatives and the arguments for and against each of these alternatives.

Historic - aims to explain the range of issues that fostered a specific decision, identification of several scenarios that could have led to the resolution of a past problem.

Delphi is particularly appropriate when decision-making is required in a political or emotional environment. The tool works formally or informally, in large or small contexts.

STRENGTHS AND WEAKNESSES

Strengths:

Its ability to ascertain expert opinions and potential policy options

Allows a group response without the attendant disadvantages sometimes experienced with group problem solving or decision-making.

Expert participants are more likely to generate reasoned, independent, and well-considered opinions in the absence of exposure to the "persuasively stated opinions of others". Because the experts do not ever participate in a face-to-face discussion, there is no danger of one or more individuals’ opinions being swayed by a more dominant or more experienced individual.

Efficiency and flexibility, especially in light of modern communication technologies such as e-mail and the Internet. Experts may be drawn from a wide geographic area, and the participants’ commitment in terms of time and money invested is minimal.

Delphi method has been shown to be an effective way to conduct research when the responses being sought are value judgments rather than factual information. Although it is more difficult to assess the "correctness" of value judgments, it is generally agreed upon that value judgments are not all equal but can in fact be more "right" or more "wrong."

Weaknesses:

A low level reliability of judgments among experts

Dependency of forecasts on the particular judges selected

Should not be used when any of the following three critical conditions are not present: adequate time, participant skill in written communication, and high participant motivation. It is estimated that a minimum of 45 days is required to carry out a Delphi study

A high degree of motivation is needed to offset the tendency for participant dropout as the study progresses. Because there is no direct contact between participants, those who are not highly motivated and interested in the subject at hand may feel isolated or detached from the process (It is easy for participants to drop out of the process)

There is a strong response of the group to conform with the statistical feedback of the panel, extreme points of view, which may provide new insights tend to be suppressed

The way the questionnaire and process is structured can lead to a bias and a Delphi study is at the mercy of the view and biases of the coordinating team, who choose respondents, interpret the information and structure the question

Participants must be knowledgeable and able to clearly communicate their ideas

Sensitivity of the results to ambiguity in the questionnaire that is used for

Difficulty in assessing the degree of expertise incorporated in the forecast

Good responses can be altered by those without a strong belief in the participants' answers

HOW TO

There are many ways in which to conduct a Delphi exercise depending on the desired outcome. However, from the research several consistent trends emerge. They are as follows:

A questionnaire is sent to experts.

Each expert gives his answers to the questions in an independent and anonymous way.

The responses of each expert are analyzed by the monitoring team. The moderator summarizes the responses to the first questionnaire and develops a feedback report along with the second set of questionnaires for the panelists

The set of responses is then sent back to experts and they are asked if they wish to revise the initial predictions.

The process is reiterated until a degree of consensus is reached by experts. Generally the number of rounds is determined in advance.

The moderator develops a final summary and feedback report to the group and decision makers

PERSONAL APPLICATION

We started with the a question: "Of the 21 first year graduate students, how many will graduate on May 21, 2011 having successfully completed their thesis or comprehensive exam?" Each person wrote out their answer without discussion and handed it in. While one person found the average, the facilitator introduced data regarding the thesis completion rate from 2005 to 2009. The average was written on the board and then a one minute discussion regarding the trend of incompletes. Then the group wrote out their next answer and another set of calculations were done. Another one minute discussion was introduced and then a third round of responses. A final discussion noted how/if the process changed answers due to the introduction of facts or consensus from the "experts."

Summary of Findings (White Team): Delphi Technique (3out of 5 Stars)

2010-04-22T13:41:00.002-04:00

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 22 April 2010 regarding Delphi Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Delphi is a method for a structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem. It was created in the 1950's by the Rand Corporation to increase efficiency and to bring together Subject Matter Experts to assess and create possible outcomes to a problem. It is a process conducted through the use of multiple rounds to introduce new ideas and come to a consensus on those new ideas along with existing ideas.

Strengths:

Its ability to ascertain expert opinions and potential policy options, including new alternatives.
Participants can be located all over the world.
Depending on the goal of the study, consensus on a forecast may be reached.
Anonymity improves participants ability to judge the presented options, not just presentation style.
Forecasts can be judged against the later reality of the situation.

Weaknesses:

Participants must trust the methodology and moderator or facilitator for this to work.
Moderator or facilitator should be objective.
Can lead to Groupthink.
Susceptible to the anchoring bias.

How To:

Pick a facilitation leader: The facilitator is an expert in research data collection, and is not a stakeholder.

Select a panel of experts: The panelist should have an intimate knowledge of the projects.

A questionnaire is sent to experts.

Each expert gives his / her answers to the questions in an independent and anonymous way.

The moderator summarizes the responses to the first questionnaire and develops a feedback report along with the second set of questionnaires for the panelists

panelists independently evaluate earlier responses and vote on the second questionnaire

The set of responses is then sent back to experts and they are asked if they wish to revise the initial predictions.

The process is reiterated until a degree of consensus is reached by experts.

Some variations to this very basic method include:

The number of iterations (the more rounds, the closer the consensus likely to be reached)
The method of selection and size of the panel
The scoring system and the rules used to aggregate the judgments of the panelists
The extent of anonymity afforded to the panelists
How consensus is defined and how disagreements are dealt with

Application:

We decided to apply the Delphi technique to expected completion of the Master of Science of Applied Intelligence program here at Mercyhurst College. At graduation, every student is allowed to walk across the stage to be recognized, however, not every student may have completed the degree requirements necessary to receive his or her diploma. Based on statistics from recent years, our class attempted to estimate how many out of 21 students in the class of 2011 would actually receive their diplomas on graduation day. In our graduating class, currently 20 students are scheduled to write a thesis and one student is scheduled to take a comprehensive exam. There were three rounds of anonymous voting:

Round 1 - Average:14.5; High: 19; Low: 7
Round 2 - Average: 13.08; High: 17; Low: 7
Round 3 - Average: 13.16; High: 17; Low: 10

After round 1 there was a 1-minute discussion to articulate why each of us voted the way we did. After round 2 we were presented with previous classes' graduating statistics which was designed to provide background information that we could use to adjust our individual estimates. After round 3, we arrived at our final conclusion as shown above.

Formal Expert Judgement: An Overview

2010-04-20T11:25:00.002-04:00

Introduction:
This paper, prepared at the Joint Research Centre of the European Commission within the frame of the institutional action SAFELIFE - Safety of Ageing Components in Nuclear Power Plants. Acknowledging that formal expert judgement has become a relatively well-established tool in connection to risk assessments, the paper aims to apply formal expert judgement to assess the structural integrity of nuclear power plant components. In the past, the Delphi method was one of two key methods that were used to convey structured expert opinion. The authors offer a description of the method, including the process and it's limitations.

The Delphi Method: Description
Developed in the early 1950s, it was mainly applied to technology forecasting, but also to many types of policy analyses. The method is based on a structured process for collecting and distilling
knowledge from a group of experts by means of a series of questionnaires combined with controlled opinion feed back.

Process:

A questionnaire is sent to experts.
Each expert gives his answers to the questions in an independent and anonymous way.
The responses of each expert are analysed by the monitoring team. Thelower 25% and the upper 25 % of responses are excluded.
The set of responses is then sent back to experts and they are asked ifthey wish to revise the initial predictions.
The process is reiterated until a degree of consensus is reached by experts.

Weaknesses:

A low level reliability of judgements among experts and therefore
dependency of forecasts on the particular judges selected
Sensitivity of the results to ambiguity in the questionnaire that is used for
data collection in each round
Difficulty in assessing the degree of expertise incorporated in the forecast
Responses can be altered by monitors in the attempt of moving the
following round of responses in the desired direction.

Despite these weaknesses, the authors note, "it must be acknowledged that there have been many poorly conducted Delphi applications and there is in fact an important conceptual distinction between evaluating a technique and evaluating an application of a technique." Therefore, they conclude that in general the Delphi method is useful in answering one
specific and single-dimension question. They argue; however, that it may not be effective for
determining complex forecasts concerning multiple factors since the collation of expert judgements suffers from the possibility that interactions between forecasted items may not be fully considered.

Improving the Delphi Method:
To improve forecasting reliability in the Delphi method, it is necessary to consider
the possibility that the occurrence of one event may change the probability of occurrence of other events included in the surveys. Cross impact analysis was therefore developed as an extension of Delphi method.

Source:
K. Simola, A. Mengolini, R. Bolado-Lavin. "Formal Expert Judgement: An Overview", Institute for Energy, Joint Research Center of the European Commission (2005).

The Delphi Technique In Developing International Health Policies: Experience From The SARSControl Project.

2010-04-20T10:30:00.002-04:00

Summary:
The study entitled The Delphi Technique In Developing International Health Policies: Experience From The SARSControl Project, main objective was to evaluate the “Delphi technique” as a tool to assist in developing international policies for SARDS and SARS-like diseases using the criteria found in the literature and its application in the SARSControl Delphi study.

Methods:

The European Community funded research project SARSControl was made up of 17 international organizations. Its policy evaluation work package was the setting of this study and the basis of our study was the SARSControl hence, data and results from it have been used for our analysis.
This paper uses the criteria defined in the literature to assess the process of using the Delphi technique in one of the largest up to-date projects aimed at developing international emerging infectious disease policies. The evaluation was done using the qualitative description of the SARSControl Delphi and carrying out a critical analysis of different aspects of each criterion.

Results:
The Delphi process, which was carried out over a period of nine months consisted of a pilot round, two written rounds and a final face-to-face meeting. Sixty infectious disease experts who were national experts in the field of infectious diseases who working at senior levels nationally and internationally, and represented their country on the Advisory Forum of the European Centre for Disease Control, Stockholm (ECDC) were approached. Of the 60, 47 accepted the invitation-written consent was obtained from them. Thirty-eight experts (from 22 countries) participated in the 1st written round and 28 experts (from 19 countries) in the 2nd written round; and 11 newly recruited experts with similar expertise as the participants from the written rounds (as five panellists invited from the written rounds were not able to participate ) from 9 countries participated in the face-to-face meeting. A possible explanation for this is that as national experts, they were extremely busy, and also the panelists were not invited early enough and nor were they informed of it at the recruitment phase. Two reminders were sent after the written round invitations. The response rate for the 1st round was 80% and the 2nd round was 74%. Seven replies received after the face-to-face meeting were excluded from the analysis. Misinterpretations of questions in the written rounds were rectified by either deleting or clarifying them in the 2nd round questionnaire.
The Delphi panel participants consisted from various health-related disciplines, however their specific experience with SARS was not known. Other relevant specialists for e.g. behavioural scientists and communication specialists were not included in the Delphi panel. European countries were well represented- France, Germany, Singapore, Sweden and the U.K. were countries with experience of 'probable' SARS cases. Learning from Chinese and Canadian experiences was one of the aims of the project. Two experts from China (with the most SARS experience) were invited to the written rounds but they were unable to participate. Canadian experts were not invited.

Conclusion:
The Delphi technique can aid the international policy development process and it can be a versatile tool, which creatively collects expert opinions and suggestions in a new topic. Central criteria to be met include representative panel composition, high panelist motivation, and effective but flexible administration of the Delphi process. Based on the assessment of SARSControl Delphi technique, it can be concluded that when rigorously administered, analyzed and reported, is a valuable method to develop international health policy recommendations for emerging infectious diseases, even though discrepancies in its application existed. The SARSControl Delphi technique was a positive experience, and can also be considered by others for similar purposes.

Strengths and Weaknesses:

This is the first methodological study to assess the Delphi technique in developing international infectious disease policies. Although this is a case study within one project, it exemplifies some crucial issues in applying the Delphi technique.
The major strength of the policy Delphi used was its ability to ascertain expert opinions and potential policy options, including new alternative ones, from countries representing different cultures and health policies on the timely issue of infectious disease control. This contributed to the policy recommendations to be put forward to policy makers to agree upon.
The major weakness of the Delphi was that the panel represented mostly European experts, missing countries with most SARS experience, and also the discontinuity from the written rounds to the face-to-face meetings due to drop out and financial constraints.

Source:
Syed, A., Hjarnoe, L., & Aro, A. (2009). The Delphi Technique In Developing International Health Policies: Experience From The SARSControl Project. Internet Journal of Health, 8(2), 5. Retrieved from Academic Search Complete database.

The Delphi Technique in Developing International Health Policies: Experience from the SARSControl Project

2010-04-20T10:16:00.003-04:00

Intro:The Delphi technique, according to the author, helps in structuring a group communication process that is particularly useful when there is little knowledge or uncertainty surrounding a complex area being investigated. This paper uses the criteria defined in the literature to assess the process of using the Delphi technique in developing international emerging infectious disease policies. The evaluation was done using the qualitative description of the SARSControl Delphi and carrying out a critical analysis of different aspects of each criterion.

The Delphi technique can be classified as follows:

The Classical Delphi-to establish facts
The Policy Delphi- to generate ideas
The Decision Delphi- to make decisions
The Group Delphi- for group discussions

Criteria: The Delphi technique must be applied systematically using the following 5 criteria:

Panel Composition (geographic and professional representativeness, size, heterogeneity)
Participant Motivation (response rate, written consent, clarity of questions, reminders)
Problem Exploration, e.g. as percentage of agreement /medians
Consensus Definition, e.g. as percentage of agreement /medians
Format of Feedback, e.g. individual responses, measures of tendency and spread of responses

Other criteria includes: number of rounds, anonymity to encouraging open expression of opinions, and sufficient resources which include time and administrative services.

How Delphi was used for this project:

Panel Composition: The Delphi process, which was carried out over a period of nine months consisted of a pilot round, two written rounds and a final face-to-face meeting. Thirty-eight infectious disease experts (from 22 countries) participated in the 1st written round and 28 experts (from 19 countries) in the 2nd written round; and 11 newly recruited experts with similar expertise as the participants from the written rounds from 9 countries participated in the face-to-face meeting.

Problem Exploration: Thirteen policy components considered important in terms of emerging diseases were used to formulate statements for the written Delphi rounds.
The two written rounds of the Delphi and a face-to-face meeting were found sufficient to obtain the outcomes in terms of generating policy options and alternatives. The 2nd Delphi round was crucial in clarifying issues from the 1 st round.

Round 1: Closed-ended questionnaire with a possibility to make comments.
The written comments (qualitative data) provided by the respondents were individually analyzed. (Took 2 months to complete)

Round 2: The questionnaire for the 2nd round was based on the results and comments received in the 1st round. (Took 2 months to complete)

Questions which had reached consensus in the 1 st round were excluded from the 2 nd
Unclear and confusing questions were either rephrased or new questions were formulated based on the panellists' comments
Unclear questions from the 1 st round to the 2 nd were dropped in the course of the Delphi written process.

Feedback for the Face-to-Face Meeting: Panel members were fed back with percentages of agreement within the panel for each statement in every round. Summaries of the comments from the previous rounds were also given.

Prior to the face-to-face Delphi meeting, its participants were informed via email about the Delphi technique and about the results of the written Delphi rounds. This was done to ensure that the participants had a common starting point as none of them had participated in the written rounds. The questionnaire responses were anonymous to other participants.

Conclusion:
The SARSControl Delphi process succeeded in its aim to generate policy options and alternatives — this in spite of the discontinuity in the involvement of the experts throughout the Delphi process. It can be concluded that from the results that Delphi technique when rigorously administered, analyzed and reported is a valuable method to develop international health policy recommendations for emerging infectious diseases and aid the international policy development process creatively collecting expert opinions and suggestions.

Source:
Syed, A., Hjarnoe, L., & Aro, A. (2009). The Delphi Technique In Developing International Health Policies: Experience From The SARSControl Project. Internet Journal of Health, 8(2), 5. Retrieved from Academic Search Complete database.

The Delphi Technique

2010-04-20T09:40:00.003-04:00

What is it?
The Delphi Technique is a method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem, according to Linstone and Turoff.

Where does it come from?
Delphi was developed by the Rand Corporation in the 1950s, funded by the U.S. Air Force, to find a way to establish reliable consensus of opinion among a group of experts about how Soviet military planners might target the US industrial system in an attack and how many atomic bombs would be needed to have a specified level of impact on U.S. military capability.

What is it used for?
It is widely used for more peaceful purposes today, but with the same underlying rationale: to establish as objectively as possible a consensus on a complex problem, in circumstances where accurate information does not exist or is impossible to obtain economically, or inputs to conventional decision making for example by a committee meeting face-to-face are so subjective that they risk drowning out individuals’ critical judgments.

The typical features of a Delphi procedure are an expert panel; a series of rounds in which information is collected from panelists, analyzed and fed back to them as the basis for subsequent rounds; an opportunity for individuals to revise their judgments on the basis of this feedback; and some degree of anonymity for their individual contributions.

The UK government commonly uses Delphi to make decisions or allocate resources in the health service, a classic context in which demand for resources will always outstrip their availability. Delphi is also mentioned in business texts under decision making techniques, along with other structured approaches such as the Nominal Group Technique. They allow complex decisionmaking and creative problem-solving in a way which avoids the drawbacks of conventional meetings with unstructured, free-flowing interaction and minimal direction such as

High variability in participant behavior and group social behavior · Discussion falls into a rut or goes off at tangents
The absence of an opportunity to think through independent ideas results in generalizations
High status or dominant personalities dominate discussions and decisions
Unequal participation among those present
Meetings conclude with a perceived lack of accomplishment

The basic method to implement Delphi as described by Delbecq et al is:

develop an initial questionnaire and distribute it to the panel
panelists independently form ideas to answer the questionnaire and return it
the moderator summarizes the responses to the first questionnaire and
develops a feedback report along with the second set of questionnaires
for the panelists
panelists independently evaluate earlier responses and vote on the second questionnaire
the moderator develops a final summary and feedback report to the group
and decision makers

Some variations to this very basic method include:

The number of iterations (the more rounds, the closer the consensus likely to be reached)
The method of selection and size of the panel
The scoring system and the rules used to aggregate the judgments of the panelists
The extent of anonymity afforded to the panelists
How consensus is defined and how disagreements are dealt with

Conclusion
This article, written by Nic Underhill, provides a very concise overview of what the Delphi technique is to the average person. Unfortunately, he does not identify any potential strengths and/or weaknesses of the technique which significantly diminishes the utility of his article. Although he introduces the reader to the Delphi technique, he does not give that person any direction in how to apply it to his/her own unique situation.

Using Delphi Technique in a Consensual Curriculum for Periodontics

2010-04-20T09:14:00.002-04:00

Fried and Leao conducted an experiment using the Delphi technique to obtain a group consensus on the curriculum for periodontics. The goal of this study was to use theDelphi technique to identify a consensus about what topics shouldbe included in a periodontics curriculum for undergraduate dentalstudents.

METHODS
Nine periodontics faculties at dental schools in twoBrazilian cities participated in this curriculum planning process. Forty undergraduate professors, all dentists, were invited to participate in the study. For the first phase of the experiment, lecturers were askedto list, in writing, items that should be included in a periodonticscurriculum for dental students. Suggested items were split intotwo groups. The first group involved theory-related items associatedwith foundational concepts and basic principles of periodontics.The second group of items, related to dental practice, was associatedwith laboratory training and clinical experience with patients.For the second phase of the experiment, previously obtainedresponses were scrutinized and collated into items. Then, acomprehensive Likert-scale questionnaire was compiled for submissionto the panel. The questionnaire offered the following optionsfor rating the importance of proposed curriculum items: "indispensable";"important"; "relatively important"; "of little importance";and "should not be included." The researchers sent the questionnaire to all forty respondents and evaluated the answers as follows:

itemwas "kept" if it reached a "50 percent plus one" consensus classificationas "essential/important"
item was eliminated if it reached a"50 percent plus one" consensus classification as "of littleimportance/should not be included"

item was included in the next round of the questionnaire if it did not reach a "50 percent plus one" consensus agreement or a "50 percent plus one" consensus rejection

During stage three, the only participants were individuals whoprovided the most extreme positive ratings or the most extreme negative ratings during the previousstage for items where no clear consensus was reached.

For the fourth and last stage, for each non-consensual itemsingled out in the third stage, the positive and negative justificationsrequired from third-stage participants were transcribed intoa new questionnaire. All the original participants were then asked to for inclusion or against inclusion of each pending item. The format used for thefourth stage questionnaire was that of a sequence of items,each of them carrying its associated positive and negative justifications,followed by corresponding "yes" and "no" answering options.Frequencies for the answers thus obtained were calculated withthe SPSS statistical package.

RESULTS
Out of the forty initial participants in the study, six (15percent) eventually dropped out. Two moved abroad, and the otherfour were unable to stay involved until the end of the experiment.In accordance with the literature, this 85 percent responserate may be considered as "good."¹⁵ Out of the thirty-four lecturers who participated in each ofthe four Delphi technique stages, thirteen (38 percent) werefemales, and twenty-one (62 percent) were males. They had anaverage of ten years (±8.19) teaching experience.

Throughout the four stages of the consensus-building Delphitechnique approach used in this study, participants identifiedvarious up-to-date scientific research trends within periodontics.However, in spite of that, items associated with some recenttechnological advances were not included in the final set of"indispensable" items. Items related to technological advancesin odontology, such as human molecular genetics and DNA probing,were initially listed as important to a periodontics curriculum.However, half way through their study, they were excluded fromthe final list of selected items. The results of their study indicate that some issues such as a public health-oriented vision,were not addressed by the panel. This suggests that the majorityof the individuals involved are focused on the treatment modelrather than on health promotion. However, such a drawback mayalso be credited to limitations of the Delphi technique, whichdoes not allow an interpersonal discussion of topics. Overall, use of the Delphi process allowed accomplishmentof the primary goal of the study, which was to identify a consensusabout what items should be included in the periodontics curriculumfor undergraduate dental students based on the perspectivesof a panel of faculty who teach periodontics at several differentinstitutions.

Source: http://www.jdentaled.org/cgi/content/full/71/11/1441

An Overview of Four Futures Methodologies (Delphi, Environmental Scanning, Issues Management and Emerging Issue Analysis)

2010-04-20T05:56:00.003-04:00

In this paper on futures research by Trudi Lang, the author compares and contrasts the Delphi method with other futures methodologies. The Delphi technique is found to be different from the other three methodologies chosen by the author for this study viz., Environmental Scanning, Issues Management and Emerging Issues Analysis (EIA).

For the Delphi technique, the positives are that Delphi studies have an excellent record of forecasting computer capability advances, nuclear energy expansion, energy demand and population growth and the technique is also said to expose real agreements and disagreements among respondents as well as giving the facilitator simple and direct control over the scope of the study.

The key drawbacks of the technique are that there is a strong response of the group to conform with the statistical feedback of the panel, extreme points of view, which may provide new insights tend to be suppressed. Also, the way the questionnaire and process is structured can lead to a bias and a Delphi study is at the mercy of the view and biases of the coordinating team, who choose respondents, interpret the information and structure the question.

The author's evaluation is that it is difficult to evaluate the accuracy and reliability of the Delphi method, because the technique is based on determining the opinion of panel members and the findings thus become person and situation specific. In addition, much of the work undertaken to evaluate the Delphi technique has been done with university students asking almanac-type questions.

The author however mentions that the Delphi technique is found to be best suited for exploration of issues that involve a mixture of scientific evidence and social values and that the Delphi method to be considered as one of last resort - to deal with extremely complex problems for which there are no other models.

In conclusion, the author states that the Delphi technique is an "inside - out" methodology when compared to the other techniques which are "outside-in" and by their very nature are all inter-related. The author advocates an integrated approach in using the futures methodologies available so that the strengths of one can make up for the weaknesses of the other.

Comments:

The author's research is thorough and a wide variety of resources have been referenced in her article. The author's conclusion is also valid; as for instance, the Delphi technique is far more successful than other techniques in gaining a consensus from a group but is weaker on allowing independent voices to be heard. In this regard, EIA provides a methodology that is more sensitive to these independent voices and could compensate for this weakness of the Delphi methodology.

Pragmatic Research Design: an Illustration of the Use of the Delphi Technique

2010-04-20T00:44:00.003-04:00

This study conducted by two scholars at Rhodes University in South Africa used the Delphi Technique in hopes of forecasting the educational needs that will have to be met to prepare students to be entrepreneurs over the next 20-40 years. This paper does not walk readers through the results of the study, but rather the practical challenges in definitions and the organization of applying the Delphi Technique.

Definitions:
Based on the author’s review of the literature, they identified five main characteristics which define the technique:

Its focus on researching the future or things about which little is known,
Reliance on the use of expert opinion,
Utilizing remote group processes,
The adoption of an iterative research process, and
The creation of a consensus of opinion.

The authors also identify three versions of the Delphi Technique:

Numeric – aims to specify a single or minimum range of numeric estimates through the use of summary statistics.
Policy - on the exploration, generation and definition of several alternatives and the arguments for and against each of these alternatives.
Historic - aims to explain the range of issues that fostered a specific decision, identification of several scenarios that could have led to the resolution of a past problem.

Entrepreneurship was defined as person who provides innovation in an economy, not owners of micro-businesses in saturated markets. The foundation of their study is based on a review of literature that suggests entrepreneurship can be formed through education.

The Experiment:
To comprehensively forecast the answer to their question, they asked three separate questions. The questions were the following:

What sector of the South African economy will most likely offer the greatest potential for entrepreneurial opportunities in the next 25 to 40 years?
What qualities are needed by graduates to equip them to be innovative entrepreneurs in the future?
What should Higher Education in South Africa do to prepare/develop students to constructively participate in the future economy as innovative entrepreneurs?

Three separate panels were created because each question requires answers from a different set of experts.

The panel for the first question was referred to members of government departments and research councils.
The panel for the second question was referred to endowed Chairs in the area of entrepreneurship.
The panel for the third question was referred to alumni of entrepreneurship programs and educationalists and academics in these programs.

The Future of High-Technology Crime: A Parallel Delphi Study

2010-04-19T19:29:00.005-04:00

This study conducted by Larry E. Coutorie in 1995 is a follow-up to a 1980s study using the Delphi Technique to forecast the future of high-technology crimes. One of the purposes of this study is to give law enforcement a forecast of where high-tech crimes are headed, since most other techniques only allow reactionary responses.

The Experiment:
The study was conducted using two panels. One was comprised of “traditional” experts, or people already in the high-tech law enforcement field, and “nontraditional” experts, member of hacker and cracker groups recommended by other experts. Two groups on experts were sent three rounds of questionnaires with the following questions, refined each time by the groups’ responses to the previous questionnaire.

In your opinion, what area(s) of high technology will be the focus of criminal activity in the next ten years?
What form(s) do you believe this activity will take?
What steps should be taken now to prepare the police to combat this criminal activity?
Do you believe the responsibility for criminal investigation of high-technology crimes will be primarily that of government or private businesses? Why?
Do you believe the responsibility for crime prevention activities regarding high-technology crimes will be primarily that of government or private businesses? Why?

Findings:
Each groups’ perspective diverged significantly from the first round of questioning onward. However, at the end of the three questionnaires, a consensus on several issues was identified.

Likely high-tech future crime areas include computer system attacks via telecommunications, a growing increase in computer-assisted fraud, and computer assisted data manipulation or theft.
Crime will take the form of software piracy, increased incidents of computer assisted counterfeiting, increased incidents of financial fraud, and increased attacks on computer systems via advanced technologies.
Preventative steps recommended include recruitment of individual with computer knowledge, increased public/private partnership, more training for law enforcement officers earlier in their career, and legislation that better defines jurisdiction.
At the time of this study experts forecasted private business would conduct the initial investigation and have an active participatory role in government investigations.
They also forecasted that private businesses would be responsible for protecting their own assets, with government assistance in identifying potential threats.

Use of the Delphi Method to Generate a Gold Standard for the Medical Coding Industry

2010-04-19T15:57:00.002-04:00

The Delphi Method was developed in the 1950’s at the RAND Corporation. The original process is still in use today. In this article the authors proposed a modified version of the process with the intent of providing a “gold standard” for medical coding for records that eliminates coder variability. They called their new process, The Delphi Method of Medical Coding, which is a patent pending process.

This proposed new method of the Delphi process was created in hopes of eliminating some of the biases that can be presented in the original method. It also eliminates some of the cumbersome written communication required in the old method.

The Delphi Method of Medical Coding

1. Multiple coders are given identical sets of medical records and asked to code them as they would normally. These results show the different possible ways of coding. Each different possible code is shown on consolidated list.

2. The consolidated list from step 1 are presented to a second set of coders. The coders then make a simple yes/no determination about whether they should be applied to the associated record or not.

3. For each of the codes on the listing, a percentage is calculated of coders who decided the code should be applied. From this percentage, determined by either a fixed percentage or some derived statistic, one of three decisions is made. These possible decisions are the code should be applied to the record, the code should not be applied, or the code is indeterminate.

Conclusion

Although this study only showed only one use of this method, it’s possible it could be applied to other areas, similarly to the versatility of the original Delphi Method. However, it would be a good idea to conduct research comparing this method with the original to see if it truly eliminates biases while arriving at a similar outcome. This new method may provide a better way of gaining active participation due to smaller amount of steps and less time spent on the overall process. The article talks about possibly even implementing this method as a game, which may provide an even easier interaction for the people involved it the process.

Engaging communication experts in a Delphi process to identify patient behaviors that could impact communication in medical encounters.

2010-04-19T15:51:00.002-04:00

Communication between patients and physicians is a very important area of study in the world of communication. The article states that evidence supports the Four Habits Model has a link between specific behaviors of the physicians with improved outcomes of the care provided.

Method

Utilizing the Four Habits Model as a starting point, a four round Delphi process was conducted with 17 international experts in communication research, medical education, and health care delivery. Each round was conducted via the internet.
Implementation of the Delphi Process:
Round 1: The experts reviewed a list of proposed patient verbal communication behaviors, identified based on a review of communications literature, within the Four Habits Model framework. The experts chose from the options of: approving the proposed list, adding new behaviors, or modifying existing behaviors from the list.
Rounds 2, 3, and 4: Each of these rounds was the same. Each behavior was rated for its fit, by agreeing or disagreeing, with a particular habit. After each round, the percentage of agreement for each behavior was calculated and the data was used for the determination of behaviors to be analyzed in following rounds. Behaviors that received more than 70% of the experts’ votes were considered as achieving consensus.

Results

Throughout the course of the four rounds, the experts started with the 14 originally proposed patients verbal communication behaviors, of which they modified 12, added 20 behaviors. They eventually retained only 22 behaviors, which included such obvious behaviors as asking questions, expressing preferences, and summarizing information.

Conclusion

The use of the Delphi appears to work very well for the expectation of finding out more effective ways of communication for physicians and patients. Without the notes of the actual rounds and how everything worked out when the experts voted on what they believed was right or wrong it’s difficult to actually make an assessment of the Delphi process.

Essential Components Of Curricular Learning Communities In Higher Education

2010-04-19T10:44:00.004-04:00

The purpose of this study was to identify the essential components of curricular learning communities in higher education. A panel of experts participated in a four-round Delphi process designed to identify these essential features. The writer used a modified Delphi process to first elicit and then rate the importance of characteristics of curricular learning communities in higher education.

Delphi Advantages:
-The Delphi technique offers the advantage of group response without the attendant disadvantages sometimes experienced with group problem solving or decision-making.
-Expert participants are more likely to generate reasoned, independent, and well-considered opinions in the absence of exposure to the "persuasively stated opinions of others". Because the experts do not ever participate in a face-to-face discussion, there is no danger of one or more individuals’ opinions being swayed by a more dominant or more experienced individual.
-Efficiency and flexibility, especially in light of modern communication technologies such as e-mail and the Internet. Experts may be drawn from a wide geographic area, and the participants’ commitment in terms of time and money invested is minimal.
-Delphi method has been shown to be an effective way to conduct research when the responses being sought are value judgments rather than factual information. Although it is more difficult to assess the "correctness" of value judgments, it is generally agreed upon that value judgments are not all equal but can in fact be more "right" or more "wrong."

Delphi Limitations:-Delphi should not be used when any of the following three critical conditions are not present: adequate time, participant skill in written communication, and high participant motivation. It is estimated that a minimum of 45 days is required to carry out a Delphi study.
-Participants must be knowledgeable and able to clearly communicate their ideas. A high degree of motivation is needed to offset the tendency for participant dropout as the study progresses. Because there is no direct contact between participants, those who are not highly motivated and interested in the subject at hand may feel isolated or detached from the process.
-Another is the problem of bias in Delphi studies that can occur from poorly worded or leading questions or selective interpretation of the results.

Instrument Design and Implementation
Round One: Initial Survey:
The first round in the current study consisted of a brief survey, designed to collect some demographic data on the participants, and one open-ended question.

Round Two: Questionnaire One:
A list of 79 features was compiled from the information obtained in the initial survey. Obvious repetitions were eliminated, though items that were similar but not exactly the same were maintained. Items were sorted into four categories: Curricular Features, Pedagogical Features, Structural Features, and Environmental Features. Participants were asked to rate each feature on a Likert-type scale, identifying each feature as an "essential" (5), "very important" (4), "moderately important" (3), "slightly important" (2), or "not important"(1) characteristic of a curricular learning community.

Round Three: Questionnaire Two:
Questionnaire Two listed only the features that had received a mean rating of 4.0 or higher in the previous round. Once again the items were placed into the four categories of Curricular, Pedagogical, Structural, and Environmental.

Round Four: Questionnaire Three:
The third and final questionnaire listed the forty features that received the highest rating (determined by mean and mode) on the previous questionnaire. Panelists were given the following information: ranking of the items from first and second questionnaire, mean score of the items from both rounds, and the number of times each item was selected as one of the three to five most important items.

In this round, panelists were asked to assign a total of 100 value points to the forty items. At the end of this questionnaire, participants were asked to answer the another open-ended question.

Summary of Data Collection and Analysis Procedures:

The following table outlines the four-round Delphi procedure that was followed in this study: (see link for table)

Link: http://www.winona.edu/advising/lcchapter3.htm

Using the Delphi Technique to Achieve Consensus

2010-04-19T10:29:00.000-04:00

How it is leading us away from representative government to an illusion of citizen participation.

This article explained the Delphi Technique and how it is used as a methodology in the education system. It was critical of the technique and explains the consequences and motivations of those who are using it.

The Delphi Technique and consensus building are both founded in the same principle - the Hegelian dialectic of thesis, antithesis, and synthesis, with synthesis becoming the new thesis. The goal is a continual evolution to "oneness of mind" (consensus means solidarity of belief) -the collective mind, the wholistic society, the wholistic earth, etc. In thesis and antithesis, opinions or views are presented on a subject to establish views and opposing views. In synthesis, opposites are brought together to form the new thesis. All participants in the process are then to accept ownership of the new thesis and support it, changing their views to align with the new thesis. Through a continual process of evolution, "oneness of mind" will supposedly occur.
In group settings, the Delphi Technique is an unethical method of achieving consensus on controversial topics. It requires well-trained professionals, known as "facilitators" or "change agents," who deliberately escalate tension among group members, pitting one faction against another to make a preordained viewpoint appear "sensible," while making opposing views appear ridiculous.

The facilitators or change agents encourage each person in a group to express concerns about the programs, projects, or policies in question. They listen attentively, elicit input from group members, form "task forces," urge participants to make lists, and in going through these motions, learn about each member of a group. They are trained to identify the "leaders," the "loud mouths," the "weak or non-committal members," and those who are apt to change sides frequently during an argument.

Suddenly, the amiable facilitators become professional agitators and "devil's advocates." Using the "divide and conquer" principle, they manipulate one opinion against another, making those who are out of step appear "ridiculous, unknowledgeable, inarticulate, or dogmatic." They attempt to anger certain participants, thereby accelerating tensions. The facilitators are well trained in psychological manipulation. They are able to predict the reactions of each member in a group. Individuals in opposition to the desired policy or program will be shut out.

The Delphi Technique works. It is very effective with parents, teachers, school children, and community groups. The "targets" rarely, if ever, realize that they are being manipulated. If they do suspect what is happening, they do not know how to end the process. The facilitator seeks to polarize the group in order to become an accepted member of the group and of the process. The desired idea is then placed on the table and individual opinions are sought during discussion. Soon, associates from the divided group begin to adopt the idea as if it were their own, and they pressure the entire group to accept their proposition.

How the Delphi Technique Works
First, a facilitator is hired. While his job is supposedly neutral and non-judgmental, the opposite is actually true. The facilitator is there to direct the meeting to a preset conclusion. The facilitator begins by working the crowd to establish a good-guy-bad-guy scenario. Anyone disagreeing with the facilitator must be made to appear as the bad guy, with the facilitator appearing as the good guy.

Next, the attendees are broken up into smaller groups of seven or eight people. Each group has its own facilitator. The group facilitators steer participants to discuss preset issues, employing the same tactics as the lead facilitator. Why hold such meetings at all if the outcomes are already established? The answer is because it is imperative for the acceptance of the School-to-Work agenda, or the environmental agenda, or whatever the agenda, that ordinary people assume ownership of the preset outcomes. If people believe an idea is theirs, they'll support it. If they believe an idea is being forced on them, they'll resist.

How to Diffuse the Delphi Technique

Three steps can diffuse the Delphi Technique as facilitators attempt to steer a meeting in a specific direction.

Always be charming, courteous, and pleasant. Smile. Moderate your voice so as not to come across as belligerent or aggressive.

Stay focused. If possible, jot down your thoughts or questions. When facilitators are asked questions they don't want to answer, they often digress from the issue that was raised and try instead to put the questioner on the defensive. Do not fall for this tactic. Courteously bring the facilitator back to your original question. If he rephrases it so that it becomes an accusatory statement (a popular tactic), simply say, "That is not what I asked. What I asked was . . ." and repeat your question.

Be persistent. If putting you on the defensive doesn't work, facilitators often resort to long monologues that drag on for several minutes. During that time, the group usually forgets the question that was asked, which is the intent. Let the facilitator finish. Then with polite persistence state: "But you didn't answer my question. My question was . . ." and repeat your question.

Never become angry under any circumstances. Anger directed at the facilitator will immediately make the facilitator the victim. This defeats the purpose. The goal of facilitators is to make the majority of the group members like them, and to alienate anyone who might pose a threat to the realization of their agenda. People with firm, fixed beliefs, who are not afraid to stand up for what they believe in, are obvious threats.

At a meeting, have two or three people who know the Delphi Technique dispersed through the crowd so that, when the facilitator digresses from a question, they can stand up and politely say: "But you didn't answer that lady/gentleman's question."

Establish a plan of action before a meeting. Everyone on your team should know his part. Later, analyze what went right, what went wrong and why, and what needs to happen the next time. Never strategize during a meeting.

link: http://www.eagleforum.org/educate/1998/nov98/focus.html

This process sounds very political in this type of setting and that personal agendas, or vendettas may come into play if a facilitator is not in control of the situation. The illusion of a representative participation is obvious in this type of situation.

Apprehending the Future: Emerging Technologies, from Science Fiction to Campus Reality

2010-04-19T05:26:00.003-04:00

Bryan Alexander, Director of Research at the National Institute for Technology and Liberal Education (NITLE) talks about the various techniques used to decipher emerging technology trends from the perspective of higher education in his article. The author talks about how the Delphi technique has been particularly useful in forecasting technology in higher education.

One of the cases where the Delphi technique is used is in the Horizon Project. Launched in 2002, this project draws on a large body of experts across academia. Over several months, the group identifies trends, ranks their impact, compares estimates, and progressively builds up a profile of emerging technologies. This is then published as the annual Horizon Report. The January 2009 report identified the following technologies:

Mobiles (time-to-adoption: one year or less)
Cloud computing (time-to-adoption: one year or less)
Geo-Everything (time-to-adoption: two to three years)
The Personal Web (time-to-adoption: two to three years)
Semantic-Aware Applications (time-to-adoption: four to five years)
Smart Objects (time-to-adoption: four to five years)

Another application of the Delphi Technique in higher education was the "The Future of Internet III" project by Pew Internet & American Life Project and Elon University. This study was much broader in scope and had a much longer timeline. The outcome from this exercise is as below:

The mobile device will be the primary connection tool to the Internet for most people in the world in 2020.
The transparency of people and organizations will increase, but that will not necessarily yield more personal integrity, social tolerance, or forgiveness.
Talk and touch user-interfaces with the Internet will be more prevalent and accepted by 2020.
Those working to enforce intellectual property law and copyright protection will remain in a continuing "arms race," with the crackers who will find ways to copy and share content without payment.
The divisions between "personal" time and work time and between physical and virtual reality will be further erased for everyone who's connected, and the results will be mixed in terms of social relations.
Next-generation engineering of the network to improve the current Internet architecture is more likely than an effort to rebuild the architecture from scratch.

In conclusion, the author states that no technique can effectively predict the future and using a combination of techniques can only help us have some idea of the future. The future is increasingly complex and "black swans" continue to occur and have enormous effects on the future.

Comments:

As can be seen from the above conclusions, they do not seem to be too radical or innovative in any way but instead seem to tug the line of what is believed to be the general consensus (for instance mobile devices being the primary connecting tool in 2020 or touch and talk user interfaces being prevalent does not necessarily need to be deciphered by experts). This is in fact the major drawback of this technique - that Delphi outcomes can be driven by a desire for consensus, rather than actual agreement, meaning that divergent ideas can and often do get quashed.

The Delphi Technique; Lets Stop Being Manipulated

2010-04-17T01:12:00.002-04:00

More and more citizens are being invited to participate in various form of meeting, councils, or boards to "help determine" public policy in one field or another. According to the author of this article Albert V. Burns, this sounds great, but in reality it is deception.

Generally, each meeting has someone designated to facilitate the meeting. The job of facilitator is to be neutral, non-directing helper to see that the meeting flows smoothly. Actually, he or she is there for exactly the opposite reason. To see that the conclusions reached during the meeting are in accord with a plan already decided upon by those who called the meeting.

The process used to "facilitate" the meeting called Delphi Technique. This technique was developed by the RAND Corporation for the U.S. Department of Defense in 1950s.

How does the process take place?

First, the person who will be leading the meeting, the facilitator must be a likable person with whom the meeting participants can agree and sympathize.

Facilitators are trained to recognized potential opponents and how to make such people appear aggressive and foolish. The audience are broken up into groups seven or eight people each.

Within each group discussion takes place of issues already decided upon by the leadership of the meeting. Generally, the participants are asked to write down their ideas and disagreements with the papers to be turned in and "compiled" for general discussion.

How do you know that the ideas on your notes were included in the final result? You Do not! You will come to a conclusion that you were probably in the minority. You do not even know if any one's ideas are part of the final conclusion.
Those who organized the meeting are able to tell the community that the conclusions, reached at the meeting, are the result of public participation.

Actually, the desired conclusions had been established, long before the meeting ever took place.

Prioritization Process Using Delphi Technique

2010-04-16T23:26:00.002-04:00

In a white paper, Alan Cline who is a member of Carolla Development, Inc. gives some insights on how to prioritize using Delphi technique.

Background and Motivation

Delphi technique was developed in 1960s as as forecasting methodology. Later, the U.S. government enhanced it as a group decision making tool. Delphi is particularly appropriate when decision-making is required in a political or emotional environment. The tool works formally or informally, in large or small contexts. For example: Taiwan used the method to prioritize their entire Information Technology industry.

Delphi Prioritization Procedure

1- Pick a facilitation leader: The facilitator is an expert in research data collection, and is not a stakeholder.

2- Select a panel of experts: The panelist should have an intimate knowledge of the projects.

3- Identify a straw man criteria list from the panel: Brainstorm a list of criteria that are appropriate to the projects.

4- The panel ranks the criteria: The panel ranks from 1(very important) to 3 (not important). The ranking should be done individually and anonymously.

5- Calculate the mean and deviation

6- Re rank the criteria

7- Identify project constrains and preference: Constrains could be budget or mandatory regulations.

8- Rank project by constrain and preference

9- Analyze the result and feedback to panel

10- Re rank the project until it is stabilized

Green Team Summary of Findings: Imagery Analysis (4 out of 5 stars)

2010-04-15T13:48:00.001-04:00

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 15 April 2010 regarding Imagery Analysis specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Imagery analysis is the extraction of meaningful information from images. It is used by both the military and the civilian sectors. Imagery analysis is highly useful in state-related matters as well as natural resource related businesses and scientific research institutes. The groups that use imagery analysis run the gamut from NGO's to multinational organizations.

There are many different types of imagery that can be used alone or in combination with others. This list can include satellite, infrared, electro-optical, and multispectral imagery. One of the most well known types of imagery is satellite imagery. Since the early days of satellite imagery there have been dramatic improvements in availability and quality that have lead to increased accessibility. Some of the largest commercial vendors are GeoEye, DigitalGlobe, ImageSat International, and Satellite Imaging Corporation. Google Earth is one of the most popular sources for images because it is commercially available to anyone with Internet access.

STRENGTHS AND WEAKNESSES:

Strengths

Capable of providing a detailed overview of large area

More manageable than on-the-ground classification efforts

Accessibility and ability of systems continues to improve

Data provided is clear and highly detailed

Can be used to corroborate other collected data

Can be applied to a variety of problems, and is very adaptable

Allows for data collection in remote areas

Relatively easy to perform cue to commercial availability

Allows for a retrospective view for an area

Weaknesses

Software and systems can be expensive

Limited by resolution, image quality, atmospheric haze, and contrast

Hard to compare images taken from different angles and at different resolutions

This methodology still requires a trained analyst to go over the tentative findings and discriminate between objects

Need for trained professionals (level of training debatable)

Chance for error if preparatory steps not followed such as accounting for clouds, resolution, etc.

Does not account for other factors such as change between the images

Can be subject deception

How To:

There appear to be no standard procedures applicable to all images at all times or for all purposes. However, several standard processes for specific purposes were identified through the literature reviewed in advance of this class. For example, with respect to classifying types of vegetation:

1.Identify landmarks

2 Align landmarks (if using photos from different time periods)

3.Standardize levels of contrast within an image (necessary when images were taken by different sensors)

4.Image Classification- divide the photograph into "clusters", spatial areas with the same characteristics then decide what those are with respect to the region and culture of an area.

5.Accuracy Assessment- compare classification results to what the ground areas truly are as revealed by aerial photographs or teams on the ground.

Personal Application:

The class looked at ten designated images; five different cities at two different heights. The class had three minutes to look at the two images of the city and identify the following: Density, Bodies of Water, Building Shadows, Landmarks, and Vegetation. These five criteria helped to narrow the range of possible cities. For example, when looking at New York City, the identification was made possible by looking for vegetation (Central Park), building shadows (consistently tall buildings), and density (buildings very close together over a wide area).

Summary of Findings (White Team): Geospatial Analysis (4 out of 5 stars)

2010-04-15T13:21:00.004-04:00

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 15 April 2010 regarding Geospatial Analysis specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Geospatial Analysis is a method using aerial imagery technology for analysis. These technologies include, Electro Optical, Infra Red, Multispectral, or Radar. This type of imagery is commercially available and useful for a multitude of applications. Analyzing geospatial imagery can provide decision makers with visual indicators that offer evidence of internal operations. Static images to aid the intelligence process by contributing to all-source analysis provided to decision makers. Imagery can be combined with cartography and mapping to determine distances, heights, demographics and population density.

Strengths:

Objective in nature, data is structured.
Easy to perform, even for untrained analysts.
Readily available open sources such as Google Earth/ it is commercially available.
Satellite imagery can provide data in remote areas that lack accurate geographic maps.
Usually more cost efficient than sending personnel to a site.
Can also provide powerful visualization to assist when shaping policy
On the ground situation can be assessed, even when outside parties don't have access to the area.
The same data can be used on multiple types of problems/issues.
Technology is continually being updated to allow for easier use.

Weaknesses:

Potentially expensive.
Static images (only a snapshot in time).
May require training depending on the level of analysis required.
Errors may occur due to lack of resolution, deception efforts, or even weather.
Imagery can generally be obtained at specific time intervals (i.e.- satellites cannot maintain stationary low earth orbit).
Unable to view objects/structures located beneath the surface of the Earth.
May require additional resources to provide specific measurements (e.g.-distance between objects, or heights of structures).
Privacy and confidentiality issues may arise as technologies and resolution advance in sophistication.

How to:

Images to be analyzed should show the same season (in case multiple images are used for analysis) and must be accurately registered (matched up) to the ground and to each other.
The images must be radiometrically calibrated to minimize effects of instrument variations and atmospheric haze (Radiometric Normalization (RRN)). This is necessary when images are taken by different sensors, to standardize radiometry, or contrast within an image
The landmarks in the images need to be aligned (geometric rectification). This is necessary for analysis over time.
A classification scheme should be decided on and designed. Classification categories ex. cultivated land or grassland, forested land vs. non forested land can be used.
Classify the images divide the photograph into "clusters", spatial areas with the same characteristics, then decide the category they should be placed.
To reduce errors in classification, techniques such as having a human check portions of computer-classified work is also done (spatial re-classification).
Classification results are compared to what the ground areas truly are (as revealed by aerial photographs) to assess the accuracy of the classification.

Application:

Geospatial Analysis, as a method, was demonstrated by looking at various images of major international cities. We had to identify them by viewing wide screen shots taken from Google Earth followed by viewing a close-up of the image. By recognizing key indicators, i.e. population density, landmarks, shadows, terrain, etc..., we were able to identify the cities (some, but not all) which taught us that as a method, it is best practiced using modifiers, particularly when resolution is lacking or distorted. The depth of Imagery analysis is dependent upon the source of the image (i.e. aerial vs. satellite) and the skill set of the analyst. This exercise, a descriptive analysis, would be an excellent starting point for those with low exposure to imagery analysis as it is low cost and easily accessible.

Satellite Imagery Activism: Sharpening the Focus on Tropical Deforestation

2010-04-13T10:22:00.008-04:00

The Rise of Satellite Imagery Activism
According to the authors (from the RAND Corp. & George Washington University), the first Landsat satellite was launched by the U.S. in 1972 and since then has opened up the doors to satellite remote sensing, especially in terms of natural resource sustainability and human activity monitoring. The original users of satellite imagery were state agencies (civil & military), natural resource related businesses and scientific research institutes. However today the groups that use satellite imagery run the gamut from NGO's to multinational organizations. Since the early days of satellite imagery there have been dramatic improvements in availability and quality that have lead to increased accessibility. First, cheap and highly powerful computing abilities have broadened the range of individuals that can work with imagery data. Second, "the advent of user-friendly software for image processing and analysis has made imagery analysis much less the purview of remote sensing specialists". Third, the cost of imagery data has drastically declined with the onset of cheaper Landsat/SPOT images, declassified U.S. & Russian intelligence imagery and many other forms of commercially available imagery. Lastly, the internet and CD-ROM's have facilitated the spread of imagery with giants like Google moving into the arena.

Imagery Activism & Tropical Forests
Due to "rising levels of global transparency" and recent software and internet abilities, there are many avenues in which citizens can bring awareness to public policy issues through the use of satellite imagery. A lot of environmental factors can be analyzed via satellite imagery such as changes in vegetation, biological stress and habitat characterization. Out of these issues, there has been the greatest focus on monitoring trends in tropical deforestation. Besides simple deforestation, "satellite imagery can be used to monitor growth, as well as the effects of human activity and exploitation...on the surrounding ecosystems". Satellite images can provide objective proof of potential logging infractions and at the same time be built up into databases to provide long-term monitoring of forests. In the last several years, the international community's gaze has fallen on the tropical rainforests of Brazil and Indonesia. Brazil in particular has seen the dramatic expansion of road networks, farms, pastures and plantations, all at the expense of the rainforests.

Conclusions
In summary, this paper examined the rise of satellite imagery in today's society and how this imagery affects the world's tropical rainforests. The paper's focus was on deforestation in specifically Brazil and Indonesia. Furthermore, the paper's aim is to attract global attention to the problem of deforestation by utilizing satellite imagery data and other geospatial technologies.

Source
http://www.aseanenvironment.info/Abstract/41016424.pdf

Tropical Cyclone Intensity Analysis and Forecasting from Satellite Imagery

2010-04-13T09:12:00.009-04:00

Introduction
This article by Vernon F. Dvorak, discusses the implications of analyzing cyclones with intensity analysis, by using satellite platforms. Dvorak notes that satellite imagery is highly useful in monitoring tropical cyclones because methods have been developed that allow analysts to estimate the intensity of the cyclone based on certain distinguishing cloud features.
The Technique
The technique of combining intensity analysis with satellite imagery yields rather good estimates in terms of forecasting storm intensity. The specific cloud features that are used to estimate cyclone intensity are derived from two categories of features: "central features" and "outer-banding features". Each of these features is assigned a "T-number" ("T" for tropical) which denotes tropical disturbances. T1 describes disturbances "exhibiting minimal but significant signs of tropical cyclone intensity", whereas T8 denotes the "maximum possible" cyclone intensity. Next, each of the cloud features is analyzed to determine whether the cyclone will remain at its modeled intensity over the next 24 hours.

The Analysis Procedure
The intensity analysis via the satellite platform consists of three stages. The first stage "requires a qualitative judgment as to how cloud features related to cyclone intensity have changed between yesterday's picture and today's". The second and third stages of analysis involve examining the overall cloud pattern and component features to see if they agree with the modeled intensity estimate that the forecasters arrived at during the first stage of analysis.
The Forecast Procedure
To come up with the actual intensity forecast, the researchers have to use either the cyclone's model curve to obtain tomorrow's intensity or adjust the curve when an interuption due to landfall or the approach of some unfavorable circumstance is near. These changes can be detected with an unexplainable change of T-number or in changes to the cloud features.
Performance
A researcher named Erickson conducted the first tests using intensity analysis via satellite imagery in 1972, and his results have shown good consistency. Another researcher by the name of Arnold (1972)used the T-number estimates in tandem with the Joint Typhoon Warning System estimates for the west Pacific and had pretty good results as well.

(Satellite Imagery of Tropical Cyclone patterns with "T" Numbers)
Source
http://journals.ametsoc.org/doi/pdf/10.1175/1520-0493%281975%29103%3C0420%3ATCIAAF%3E2.0.CO%3B2

Validation of SPOT-5 Satellite Imagery for Geological hazard Identification

2010-04-13T06:42:00.003-04:00

Summary
This study was developed to determine whether SPOT-5 satellite imagery could be used to develop geographic hazard maps for areas lacking available resource. The study took place in Nicaragua, and compared SPOT-5 analysis with newly created threat maps, and additional geographic monitoring. Maps were created using SPOT-5 imagery for hazard inventory, hazard susceptibility, and vulnerability mapping. The study concluded that SPOT-5 imagery could be used to effectively develop geographic hazard maps. The system was however limited by the level analysis needed to create the hazard maps.
Strengths
Satellite imagery was capable of providing necessary data in remote areas that lack accurate geographic maps. Additionally it allowed for direct GIS system and an ability to gather data despite poor weather conditions.
Limitations
Despite the SPOT-5 test success, the full process of creating the maps required large amounts of additional analysis. The system was also unable to easily identify certain features needed for hazard mapping, which could lead to possible issues without additional support from standard measurement systems

Source:http://www.cartesia.org/geodoc/isprs2004/comm1/papers/51.pdf

The Use Of Satellite Imagery In Landslide Studies In High Mountain Areas

2010-04-13T06:02:00.004-04:00

Summary

This is a case-study that compares the application of Landsat ETM+ and IKONOS imagery when assessing a natural terrain's susceptibility to landslides. This study looked at six areas located in the upland areas of Nepal and Bhutan. In each case, the imagery has been used both to directly map landslides and to examine the occurrence of factors that might be important in landslide initiation, such as water seepage. The results from the imagery were bench-marked using field surveys.

The results of this study demonstrated that Landsat ETM+ continues to be the most cost-effective imagery tool for mapping landslide susceptibility. This is due to the fact that it is relatively low-cost and has high spectral resolution. However, the researchers state that the spatial resolution is still a significant limitation to Landsat ETM+.

The high resolution, multispectral IKONOS imagery is not limited in the same way. With IKONOS, even small landslides are able to be mapped in great detail. However, according to the report, this type of imagery is less useful for factor type mapping. The report concluded that of the high cost of IKONOS will prevent most developing countries from developing and utilizing the tool, resulting in Landsat ETM+ being the better tool for mapping landslide susceptibility.

Sources: http://www.securinglivelihoods.org/nepal/files/nepal%20case%20study.pdf

Processing Satellite Imagery To Detect Waste Tire Piles

2010-04-13T05:30:00.004-04:00

Summary

This article, posted on www.techbriefs.com, discusses the development of a new methodology in satellite imagery analysis. The developers, Joseph Skiles, Cynthia Schmidt, Becky Quinlan, and Catherine Huybrechts, state that they have created a new methodology for processing commercially available satellite spectral imagery to identify and map waste tire piles in California. The methodology uses a combination of previously commercially available image-processing and georeferencing software. This is then used to develop a model that identifies tire piles.

Tire piles are difficult to distinguish in satellite imagery because of their low reflectance levels. Often times tire piles are mistaken for shadows or deep water. The developers of this methodology claim it attempts to correct these misinterpretations by using software that implements the Tire Identification from Reflectance (TIRe) model. The development of the TIRe model incorporated lessons learned in previous research on the detection and mapping of tire piles by various methods of analysis (manual/visual and/or computational) of aerial and satellite imagery.

Strengths

Reduces time spent surveying regions for tire sites

Weaknesses

This methodology still requires a trained analyst to go over the tentative findings and discriminate between tires, water, vegetation, etc.

Source: http://www.techbriefs.com/component/content/article/2486

Spatial Information Technologies in Critical Infrastructure Protection

2010-04-13T05:17:00.005-04:00

Summary
This document prepared by the National Consortia for Remote Sensing in Transportation(NCRST), addresses the necessity of remote sensing technologies in protecting critical transportation infrastructure. The document discuss the varying concepts on defining critical infrastructure, threats to the infrastructure, protection of infrastructure and disaster management needs. The document then discusses the critical role of remote sensing technologies. While they have become a vital tool in critical infrastructure protection, the systems are limited by the users and their status.

Strengths
Remote sensing provides low cost, multi-purpose, wide area “at-a-distance” data. That can assist in modeling, identification of critical infrastructure. It can also provide powerful visualization to assist when shaping policy.

limitations
While remote sensing data is useful, it may be unusable due to timeliness and expertise required to convert data. Some sensing may also be limited by its sensitive nature, limiting its use. Additional concerns stem from the interoperability of data sets and systems.

Source:http://www.ncgia.ucsb.edu/ncrst/research/cip/CIPAgenda.pdf

High-Resolution Satellite Imagery and the Conflict in Sri Lanka

2010-04-13T01:22:00.003-04:00

In May 2009, the Science and Human Rights Program of the American Association for the Advancement of Science (AAAS) acquired and analyzed commercial high-resolution satellite imagery of the Civilian Safety Zone (CSZ) and surrounding area in northeastern Sri Lanka. The project was done at the request of Human Rights Watch and Amnesty International, who expressed concern over the status and safety of civilians due to the heavy fighting occurring 9-10 May, 2009. Comparing the May 6 and May 10, 2009 images of the CSZ, AAAS found significant removal of IDP shelters. In addition, imagery showed evidence of bombshell craters, destroyed permanent structures, mortar positions, and 1,346 individual graves. AAAS’s analysis was based on images from various publically accessible commercial satellites, US Army Field Manuals, and open-source information from public statements and media reports.

Strengths - Satellite imagery analysis is a useful way to assess the situation on the ground during conflicts in which no outside parties are allowed in the area.

Weaknesses – None

Source: http://shr.aaas.org/geotech/srilanka/srilanka.shtml

Landsat Satellite Images Change Detection Methods

2010-04-13T00:06:00.005-04:00

Minnesota's Department of Natural Resources (DNR)'s website depicts its use of Landsat Thematic Mapper (TM) images to map Minnesota land cover. Landsat TM images are digital and differ from traditional digital images in their ability to express additional measures of brightness beyond simple RGB brightness. Specifically, Landsat records an additional four sets of brightness from the near, middle, and thermal infrared portions of the electromagnetic spectrum. Landsat images are useful in showing changes between two images taken of the same area but years apart. Image differencing, or subtracting the original image from the new image to determine changes, is a simple technique in remote sensing.

Preparatory Steps

Both images should show the same season and must be accurately registered (matched up) to the ground and to each other. Remove areas with clouds and distinguish between forest and nonforest areas. The images must be radiometrically calibrated to minimize effects of instrument variations and atmospheric haze.

Analysis

Values in the difference image that are red or orange show loss of vegetation, while greens depict growth of vegetation. The example uses a simple bell curve in which minor changes in vegetation account for 80% of the total changes and significant changes occur on either side of that distribution.

Strengths

Relatively easy to perform
Objective in nature

Weaknesses

Chance for error if preparatory steps not followed
Does not account for other factors of vegetation change between the images
Unlike satellite image analysis, it is not possible to compare analysis units and determine which has shown the most vegetation loss in this case, since the statistical thresholds display roughly the same amount of change and non-change regardless of what has actually happened in the area.

Source: http://www.ra.dnr.state.mn.us/changeview/change_tech.html

Tsunami Satellite Image Analysis Reveals Dramatic Water Quality Changes

2010-04-12T22:18:00.004-04:00

Satellite Imagery Analysis

Summary:
An article in XPress Press, via Applied Analysis Inc. describes how Satellite Imagery Analysis was used to determine water quality levels after the Tsunami that hit Sri Lanka and India. Applied Analysis Inc., an American company, used satellite imagery analysis processes originally designed for military use, to determine the clarity of the water. Applied Analysis Inc. used the IKONOS imagery software for their analysis, and believes that the technology they used will be able to better identify other problems with water supplies in the near future.

Strengths:

Can be used on multiple types of problems/issues.
Continually upgraded technology.
Has relatively clear data that is visible.

Weaknesses:

None noted.

Link

http://www.xpresspress.com/news/AppliedAnalysis_011305.html

Satellite Image Analysis Reveals South Ossetian Damage

2010-04-12T17:42:00.008-04:00

Satellite Imagery Analysis

Summary:
An article from ScienceDaily on October 9, 2008 discusses the usage of satellite imagery analysis to determine the damage done in the South Ossetian during the Russian-Georgian conflict. The article uses analysis done by the American Association for the Advancement of Science at the request of Amnesty International USA. The AAAS study examined damage to 24 villages near the city of Tskhinvali, which is considered the main city in South Ossetia. The AAAS looked at the damage of structures on August 10th, and compared them to the damage on August 19th. They determined that Tskhinvali sustained the greatest amount of damage (182 structures) between the 10th and 19th of August. The details of how they believe the buildings were destroyed corroborated the stories from the ground that fires were the main cause. The AAAS study combined eye-witness accounts of destruction with objectively interpreted satellite images purchased through three commercial vendors: GeoEye, DigitalGlobe and ImageSat International. They also used the software packages ERDAS Imagine and ArcView to leverage the power of remote sensing technology and Geographic Information Systems (GIS), respectively.

Strengths:

Used to corroborate reports on the ground.
Can be very detailed.
Easy to compare images from different dates.

Weaknesses:

Need of trained professionals.
Software can be expensive.

Source:

http://www.sciencedaily.com/releases/2008/10/081009144105.htm

A Comparison of Aerial Photography, Landsat TM and SPOT Satellite Imagery

2010-04-11T16:29:00.002-04:00

A study was done on the tropical wetland environments of northern Australia; population growth and environmental problems are encroaching and threatening the wetlands. Because of the remoteness and fragility of the area, remote sensing was an appealing method for obtaining information about the land. The purpose of the study was to investigate the utility of image data sets to see which type of image produced the best resolution for mapping the environment.

Method:
1.) In the study, Landsat, TM (satellite images) , SPOT XS and PAN (remote sensing satellites) and large-scale, true-color aerial photography were evaluated for mapping the vegetation.

2.) Five sample points were placed at 1km intervals. Each point was labeled and information about the location was recorded. To ensure the correct cover types were recorded, both photographic and written records were collected at each site. This procedure provided 12 observations per sample point and a total sample size of 240 observations for the entire swamp.

3.) Landsat, TM, SPOT, and large-scale photography was used for each location.

4.) Images were evaluated.

Results:
The study concluded that aerial photography was superior to satellite imagery for detailed mapping of vegetation environment studied.

The results suggest that either Landsat TM or SPOT XS imagery is adequate for mapping these generalized land cover classes. But the resolution needed for the classification of the vegetation was acquired through the aerial imagery. When comparing the two satellite imagery data sets, the broader spectral range of the Landsat TM data appeared to more than compensate for the superior spatial resolution of the SPOT imagery.

The researchers note that the most useful technique will depend on the application and is closely related to the physical characteristics of the features being mapped.

Study can be accessed through EBSCOhost:
Vegetation mapping of a tropical freshwater swamp in the Northern Territory, Australia: a comparison of aerial photography, Landsat TM and SPOT satellite imagery by K. R. HARVEY and G. J. E. HILL

Geographic Business Intelligence

2010-04-11T15:28:00.002-04:00

The geographic-search company MetaCarta, which was originally funded by the Department of Defense and others in the government sector, is now directing its technology at businesses. MetaCarta introduced an enterprise feature for GIS that organizes data geographically and allows for visualization between structured and unstructured data. This allows the user to search database using a map as a filter. Claudine Bianchi, vice president of marketing at MetaCarta states that in corporations, 80% of unstructured data has a geographic reference; whether it is an exact location or any type of point of reference.

MetaCarta is targeting specific industries where geographic information is a critical part of the business, such as the critical infrastructures, educations, and fraud detection.

MetaCarta sites this example for its use in the business sector: “Before an oil company commits to spending millions of dollars to drill a new well, geographic intelligence could be used to comb silos of information about that specific location, illustrating on a map whether the area has been drilled previously, when the lease expires, and other relevant trends about the site.”

MetaCarta's GIS technology comprises three components:

Geographic Text Search - combines text and geographic search capabilities for structured and unstructured data that can then be displayed on a map

GeoTagger - provides XML metatagging of geographic sites so they can be stored, and indexed

Geographic Data Modules - a collection of places, names and coordinates with assigned significance values and natural-language processing based on business or topic

Damage Detection From High Resolution Satellite Images For The 2003 Boumerdes, Algeria Earthquake

2010-04-11T01:31:00.003-04:00

Application of Satellite Imagery Analysis:

The researchers used imagery from a high resolution commercial satellite, QuickBird, to assess the damage to buildings that resulted from an earthquake of magnitude 6.8 that struck the coast of Algeria on May 21, 2003. The study focused on the two cities of Boumerdes and Zemmouri. Using photographs from before and after the event, researchers coded buildings as either a 1-5 based on level of destruction. The higher the number, the greater the damage. They also noted the location of tents constructed to house refugees. This study proves the efficacy of remote damage assessment in the field of disaster management.

Strengths:

Satellite photos cover a large area at one time
More manageable than on-the-ground classification efforts

Weaknesses:

Hard to compare images taken from different angles and at different resolutions
Shadows can affect interpretation

Results:

The study revealed that destruction classifications based on 2 images (from before and after the event) were more stable and consistent across five raters than those based on just the photo from after the earthquake. The precision level was 80% meaning that the raters were in agreement about the level of damage to a structure for 80% of the buildings. Similarly, the number of damaged buildings was greater using both sets of photos than just the ones depicting the earthquake's aftermath. The use of both sets of images was more crucial for buildings with low-grade damage.

The authors also compared their satellite-aided results to those from a team of Algerian engineers conducting classification of building damage on the ground in Boumerdes. The comparison indicated that overall, people using satellite images to assess damage tended to under-estimate the degree of structural damage. The authors did not quantify this effect, but noted that it warrants future study.

Source:
http://staff.aist.go.jp/m.matsuoka/others/13wcee_yamazaki_kouchi.pdf

Using A Time Series Of Satellite Imagery To Detect Land Use And Land Cover Changes In The Atlanta, Georgia Metropolitan Area

2010-04-09T01:36:00.004-04:00

Application of Satellite Imagery Analysis:

This article detailed a study in which researchers used a series of Landsat MSS and Landsat TM images from between 1973-1998 to assess the impact of land use/cover change on temperature change and air quality in Atlanta. The study indicated disturbing trends of deforestation and urban sprawl, but it also yielded worth-while information about the use of satellite photograph interpretation.

Strengths:

Provides an excellent overview of a region
Allows a retrospective view
Can provide highly accurate results

Weaknesses:

Relies on interpretation
Limited by resolution, image quality, atmospheric haze, and contrast

How to (Steps):

Geometric Rectification- align landmarks (necessary for analysis over time)
Radiometric Normalization (RRN)- necessary when images were taken by different sensors, to standardize radiometry, or contrast within an image
Design of Classification Scheme- decide on classification categories ex. cultivated land or grassland
Image Classification- divide the photograph into "clusters", spatial areas with the same characteristics, then decide into which category they should be placed
Spatial Reclassification- an effort to reduce errors in classification involving techniques such as having a human check portions of computer-classified work
Accuracy Assessment- compare classification results to what the ground areas truly are as revealed by aerial photographs

Accuracy of Method:

The study results stated that user accuracy was over 80% in 5 out of 6 land-type classification categories. The accuracy for the sixth was in the high seventies. Landsat TM images which have a higher resolution yielded slightly more accurate results than those from the Landsat MSS images especially for high and low density urban areas and forests. User accuracy for these categories was 96%, 98%, and 98% respectively.

Source:

http://vk.cs.umn.edu/sboriah/LandCoverBib/YangYL2002.pdf

Green Team Summary of Findings: Red Teaming (4 out of 5 stars)

2010-04-08T13:50:00.003-04:00

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 8 April 2010 regarding Red Teaming specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description

Red Teaming is an interactive process conducted during crisis action planning to assess planning decisions, assumptions, processes, and products from the perspective of friendly, enemy, and outside organizations. Red Teaming has also been described as the "capability-based analytical or physical manifestation of an adversary, which serves as an opposing force."

Red Teams evaluate a target or tactic, but not the likelihood that a particular target will be attacked. Successful red teaming offers a hedge against surprise and inexperience and a guard against complacency. Effective red teaming can define the threshold of detection, suspicion, and action.

Red Teaming is the only type of alternative analysis method that is mandated by congress. SEC. 1017 of the Intelligence Reform and Terroism Prevention Act of 2004 states that "Not later than 180 days after the effective date of this Act, the Director of National Intelligence shall establish a process and assign an individual or entity the responsibility for ensuring that, as appropriate, elements of the intelligence community conduct alternative analysis (commonly referred to as ‘‘red-team analysis’’) of the information and conclusions in intelligence products." (Source: http://www.nctc.gov/docs/pl108_458.pdf )

How to:

1) Determine the objective or desired result

2) Communicate with government, private partners, or other stakeholders in the process

3) Determine the scale or type of exercise and develop the scenario

4) Create a Red team composed of Subject Matter Experts, external to the Blue team’s sources.

5) Preparation by the Red Team. Team members should immerse themselves in learning everything they can about what has gone before in the crisis at hand and what the enemy and other adversaries are thinking. (Perhaps by creating a checklist of the information that the team needs to know.)

6) Meeting between the Red Team and Blue planners to explain critical points of the Red Team’s purpose, in order to alleviate friction.

7) Conduct and evaluate the exercise

8) Prepare documentation

9) Evaluate the performance

10) Develop the improvement plan

11) Make required and desired improvements

12) Exercise again

Tips for Success:

At least three people serve on each team

Team members must have no significant prior connection with the company that is presenting.

Devote the necessary time and attention to the process.

Red team members should be given at least a week to read the materials to be used in the presentation and do a bit of personal research.

Strengths and Weaknesses

Strengths

Lets Players consider the system as a whole

Reduce Risk

Avoid Predictable Patterns

Preclude mirror-imaging

Perturb the organization

Overcome bias

Improve adaptability and flexibility

Yields a closely synchronized planning staff

Reveal overlooked planning opportunities

Provides confidence to the Blue team

Provides an independent capability to evaluate concepts, plans, and operations from multiple different perspectives

Provides and understanding of the opposition through their eyes

Weaknesses

If not everyone agrees on the value of the exercise, it can become ineffective

The process may lose its independence and be “captured” by the bureaucracy

Could be too removed from the decision-making process and become marginalized

The process may destroy the integrity of the process and lose the confidence of decision-makers by “leaking” its findings to outsiders

There must be trust of the blue team for success of the exercise

It is only a simulation and is not always an accurate representation of the enemies decision making

The process is only effective if there is a true understanding of the opponent

The process does not account for independent thinking of the opposition

It can take time and effort to step back and view the system like an outsider, or even an insider who intends to harm

Personal Application of Technique

The class was divided into groups: Red Team/Blue Team and two referees. Red Team (offense) had 8 members and Blue Team had 4 members. Red had 1 safety card and Blue had 2 which would protect the members if they were within arms length; the way to determine who was "out." The simulation was over when either the Red Team captured the VIP or the Blue Team defended at the end of the alloted time. There were rules on specific ways to move; 90 degree angles, straight lines, and a minimum of 5 steps and max of 10. The main portion of the application was spent planning each players movement because all movements had to be written down before the simulation began. At the end of ten minutes, both teams met to engage in the simulation. Both teams moved at the same time and enacted the plans. After the simulation (Red Team captured the VIP within four moves) all groups met to discuss what both sides could have done differently and what either team was more likely to do.

Summary of Findings (White Team): Red Teaming (4 out of 5 Stars)

2010-04-08T13:47:00.003-04:00

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 8 April 2010 regarding Red Teaming specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Red Teaming is a broad range technique that covers both methods and modifiers depending on its utility and depth. Congress mandates the use of Red Teaming in National Security fields as referenced in the Intelligence Reform and Terrorism Prevention Act of 2004 SEC 1017, "Not later than 180 days after the effective date of this Act, the Director of National Intelligence shall establish a process and assign an individual or entity the responsibility for ensuring that, as appropriate, elements of the intelligence community conduct alternative analysis (commonly referred to as ‘‘red-team analysis’’) of the information and conclusions in intelligence products." (http://www.nctc.gov/docs/pl108_458.pdf) However, its use extends to law enforcement and competitive intelligence fields as well.

Based on current research and articles, there is no universal definition. It can cover ideas as simple as playing devil's advocate, and as complex as a full scale war simulations conducted at the National Training Center at Fort Irwin Military Reservation for the United States Military.

Strengths:

Applicable to National Security, Law Enforcement, and Business sectors.
Reduces risk by a means of internal auditing
Precludes mirror-imaging
Mitigates surprise
Avoids predictable patterns
Helps overcome bias
Improves adaptability and flexibility
Helps players view system as a whole,as well as individual components
Identifies decision maker choices for strategic players
Helps prevent bad investments- time, effort, money, resources
Improves the quality of questions asked about particular situations
Provides "awareness training" and improves safeguards of a system, particularly in an IT or computer networking situation
Challenges taboos and assumptions
Revealing the consequences of different perspectives; in-particular the perspectives of those with different goals and risk profiles

Strength is dependent upon the team compiled; composition, goals, management support, relationship with Blue Team, rules of engagement, and available information.

Weaknesses:

There is not one agreed upon definition
Full extent of an opponent's actions may not be considered
Red team may not take their responsibilities seriously
Could lose its independence and be “captured” by the bureaucracy
Red Teamers may not be allowed to act outside of Blue Team norms
Suggestions of Red Team may not be incorporated into the organizational structure without proper follow-up
Members of the Red Team may not be able to access the same knowledge as the real attackers
Red Team may not accurately represent real opponent's decision making process

How To:

Determine the objective or desired result.
Communicate with stake holders involved in the exercise including management / decision makers on the scope, scale and type of exercise.
Based on the exercise, create a Red team composed of Subject Matter Experts, external to the Blue team’s sources.
Preparation by the Red Team. Team members should learn everything they can about what has gone before in the crisis at hand, the blue team's plan and what the enemy and other adversaries may be thinking. (Perhaps by creating a checklist of the information that the team needs to know.)
Meeting between the Red Team and Blue planners to explain critical points of the Red Team’s purpose, in order to alleviate friction.
Red team creats a plan / Course of Action (CoA).
An exercise / simulation is conducted (Ex: A War Game).
Exercise is evaluated and improvements identified.
The required and desired improvements are incorporated.
Exercise and evaluate again till the desired objective is reached.

Application:

Our class played a game with 4 players comprising the Blue Team, 8 players comprising the Red Team, and 2 referees to enforce the rules. It was similar to "Capture the Flag" in that the Blue Team's goal was to defend an object and the Red Team's goal was to capture the object. We conducted the exercise within the confines of our department's building. The rules of the game were as follows: both teams must remain within the building; before the game began, both teams were required to create a plan of attack and could not deviate from that plan once play began; the teams could divide their members and start from any of the four entrances to the building; once play began, each player on each team must move at least 5 steps but no more than 10 steps (a step is defined a heel-to-toe); players must move in straight lines or at 90 degree angles; if two players come within arms' length of each other they are both eliminated, unless they reveal a safety card which protects the owner from elimination one time only.

Here is how the game played out: the Blue Team, having only four members, took their position surrounding the object and worked their way outwards trying to cover each entrance to the building, but that strategy proved ineffective given the Red Team's strategy to overwhelm one entrance and use the other three as decoys. Therefore, the Blue Team only had one player to defend against five Red Team players. Obviously, once the Red Team eliminated the lone Blue Team defender, they easily won the game.

In our debrief, it was obvious that the Red Team would be victorious given their advantage in the amount of players they had. It was possible for the Blue Team to prolong the game, but eventually they would be overwhelmed by the Red Team. However, an interesting aspect came up: just because the Red Team won, does that mean that they do not need to alter their strategy for the future? It's a question that we believe should be asked when performing Red Teaming exercises.

Seeing Red: Creating a Red-Team Capability for the Blue Force

2010-04-06T11:24:00.003-04:00

According to Colonel Gregory Fontenot, US Army retired, red-teaming is vital for understanding the ways enemies will fight. In his article, he explains that Red-teaming has been used since the 19th century as a tool (combined with war-gaming) and has continued to improve decision making when confronting and responding to adaptive enemies. After the 2003 Defense Science Board's (DSB) study on Red-teaming commended the added value from such exercises, various initiatives were employed to conduct red-teaming; however, the DSB could not could not find a commonly agreed-on description of red-team capabilities and functions or a means to assure quality of effort. The solution, according to Fontenot, was the University of Foreign Military and Cultural Studies (UFMCS) at Fort Leavenworth, Kansas, curriculum for education, training, and practical experience for red-team leaders and members. He further highlights red-team capabilities, benefits, and reasons for failures which are outlined below.

Red-Team Capabilities:

Expand problem definition
Challenge planning assumptions
Provide independent view of friendly and enemy vulnerabilities
Provide understanding of adversary through his cultural lens
Identify 2nd- and 3rd-order effects of plans
Reveal overlooked opportunities
Anticipate strategic implications
Provide alternate courses of action
Coordinate scientific and technical examinations

Red-Team Benefits:

Reduce risk
Preclude mirror-imaging
Mitigate surprise
Perturb the organization
Avoid predictable patterns
Overcome bias
Improve adaptability and flexibility

Red-Team Failure (reasons for):

Does not take its assignment seriously
Could lose its independence and be “captured” by the bureaucracy
Could be too removed from the decision-making process and become marginalized
Could have inadequate interaction with blue teams and be viewed as just another
sideline critic
Could destroy the integrity of the process and lose the confidence of decision-makers
by “leaking” its findings to outsiders

Conclusion

Red-teaming is an excellent tool for understanding and dealing with adaptive enemies. The value of red-teaming has been acknowledged by the U.S. military and has caused re-organization to incorporate specific curricula to educate and train red-team leaders. Red-teaming is often combined with devil's advocacy and war-gaming and is most effective when used with other tools/techniques.

Source
Colonel Gregory Fontenot, Military Review, Sep-Oct 2005
Accessed via http://www.au.af.mil/au/awc/awcgate/awc-sims.htm#redteams

Red Team, Blue Team: How To Run An Effective Simulation

2010-04-06T10:51:00.002-04:00

The military does it. The Government Accountability Office does it. So does the NSA. And the concept is making its way into the corporate world, too: war gaming the security infrastructure. Originally, the exercises were used by the military to test force-readiness. They have also been used to test physical security of sensitive sites like nuclear facilities and the Department of Energy's National Laboratories and Technology Centers. In the '90s, experts began using red team-blue team exercises to test information security systems.

This is one of the easiest ways to identify security vulnerabilities, and it also helps with an issue key to any successful red team-blue team exercise: buy in. Yes, it's one of the most overused phrases in a consultant's vocabulary, but the approval of management and employees is essential when testing information security systems.

The goal of a red team-blue team exercise is not just to identify holes in security, but to train security personnel and management.

Weakness: If not everyone agrees on the value of the exercise, it can quickly devolve into defensive posturing and wasted time. After all, you may be asking higher-ups for the time and budget required to fix flaws the exercise discovers.

An attacker will disregard more than rules; he or she will disregard the company's norms. Consider who your attackers may be. Power plants may be targeted by terrorists. Banks by criminals. Anyone by a disgruntled ex-employee. It can take time and effort to step back and view the system like an outsider, or even an insider who intends to harm.

Strength: One of the values of a tabletop exercise is that it lets players consider the system as a whole. Most companies that don't house nuclear materials are unlikely to engage in full-scale physical exercises with armed forces storming their building, but it's important to consider physical security when developing whiteboard attacks. A tabletop exercise provides the opportunity to reflect and assess response options as well as attacks. And then think about what possible breaches might mean. "This gives the blue team, the defenders, confidence," says Assante. "It's also very useful to the red team. You see vulnerabilities in a whole new light. And they bring that training back" to their coworkers.

Once you've fixed the holes your whiteboard exercises identified, however, a live attack-and-defend exercise can provide a whole new level of insight, but it's not an activity to be taken on lightly. In some cases, vulnerabilities can be safely demonstrated on a live corporate network, but it's not wise to launch a real attack against your production systems.

Examples: Even at National Labs, employees are often the weakest link in a security plan. But even if you don't have to worry about employees copying classified material onto home computers, it's important to think about how an enemy could exploit weaknesses in your employees' behavior. Do they prop-open automatic doors? Click on e-mail attachments from strangers? You can test for these problems and similar ones. Assuming you have a written security policy and employees are aware of it, you may not want to announce a red-team exercise, since your goal is to determine the risks of normal behavior. Other managers have left USB devices lying around office buildings to see who picked them up and plugged them into their computers. They've also sent phishing e-mails to employees to see who would take the bait.

Conclusions: "Many people migrate from a wired network to a wireless one assuming it works exactly the same, because from their perspective it does work the same," explains Sandia Parks. "They don't realize that there are different characteristics that provide different attack surfaces."

"Red-teaming is good at helping the customer understand interdependencies," says Clem, who advocates bringing a red-team mentality to design decisions. He wants his clients to think, How does that added functionality affect security? What could the bad guy do if we do that?

Hypergame Analysis, Part 1

2010-04-06T10:27:00.006-04:00

In situations of competition and conflict, no single player can dictate the outcome. What occurs depends on the strategy each player pursues. In turn, the strategy each player pursues depends on the strategy each player believes his or her opponent will pursue, and so on. Analysts often use game theory to model such situations.

In 1977, Peter Bennett introduced hypergame analysis, an elegant and useful extension to game theory. Unlike standard game theoretic models, Bennett’s concept permits players to perceive different games. This feature better approximates real-world conditions and, in particular, allows analysts to model situations involving manipulation, stratagem, and deception more directly.

In hypergame terms, a situation in which both players correctly perceive the same game is designated a level-zero hypergame. A situation in which both players believe they are playing the same game while at least one player misperceives the game is designated a level-one hypergame. A situation in which at least one player perceives the other player’s (assumed) misperceptions is designated a level-two hypergame.

Example:
A scammer offers a great deal to a mark on the street: “My friend’s business has failed,” says the scammer, “and I’ve got a van full of DVD players I need to sell quickly at a great price.” The mark hesitates. Maybe they’re stolen, he thinks. He decides to take a look anyway. The scammer opens the back of a van containing stacks of boxes. He opens one to reveal an off-brand but slick-looking portable DVD player. “This is yours for $20,” he tells the mark, who weighs the opportunity. The stuff’s obviously boxed, the mark tells himself; maybe it’s not stolen after all. He ignores his initial misgivings, hands over a twenty, and walks away with a mint-in-the box DVD player, or so he believes. When he gets to his car, he eagerly opens the box and discovers a brick. He drives back to the scene of the crime, but the scammer is gone. This situation is easily described using the hypergame framework. The mark assumes the two are playing the same game. In this game, the scammer’s options are {(sell a stolen player) (sell a legitimate player)} while the mark’s options are {(buy a player) (walk away)}. The mark’s challenge, then, is to decide whether the players are stolen. If the mark doesn’t care either way, then the choice is easy: buy a player. The scammer is playing a different game. For the sake of this example, let’s assume the scammer keeps a couple of real DVD players handy in case he suspects the mark might blow the con. In the scammer’s game, then, the mark’s options are {(buy a player) (walk away) (blow the con)} and the con’s options are {(sell a broken player) (sell a working player)}. If all goes well for the scammer, the mark never suspects (1) the scammer is playing a different game and (2) the scammer is playing a higher-order game–that is, the scammer is not only playing a different game but is aware of the mark’s misperceptions. This yields an advantage to the scammer. As long as the mark doesn’t suspect that most of the boxes contain bricks, he believes his choice is simply an ethical one: should I buy possibly stolen merchandise ? The concept of higher perspectives is sometimes referred to as expectation.5 Expectation is arguably as critical to the hypergame approach as is the more basic concept of different games.

Weaknesses: Hypergames of level three and higher are possible but challenging because they require increasingly convoluted mental recursions (I think he thinks I think he thinks, and so on). Hypergame analysis as described in the existing literature can be fairly complex. It is not something an interested analyst or red teamer will typically pick up in a day. When it is used, it is usually delegated to a specialist, who must then translate the outcome back into terms a decision maker can absorb.

Strengths: As the example illustrates, the player who correctly perceives a level-two hypergame enjoys a clear decision advantage over a player who believes the two sides are playing the same game. This situation does not necessarily arise by chance, and a clever player will aim to create and exploit such conditions. As a result, the benefit of hypergame modeling to a red teamer or decision maker rests not strictly in describing a situation but also in modeling a situation explicitly in order to gain a position of advantage.
Awareness of the hypergame construct encourages a player to avoid granting his or her opponent a position of advantage.

Conclusions: In any game-like contest, a player should always remember to ask “what do I perceive, and what does my opponent perceive?” To be avoided, for example, are states in which you, a player, believe you and your opponent are playing the same game when your opponent is actually playing a level-two hypergame. To be sought are states in which these roles are reversed.

Red Teaming: A means to Military Transformation

2010-04-06T10:11:00.005-04:00

In John Sandoz article was written to address the task ordered objective of joint operational concepts and joint experimentation to assist the Department of Defense(DoD) in attaining the objectives of Joint Vision 2020.

The Role of Red Teaming in Defining Threats
Sandoz writes that in an uncertain security environment, the DoD needs to consider future threats from three perspectives:
1) The evidentiary threat- are studied collected against, analyzed, and reported on exclusively by the intelligence community.
2) Technically feasible threat - are becoming more difficult to asses. This is mainly due to the fact that potential opponents can readily purchase many technologies and systems abroad.
3) The adaptive threat- is difficult to define because U.S. military forces may not know the unexpected ways in which its opponents could counter war-fighting capabilities.

Sandoz makes the point that in all three cases red teaming should play a key role in developing and evaluating potential threats.

Advantages of red teaming:
1) If done at an inter-agency level , adaptive Red Teams can also inform the national policy process by examining alternative strategies and the roles of government agencies, allies, and non-governmental organizations (NGOs) can play in achieving national policy goals.
2) As part of a disciplined process of red-blue interaction , vigorous red teaming can inform the transformation process and lead to a more robust and relevant future military capabilities.
3) Red teaming and joint experimentation can help prevent bad investments and at the same point, provide a means for U.S. forces to become more agile.

Challenges of red teaming:
1) Prevailing cultures and processes that are intolerant of surprise.
2) Adaptive red teaming does introduce elements of uncertainty and can be disruptive to individual programs.

Three levels where interactive red teaming could support joint concept development and experimentation:
Level 1- Red teaming could challenge ones strategic context and visions of future military capabilities by inventing and exploring counter-strategies and challenging scenarios.
Level 2- Red teaming could challenge new operational concepts in ways that future adversaries might use to thwart U.S. military forces in accomplishing their assigned missions.
Level 3- Red teaming activity could be in direct support of experimentation, including the OPFOR for specific experiments. Red teaming at this level would develop context and concepts for opposing specific Blue operational concepts.

Conclusion:
Sandoz concludes that a broader approach to red teaming , featuring a disciplined process of Red-Blue interaction, could inform and help guide transformation at several levels of the national security process in the three levels as stated above. He goes on to write that because war is a phenomenon between thinking opponents, a broad approach to interactive red teaming is important to inform one's thinking about future military challenges and explore ideas for dealing with them .

The “Red Team” Forging a Well-Conceived Contingency Plan

2010-04-06T09:31:00.004-04:00

Editorial Abstract: Independent peer review by recognized experts is crucial to the production of any quality product, whether a professional journal or war plan. Colonel Malone and Major Schaupp discuss evolving efforts to use “Red Teams” to incorporate this kind of review into the crisis-action planning process. Employing such teams at critical phases during both the planning itself and the mission rehearsal of completed plans will yield more robust and vetted war plans.

Throughout the lengthy planning effort for Operation Allied Force in 1998–99, allied leaders and planners widely adhered to a significant assumption. When the order arrived to execute the operation- on the very eve of hostilities- that assumption continued to prevail.

What if an enemy, “Red,” announced his intended reaction to a “Blue” campaign plan before Blue executed it? What if Red obligingly pointed out the flaws in Blue’s plan that he intended to exploit and revealed several hidden weaknesses of his own? Surely, once Blue optimized his strengths and protected his vulnerabilities, the operation would stand a much greater chance of success.

Furthermore, what if representatives of the press and the public confided to Blue planners the elements of the operation that concerned them most as well as those with which they might take issue? What if national leadership explained in advance some of the “wrenches” they might throw into the works during execution? What if senior war-fighting commanders and higher headquarters staffs worked alongside the planners to ensure correct understanding of every facet of their guidance and answered the planners’ key questions? If all these pieces of information were synthesized into the plan during the planning process, the plan would have a better chance of surviving any contingency.

For example, Gen Gregory S. Martin, commander of United States Air Forces in Europe (COMUSAFE), tasked his command’s first Red Team to assess an offensive air and space campaign. After analyzing requirements and considering the restrictions imposed by the “need to know,” the Red Team leader formed the team with SMEs from the following areas:

• air operations and strategy
• command and control (C2)
• joint operations
• logistics
• space operations and strategy
• intelligence, surveillance, and reconnaissance (ISR)
• combat search and rescue
• information operations and information warfare
• law
• politics

Weaknesses:
When possible, the commander should draw Red Team members from sources external to the Blue planning organization. Although this may seem intuitive, it is not always easy to accomplish. Most organizations that have the necessary experts are usually fully employed- indeed, the Blue planning orga-nization itself is a perfect example. A commander may be tempted to dual-hat his or her own Blue planners as Red Team members; after all, what better people to assess a plan than the ones most intimately familiar with it? But this seemingly workable solution is fatally flawed: one of the prime benefits of Red Teaming is an independent review of Blue products and reasoning- a second set of eyes on the plan. Try as it might, even the most talented planning group cannot discern its own oversights- if it could, those oversights would not occur in the first place. As concerned as Blue planners must inevitably be with the details, it is sometimes difficult for them to stand back and see the big picture.

Strengths:
When one considers the overall mission of the Red Team- generating a more effective plan- it becomes clear that the team is not consistently “red.” At times, rather than challenging Blue reasoning, its members will provide assistance to the planners, offering another perspective or additional information. This is especially true of the senior mentor, a vital participant in the process although not technically a member of the Red Team. This periodic functional shift on the part of the Red Team- from devil’s advocate to planning partner- does not detract from the overall effort. On the contrary, it broadens the range of thinking and contributions of the entire group, enhancing the planning effort.

Red Team Rules of Engagement
As the Red Team prepares to integrate into the planning effort, it must acknowledge a simple fact: very few people perceive a review and assessment of their efforts as benign. Even assistance, which is ultimately what the Red Team provides, is often not welcome, especially when it comes from people unknown and external to the Blue planning team. To mitigate this friction, the Red Team should meet with the Blue planners as early as possible to explain a number of critical points about a Red Teaming effort. The following ROEs should apply to every Red Teaming event throughout the process:

• The commander’s perceived intent should not limit innovation (e.g., drive certain COAs).
• Red Teaming events are meant to be interactive, candid discussions reminiscent of the flight debrief after a mission.
• The principle of nonattribution is in effect.
• Participants should remain objective in their contributions to the effort; personal agendas or personality conflicts are not welcome.
• Participants should stay professional- no fighting in public.

The first item in this list addresses a problem that can be insidious and deadly to a well-developed plan: the natural tendency to favor a war-fighting commander’s perceived intent in developing COAs. Too often, a planning staff presents the commander with several COAs, knowing full well that all but the perceived favorite are throwaways. As a result, staffers sometimes spend little time seriously developing the COAs.

As the Red Team moves into action, its ability to gain the confidence and trust of the Blue planners is absolutely critical. Failure in this area will lead to Red Team failure. One cannot overstate the importance of avoiding an “us against them” relationship between Blue and Red. Again, the commander’s early buy-in and influence in this area, as well as adherence to the ROEs outlined above, will pay large dividends to the process. When this groundwork is laid successfully, the Blue team will understand why the OPFOR, for instance, is doing its utmost to simulate a realistic, hostile enemy.

Conclusions
USAFE’s early Red Teaming efforts will continue to evolve. Development of the commander’s Red Team becomes more focused with each effort. One thing is already clear- Red Teaming adds great value to contingency planning. It would likely do the same for deliberate planning. Air and space staffs should consider the doctrine already in place, as well as the ideas expounded here, with a view toward making Red Teaming a staple of the planning process.

Field Marshal Helmuth von Moltke’s adage “no plan survives contact with the enemy” is true. But through Red Teaming, a plan can be refined after each contact with a Red Team. This process is valuable because it brings a contingency plan, together with the reasoning and information behind it, under the scrutiny of a well-simulated enemy. Better still, the Red Team can imitate outside agencies, higher headquarters, and even “Murphy’s Law.” A plan that survives this kind of treatment should be healthy indeed. To modify Gen George S. Patton’s famous quotation, “A good plan, well rehearsed, is better than a perfect plan unrehearsed.”

Red Dawn: The Emergence of a Red Teaming Capability in the Canadian Forces

2010-04-06T08:29:00.005-04:00

In Matt Lauder's article , written in the summer of 2009 for the Canadian Army Journal, set out to briefly identify and explore examples of red teaming from across the private and public sectors. By using these examples he was able to draw from them in order to outline the characteristics of red teaming and propose an integrated, and working , definition of red teaming for the possible use by the Canadian Forces (CF).

Lauder breaks red teaming down into 3 sectors:
1) Civilian applications
2) Military applications
3) Red teaming in the CF

1) Civilian applications

Some examples of red teaming in civilian applications include Jack Davis teachings at the Sherman Kent Centre, National Security labs within the U.S. Department of Energy , and the Forensic Audits and Special Investigations Team (FSI) of the U.S. Government Accountability Office (GAO) have all conducted similar penetration tests using this technique of red teaming.

2) Military Applications
John F. Sandoz teaches red teaming at the Institute for Defence Analysis .

Red teaming is also being taught at the University of Foreign Military and Cultural Studies (UFMCS).For the UFMCS, the goal of red teaming is to enable planners and decision-makers to avoid group-think, mirror-imaging, and cultural miscalculations.

3) Red Teaming Canadian Forces
In this article it is pointed out that red teaming is done in a much
more informal and irregular manner, and more often in a tactical-training setting. The CF did use red teaming as a technique to help them prepare for the 2010 Winter Olympics.

Why Red Teaming is important?
Lauder outlines two main reasons as to why red teaming is important:

1) Red teaming mitigates complacency, group-think, and mirror-imaging (i.e. imposing blue force
behaviours and tactics on the adversary; in other words, seeing the adversary as we
see ourselves).
2) Red teaming is a process by which blue force may be able to deepen its understanding of, and therefore the ability to respond to, the adversary.

Conceptual Framework of Red Teaming:
Red Teaming broken down into four broad and generic organizational processes:
1) Innovation
2) Planning and Analysis
3) Training and Professional Development
4) Operations

Lauder's six key characteristics of red teaming:
1) Trust
2) Positional Authority
3) Relative Independence
4) Expertise
5) Adaptability
6) Flexibility

The following are a number of areas that require further investigation:

1) What are the qualities and characteristics of good and effective red teamers and
how are red teamers selected?
2) What type of training is required for red teamers?
3) Is there a particular red team composition that is more effective than others?
4) What kind of learning environment is most effective?
5) Does the role of the red team differ in certain environments (i.e. does the role differ
across settings and levels)?
6) What type of interaction is necessary (between red and blue) to encourage
learning?

Conclusion:
In general, civilian applications tend to use red teaming on the tactical level (in particular, but not exclusively, to test physical or synthetic networks, systems, or operational programs), whereas military applications tend to be employed on the operational and strategic levels, and largely within a planning setting or in a decision-support role (although, in the CF, red teaming appears to be most often utilized in exercise or training environments). It is clear that, while application of the red teaming concept may differ across sectors, both the civilian and military communities utilize red teaming in an active, rather than a passive, fashion, and that red teamers must possess a deep understanding of the adversary (i.e. thinking and behaviour) for the purposeof role-playing the adversary (or, advising as to what the adversary may think and do) in training, planning, or operations (i.e. live) setting. Moreover, it is apparent that red teamers must see themselves as

Effectively Using Red Teams

2010-04-06T07:36:00.003-04:00

Dave Herndon's article details how companies should use Red Teams to get their proposals to win contracts.

When to Review
This presents a dilemma to many firms because the later they review proposals, the more robust the review may be. On the other hand, reviewing a proposal late may leave insufficient time to implement the recommended changes. Herndon argues that the best way to avoid this issue is to conduct more than one review during the process.

Herndon identifies three types of red teams:
1) Evaluating-and-Recommending Fixes Red Team - reviews the proposal for a broad range of factors, including:

Compliance
Completeness
Responsiveness Presentation
Sell

This team makes recommendations on how deficiencies can be fixed. Normally, this team does not possess customer expertise to formally score a proposal with solicitation evaluation factors but it can provide an informal quality score (excellent, good, marginal, and unacceptable) of each section for general evaluation purposes.

2)Customer-Evaluation-Simulation Red Team - attempts to simulate the customer's formal proposal evaluation process

This team measures proposals by:

Evaluating each solicitation requirement
listing proposer benefits and deficiencies
identifying needed clarifications for each solicitation requirement

In order for this red team to score a proposal effectively, its members must have a comprehensive understanding of the customer's requirements including the budget for the proposed work and political agendas.

3)Running Red Team - When a proposal is on an extremely tight schedule, a running red team is often an effective method of proposal evaluation. When a writer completes a section draft, he or she immediately gives it to the running red team for a quick response evaluation.

Composition of Red Team
Avoid using senior company executives on the red team unless they agree to give full-time effort to the review. The most important member of a red team is the red team manager. The ideal individual is someone totally familiar with the proposal review process, the proposal preparation, and the customer's requirements.

Red team members normally include:

Outside proposal professionals
Customer specialists
Employees who thoroughly know the bidder's capabilities, products, services, and past performance history
Subject matter experts

Red Team Planning Procedures
The red team evaluation should be planned early. The capture manager, proposal manager, and red team manager should then determine the type of red team to be used, its exact function, and a list of desired red team members. Red team procedures should include a breakdown of tasks for each red team member and a schedule for red team activities.

Preparing the Proposal for Red Team Evaluation
The most important thing in preparing a proposal for review is having it complete. Herndon recommends that the proposal be given a hard edit prior to red team review and a detailed compliance matrix should be included. This matrix should be in a check-off-list format that follows the requested information of the solicitation proposal instructions, evaluation factors, and statement of work.

Red Team Evaluation
After receiving the proposal, the red team evaluation procedures will include:

Final assignment review
Finalization of review schedule
Coordination with proposal team for debrief and follow-up actions
Review of total proposal against the solicitation requirements
In-depth review of assigned sections, noting deficiencies and strengths, identifying needed clarifications, providing recommendations, and completing evaluation forms
Compilation of comments into single book
Debrief of proposal team

The red team should meet and prepare a formal debriefing to the proposal team. The red team should concentrate their presentation on a realistic approach to improve the proposal.

Post Red Team Evaluation Actions
After the red team evaluation, the red team members should assist the proposal team in making the recommended fixes. When this responsibility is understood in advance of the review, the red team comments will invariably be more realistic.

Red Teaming: The Art of Ethical Hacking

2010-04-06T05:43:00.006-04:00

In Chris Peake's article, he discusses information security broadly but also how Red Teaming may be used by companies to perform security assessments on their networks/systems. There is an "Infosec Process" containing five components:

1) Assess the current state of risk by evaluating the existing security methods, measures and policies.

2) Based on the Assessment, design a security posture by creating policies that effectively manage the risk to the system/network.

3) Identify and implement the technical tools and physical controls necessary to manage risk.

4) Provide awareness training to the company to protect sensitive information through the cooperation and involvement of the employees.

5) Audit the system/network to confirm that the controls and employees adhere to policy.

This is a revolving process that should be performed continually by companies according to Peake. Red Teaming falls under the assessment stage of the Infosec Process (#1). The Red Team uses tools to probe for vulnerabilities and can project possible threats based on the scope of the assessment requested by the customer. However, the Red Teaming approach is attempting to circumvent security only need to find a single vulnerability, while security professionals need to find all possible vulnerabilities for a given system in order to assess the associated risk. A thorough Red Team assessment should provide an accurate situational awareness of the security posture of a given system/network. But identifying risk through Red Teaming and other methods cannot provide information security alone; the company/organization must continue through the Infosec process in order to appropriately manage risk and provide security protection.

A Red Team assessment evaluates various areas of security in a multi-layered approach. The Red Team tests policy compliance of the security controls at each layer (Operating System, Application, Host, LAN, Perimeter) and the control is tested in a manner specific to the area of security to which it applies. There are six areas of security where vulnerability assessment testing occurs:

-Internet Security
-Communications Security
-Information Security
-Social Engineering
-Wireless Security
-Physical Security

Red Teaming is “ethical hacking.” As such, it must be carried out with the utmost confidentiality, discretion, and clarity. Typically, Red Teams are third-party entities hired to make an impartial assessment of the network or system. The customer sets the scope of the project to specify the area of information to be assessed. The Red Team is responsible for supplying the customer with a detailed plan as well as a list of methods and tools that will be used during the evaluation. Any testing performed outside the scope stated by the customer, can be considered an unwarranted attack by the Red Team.

The most important requirement for Red Teaming is customer consent. Because, by definition and purpose, the Red Team takes an attacker-like approach to testing security, to begin an assessment without explicit permission is legally perceived as an unwarranted attack on the system/network. This being said, many Red Team evaluations are purposefully kept from network and system administrators as a means of testing personnel response to security events. The scope of the Red Teaming assessment can be very general or very specific when defining what the assessment will include or address. The scope of the project depends on time or cost of the assessment and/or on the objective of the assessment as defined by the customer.

Red Teaming is commonly mistaken as just penetration testing (pen-testing) when in fact, pen-testing is a component of the Red Teaming assessment. But pen-testing cannot provide a complete security analysis alone. If a system/network is penetrated, the test proves that there is at least one vulnerability that can be used to gain access to the system/network. And if the pen-test was unsuccessful, the test only proves that the person performing the pen-test was unable to find any exploits in the system, it doesn't guarantee that there are no vulnerabilities present.

A good rule of thumb for companies to follow when planning Red Team assessments is to identify the weakest areas or the "low-hanging fruit" and have these areas tested for vulnerabilities. Hackers will target a specific vulnerability to gain access (rather than numerous) to avoid detection.

Ethical hacking must strictly follow pre-approved testing guidelines that are established with the customer. The team must also document all the steps/procedures in testing in order to retrace the team’s actions in case of an incident due to testing or for retesting/verification of results if necessary. Upon completion of the Red Teaming effort all results should be submitted to the customer in a final report detailing the vulnerabilities that were discovered and how each was discovered. The report should also make an assessment of the overall level of risk of the network/system in addition to the risk level of each vulnerability. The final report is as important as the testing itself because it will direct the customer to take additional security steps.

Finally, when assembling a Red Team it is important to have specialists in a wide variety of areas (Peake lists 21 separate specialties) in order to provide the most thorough security assessment.