Summary of Findings: Elicitation (4 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in October 2016 regarding Elicitation as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use structured data.

Description:

Elicitation is a modifying technique for collecting intelligence. Elicitation is used to discreetly gather information. It is a conversation with a specific purpose: collect information that is not readily available and do so without raising suspicion that specific facts are being sought. It is usually non-threatening, easy to disguise, deniable, and effective. The conversation can be in person, over the phone, or in writing. Conducted by a skilled collector, elicitation will appear to be normal social or professional conversation. A person may never realize they were the target of elicitation or that they provided meaningful information.  

Strengths:

• Can acquire important information that cannot be retrieved by conventional methods
• Is flexible and useful in different types of situations
• Can be used to supplement another type of collection technique to gather more information
• Provides the ability to create and build future relationships and networks

Weaknesses:

• Difficult to pull of without alerting the other party to your intentions
• Can potentially cause a significant operation security risk if collection is detected
• Language barriers limit the efficiency and effectivity of the method.
• Difficult to replicate

How-To:

• Identify a target with the information you wish to acquire.
• Obtain background information on the target to identify talking points
• Plan ahead how you wish to get the information from the target without raising suspicion
• Approach the target in a non-threatening environment and strike up a conversation
• Walk the target through your pre-planned conversation in an effort to extract the knowledge you require
• Once you’ve elicited the information you required, walk the target back to a more broad topic of conversation to disguise your line of questioning

Application of Technique:

Students in a class were asked to participate in a role playing exercise where there were four designated collectors and four designated targets.  Both the collectors and targets were given background information on one another but the differences in personal mission for the exercise.  The collectors were given a specific question they were try to find an answer to through indirect means.  The targets were given some piece of information that they were trying to protect.  At the end of the experiment the collectors were debriefed on the information they were able to gather from their targets.

Elicitation Exercise: The Diplomatic Party

The US embassy in Warsaw is hosting a party. The party members will be four members from the White Team. The party also has four members from the Green Team who will play the part of a respective adversarial intelligence-gathering component.

The Green Team will try to use elicitation techniques to gain information from the White Team members.

White Team: come up with a character you will play during the party. You can chose anything that you believe will be at a diplomatic party such as military officer, diplomat, government official, industry representative (rare but if relevant or invited), or any other.

Once you have a role in mind, develop a short backstory and a piece of information that your Green Team counterpart will have to get. Just make it up, it does not have to be very real to life, but should be plausible.

Green Team: The information you make for your Green Team person should be enough to give them exactly the question they need to use with the elicitation process as well as some basic information to sort of speed the process along.

White Team:

Major General Roman Popov, Russian Ground Forces

Major General with Russian diplomatic mission in Warsaw

Elicitor - US intelligence collector under diplomatic cover (top tip, do not open with that) Question:

What is the Russian Army’s current plan OR most likely plan for Ukraine operations?

Do not allow Popov to know what you are after.

Popov’s Secret:

The Kremlin is waiting to see what the new President does, but the sanctions are starting to strain the military. Barring anything severe, plans are being made that the Russian Armed Forces will quietly start to withdraw from Ukraine in mid-2017.

Vladimir Norov

Consultant of the Office of the President of Uzbekistan

Elicitor - Cover is a Chinese diplomat, but secretly is a Chinese intelligence operative.  

Question

Will Uzbekistan strengthen its security ties with Russia?

Note: Elicitor NOT ALLOWED to ask question directly or will lose exercise.

Vladimir Norov’s Secret:

Russia offered to forgive Uzbek debt if the interim President agreed to slow exports to China by half, or 15%.  Norov knows the interim President does not like the Chinese and will almost certainly accept the offer and strengthen ties with Russia.

Mary Smith

Member of the Conservative Party of the United Kingdom (UK)

Elicitor - Cover is an American diplomat, but secretly is an intelligence operative for the Labour Party of the UK (The Labour Party is one of the UK’s two major parties, along with the Conservative Party).  

Question:

What is Theresa May’s main negotiating goal for Brexit?

Mary Smith’s Secret:

As long as other Brexit requirements remain, Theresa May is most willing to negotiate a deal which includes the UK staying in the European single market after Brexit.

Cpt. James Patterson, US Army

Commanding Officer of the Military Police Unit assigned to provide security to the U.S. embassy in Warsaw.

Elicitor - Cover is as an American journalist, but secretly is an intelligence operative for the Russian Government

Question

(Primary) What are the security measures in place at the embassy as far as troop strength and weaponry?

(Secondary) What is the evacuation plan for the Ambassador during an emergency?

CPT James Patterson’s Secret:

(Primary) The embassy is currently being guarded by one squad (12) of active duty Army MPs equipped with crappy M16s and M9 pistols. The rest of the soldiers are off duty in their barracks located in the building next door to the embassy.

(Primary) The soldiers are tired, complacent, and are ready to redeploy home after a long year of embassy security in which nothing happened.

(Secondary) The escape plan for Ambassador Jones involves an underground tunnel which he is to be rushed through which leads to the back of a shoe store a quarter mile away from the embassy. Two MPs are stationed at the rear of the shoe store to secure the location.

For Further Information:

Spy Game Movie Training Sequence:

https://www.youtube.com/watch?v=dbixeLf-98w

Collection Information:

https://www.fbi.gov/file-repository/elicitation-brochure.pdf

The Scharff Technique: On How to Effectively Elicit Intelligence from Human Sources

http://www.intersolglobal.com/wp-content/uploads/2015/11/Scharff-technique-on-how-to-effectively-elicit-intelligence-from-human-sources.pdf

Hanns Scharff:

https://en.wikipedia.org/wiki/Hanns_Scharff

Military Interrogation Techniques Training - Intelligence Gathering Tactics/Methods (1943):

https://www.youtube.com/watch?v=EO9vaVqhr38

Military Interrogation Manual: FM 34-52

https://www.loc.gov/rr/frd/Military_Law/pdf/intel_interrogation_may-1987.pdf

Elicitation Counterintelligence:

http://www.dss.mil/documents/ci/CIBooklet.pdf

Elicitation & Recruitment: Can You Recognize It:

http://www.dss.mil/documents/ci/CI-Elicitation.pdf

Human Intelligence in Law Enforcement:

http://www.spiaa.com/pdfdoc/SPI%20HUMINT%20Class-glassford.pdf

Elicitation of information and response biases of repressors, sensitizers, and neutrals in behavior prediction

Summary:

Martin Kaplan discusses the effects of interviewer biases in the processes of information elicitation in this article.  The author used participants to test the effects different personalities have on garnering information from others.  Interviewers were asked to make behavior predictions to test how accurately they were able to elicit information from a subject.

The author tested two main hypotheses:  differences in the interviewer's predictions can be explained by personality groups eliciting different information, and that if each group elicits different information, then the information would be consistent. The experiments would test if actual differences existed and were consistent.

Kaplan conducted three tests to attempt to isolate a bias variable and prevent differences in perception to affect the interviewers judgement when forecasting an individual's behavior.  The subjects were divided as 'repressors', 'sensitizers', and 'neutrals'. The subjects were divided based on their scores on the Byrne Scale of Repression-Sensitization test.  Twenty random subjects were selected from each group as 'judges' and each judge was paired with a neutral 'target'.

The author then explains the specifics of the testing and instructions given to all participants.  He explained how he analyzed each group's discussions and interviews and compares factors like speech time of judges and of targets and how participants rated their own personalities.  Kaplan made his own predictions on how he thought the judges predictions would differ.

What Kaplan found is that sensitizers tended to speak more in the interviews than their targets did. A significant difference in relevance to criteria behavior was not found between the groups.  Kaplan found that what he expected regarding the differences between behavior predictions: sensitizers were more negative, repressors were more positive.

Critique:

While this study shows that conclusions about information elicited from a subject can be affected by internal biases, the experiment was limited in scope.  It's important to mention that the researcher tried to counter potential external variables from affecting the study because this limited the affect that personal perception had on the information elicited from the subject.  This study is also pretty old, but because of the type of analysis it attempts to do it seems to be still highly relevant the methodology.

KAPLAN, M. (1968). Elicitation of information and response biases of repressors, sensitizers, and neutrals in behavior prediction. Journal Of Personality, 36(1), 84-91. doi:10.1111/j.1467-6494.1968.tb01461.x

Friday, November 11, 2016

Knowledge Elicitation Techniques in a Knowledge Management Context

Summary:

This journal article discussed and confronted the use of knowledge elicitation techniques in its application toward the knowledge management context of organizational practices. The authors mostly used a qualitative informative standpoint in this paper with a bit of a quantitative background due to the study being run by one of the authors in a "Knowledge   Engineering" course who had 20 years of experience. Beginning it all the authors first described the situation that much of the knowledge in a business is in the individual employees, and the issue is that organizations have trouble eliciting such information to the organizational level to gain competitive advantages. The knowledge that employees know but do not recognize that a business could benefit from is termed in the paper as tacit knowledge.

1. The authors then reviewed current literature and moved into the differences between Knowledge Management (KM) and Knowledge Engineering (KE). KM is the managing of knowledge in an organization, but most of the research delves into the organizational level, and not the individual level. Where the individual level is where much of the tacit knowledge is located. While KE is a multidisciplinary domain (i.e. cognitive science, knowledge elicitation, structuring, and formalization) that is a subfield in intelligent system development research, which offers methods to increase and process elicitation of knowledge from people(i.e. employees). Lastly, the authors state the two biggest reasons for knowledge gaps in not sharing is the lack of motivation of individuals to share, and organizations not providing opportunities or resources to share in businesses.

2. After discussing the differences between KE and KM and the shortfalls of each, the authors/researchers then put in their development to intermix KM and KE to cover the shortfalls of KM by mixing it with KE. Through this process the authors put forth the input that through the use of expert(s) and analyst(s) an organization (business) can decrease the information gap between tacit knowledge and sharing it. By using an analyst(s), whose job is to elicit information from the expert(s) (i.e. employees). The reason one must combine KE and KM is because KM does not use an analyst to bring out the tacit information, and an analyst can use elicitation methods to get at the tacit knowledge that could aid in giving an organization the upper hand.

3. A graphic picture chart depicting the types of methods an analyst can use to access the tacit knowledge of the expert is shown below. Each method has their own strengths and limitations, but the strongest one to note the researchers found was the use of the verbal protocol of “thinking aloud,” which allows the analyst to see the logic process an expert is going through. Therefore, it comes down to making sure the right method is selected for the right scenario at hand, so the analyst can get at the tacit knowledge of the expert that can aid the organizations competitive advantage.

 

In conclusion, the scientists acknowledge that the first step before using and selecting a method is to identify the distinctions between tacit verse explicit, and individual verse collective knowledge. This applied with the KE and KM techniques with the usage of an analyst and expert, will reduce the knowledge gap between sharing and tacit information that isn’t used. In the 21^st century knowledge assets are a fundamental issue when it comes to competitive scenarios and advantages, thus the development and deployment of such assets is a critical strategic factor for organizations.

Critique

I think the authors did a good job expressing the shortcoming organizations have when it comes to people not wanting to share information. I myself have seen it first hand, particularly when certain things that may cost very little can save a labor forces time in completing an objective, and yet nothing is done because people don’t want to put themselves out there just to ask for something. One shortcoming was the fact that when the researchers applied their intermix of KE and KM, it was in house; meaning one of the researchers took twenty years of experience they had and mixed it with practitioners who took the authors course and gave input for it in order to develop their data. They didn’t list any numbers of subjects or other quantitative figures to give their study more robustness. Therefore I think to improve the paper the authors should have included information regarding this.

Sources

Knowledge Elicitation Techniques in a Knowledge Management Context

Gavrilova, T., & Andreeva, T. (2012). Knowledge elicitation techniques in a knowledge management context. Journal of Knowledge Management,      16(4),523-537.<http://eprints.maynoothuniversity.ie/7052/1/TA-Knowledge-            elicitation.pdf>

A Walkthrough of Requirement Elicitation Techniques

Summary

The development and maintenance phases of software systems often encounter issues due to poor requirement planning, management, and execution. In this article, authors P. Sharmila and R. Umarani discuss various elicitation techniques used within the field of requirement engineering. Designing large and complex systems, requires the use of multiple phases such as, elicitation, analysis, specification, and verification. This paper places its main focus on the elicitation phase within the processes of requirement engineering. This article notes, “mistakes made in elicitation have been shown many times to be major causes of system failure or abandonment and this has a very large cost either in the complete loss or the expense of fixing mistakes.” Sharmila and Umarani state that “Elicitation is all about determining the needs of stakeholders and learning, uncovering, extracting and/or discovering needs of the users and other potential stakeholders.” Requirement elicitation techniques focus on the needs of the customers and users, so that systems are built as close to the exact specification requested. The goal of the authors in this article, was to describe requirement elicitation through the use of various techniques such as introspection, interviews, and surveys/questionnaires.

Introspection:

Introspection can be described as the process by which a reliance on one’s own observation, inner thoughts, and desires are emphasized. The authors criticize introspection because it mainly highlights the thoughts and imagination of the expert who is developing software instead of focusing on user needs. Combining both expert imagination and user requirements contributes to a stronger platform. While introspection alone is not a valuable technique for requirement elicitation, it becomes more attractive and desirable when coupled with other techniques.

Interviews:

Within interviews, elicitation acts as a key component for gaining the desired answers. Delivering the requirements of the organization in an interview and then asking the right questions, provides the employer with insight into the candidates past and their qualifications. While an individual may answer a question vaguely, it is the job of the interviewer to craft questions in a particular way in an effort to gain better knowledge of the applicant. Sharmila and Umarani specified that there are multiple objectives for an interview. Fact finding, fact verification, and fact clarification are among those listed objectives for an interview. 

Surveys/Questionnaires:

Surveys and questionnaires can be seen in a positive light for the purposes of elicitation. Useful in gathering data from large groups of individuals, surveys and questionnaires can focus on a fixed set of questions. Another positive attribute to surveys and questionnaires is the fact that they are structured and offer opened ended input tailored to a specific project.

Source:

Sharmila, P., & Umarani R. (2007). A Walkthrough of Requirement Elicitation Techniques. International Journal of Engineering Research and Applications, 1, 4.