Wednesday, September 14, 2016

Summary of Findings: Devil’s Advocacy (3.5 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in September 2016 regarding Devil’s Advocacy as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.


Devil’s Advocacy is an analytic modifier that tests assumptions and critiques existing analysis. This is done through the use of a specific individual or team that actively looks for potential inaccuracies or problems within an analysis. This can also be done by the advocate(s) supporting a minority point of view and championing it versus the prevailing analysis.


  • Works with unstructured data
  • Provides alternative viewpoints if the Devil’s Advocate is an expert and at the same level as the rest of the team.
  • It can be simple to set up and create, and is flexible enough to use in any environment.
  • Can help to uncover biases and assumptions in reasoning


  • May not be successful if the Devil’s Advocate role is not formalized and  respected within the team
  • Has the potential to create tensions in analytical teams
  • If done incorrectly, it has the potential to unnecessarily reduce confidence (by intelligence consumers) in an adequately supported estimate
  • May not work if team members are unable to accept or perceive the ideas of the Devil’s Advocate
  • If pressed for time, it is difficult to adequately implement


  1. Designate a person/team to look at a previously established assumption or analysis.
  2. That person/team then proceeds to use available data and experiences to challenge the assumption or analysis by providing alternative explanations, outcomes, or reasonings that are just as likely to occur given the information available.  
  3. Present these finding to an individual or group willing to actively listen to the Devil’s Advocate’s findings.

Application of Technique:

A group of Graduate Students at Mercyhurst University developed a Devil’s Advocate exercise about North Korean nuclear development strategies.  Various types of role-played national security  intelligence assessments were developed: OSINT, HUMINT, SIGINT, GEOINT, IMINT.  See “National Security oriented Devil’s Advocate Exercise used in class [North Korea]” below.

In class the participatory team challenged “All-Source Team Black’s” overall estimate on North Korean nuclear weapons and developed reasons to accept or reject the overall analysis for it’s legitimacy of sources and built in key assumptions.  Instructions on how to use the process were issued from the “CIA’s Analytic Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis” Also, students were able to provide personal input on their personal understandings of the situation.  The team had 15-minutes to work through this problem.

For Further Information:

National Security oriented Devil’s Advocate Exercise used in class [North Korea]:

CIA Analytic Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis, pg. 17:

Devil’s Advocate Wikipedia:

Red Team: How to Succeed By Thinking Like the Enemy, pg. 216-219:

1 comment:

  1. Allegorically, one who takes an opposite position for testing a contention, or just to be perverse.

    The term 'Devil's advocate' was brought into English in the eighteenth century from the medieval Latin expression 'advocatus diaboli'. To describe someone as a Devil's advocate now is to suggest that they are mischievous and opposing, being opposite for it. In medieval Europe, Devil's advocate wasn't seen so contrarily; it was, similar to "chamberlain" or 'cordwainer', a vocation title.

    There are various mentions in Vatican records dating from the mid 1500s of a casual part called 'Diaboli Advocatus'. In 1587, the administration of Pope Sixtus V (disappointingly, there hasn't yet been a Sixtus the Sixth) established the formal post of Promoter of the Confidence, referred to informally as the 'Advocatus Diaboli', which surely must have been the same part as 'Diaboli Advocatus'. The set of working responsibilities wasn't especially onerous, until the point when someone was assigned for either beatification and canonization, and soon thereafter the 'Devil's Advocate' was required to draw up a list of arguments against the chosen one getting to be plainly blessed or consecrated.

    The first occasion when that the present type of the expression was used in print appears to be in the 1760 humorous content Impostors Identified:

    By rising up and having the genuine impact of the Devil's advocate.