Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University, in September 2018 regarding Analysis of Competing Hypotheses as an Analytic Method, specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.
“Analysis of competing hypotheses, sometimes abbreviated ACH, is a tool to aid judgment on important issues requiring careful weighing of alternative explanations or conclusions. It helps an analyst overcome, or at least minimize, some of the cognitive limitations that make prescient intelligence analysis so difficult to achieve.” - Richard Heuer, Central Intelligence Agency
Analysis of Competing Hypotheses (ACH) is a structured analytic technique designed to give analysts a framework for organizing information to formulate an estimate. ACH also is useful in resolving goal conflicts in a simple and repeatable manner. ACH gives analysts a skeleton in which to enter in definitive, contrasting hypotheses and assess the evidence on how it confirms or disconfirms the given hypotheses. The hypothesis with more confirming evidence becomes the more likely hypothesis. While ACH is a well-known methodology, it has not been thoroughly tested to assess its validity in academic literature.
- Consideration of alternative viewpoints beyond the ultimately proposed analytical theory
- Designed to simplify and analyze intelligence problems
- Mitigates cognitive biases
- Analyst’s reasoning is more transparent
- Easy to use
- Allows for categorizing and sorting of evidence in useful ways, e.g., time, relevance, source reliability
- Unclear how much evidence is appropriate/most effective
- Too much or too little evidence can negatively influence the analyst’s estimate
- Relationships between pieces of evidence unclear/unidentified
- Some evidence is only important when considered in conjunction with other evidence
- Doesn’t identify pre-existing assumptions
- ACH can eliminate necessary context
1. Identify possible hypotheses
2. Compile a list of evidence
3. Enter evidence into matrix
a. Hypotheses across the top
b. Evidence down the side
4. Determine whether evidence is consistent or inconsistent with each hypothesis
a. Consider (and weigh when necessary) the credibility of source for evidence
b. Consider (and weigh when necessary) the relevance of evidence to the hypothesis
5. Draw conclusion regarding which hypothesis is most consistent/inconsistent with the evidence included in the matrix
a. Use conclusion to support estimate
b. If conclusion does not support either hypothesis, collect more evidence and reevaluate ACH matrix
Application of Technique:
We wrote an investigative narrative regarding a crime and the analysts were asked to determine the likelihood that a suspect did or did not murder the victim. The narrative was presented to the analysts in paragraph form and evidence was included throughout the narrative.
After reading the narrative, analysts were asked to conduct the ACH as individuals, identifying their own key pieces of evidence and determining for themselves whether they were consistent or inconsistent with each hypothesis. After an allotted time to conduct their own ACH’s, analysts were then asked to share their conclusions with the class. The ensuing discussion brought forth issues regarding which evidence was important to the investigation, which evidence was most relevant to the hypotheses, and which sources for the evidence were most credible.
For Further Information:
1. Heuer on ACH: Chapter 8 of “Psychology of Intelligence Analysis” from the Central Intelligence Agency
2. Wheaton on ACH: Analysis of Competing Hypotheses and Structured Analysis of Competing Hypotheses: What Are The Links With Foresight?
3. Brasfield on ACH: Forecasting Accuracy and Cognitive Bias in ACH
4. Van Gelder on ACH: https://timvangelder.com/2007/12/31/hypothesis-testing-whats-wrong-with-ach/