How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis?
Summary and critique by Jared Leets
Summary:
Richards Heuer Jr discusses the three different approaches to Analysis of Competing Hypotheses (ACH) and four significant steps in the intelligence analysis process which include researching relevant information, organizing information to help with analysis, reviewing the information to make a credible evaluation, and finally writing the product.
Heuer Jr states that when searching for information, the purpose is to get the analyst to challenge his or her initial mindset. Researching alternative hypotheses helps the analyst broaden his or her search for information that they typically would not look for. When analysts initially work with ACH they consider it beneficial due to the fact that it forces them to think about evidence and outcomes that had never occurred to them. In the first step ACH is an easy strategy for questioning a complex problem. Every analyst should seriously consider questioning their traditional way of answering a question to a problem.
The second step is organizing the information. Two principles, decomposition and externalization, exist to help cope with human mental perception and memory. Decomposition refers to breaking down the problem while externalization refers to removing the problem out of someone’s head and writing it down in order to simplify the problem and show the variables. The purpose is to show all of the evidence and what type relationship they share if any. The three types of ACH manual, automated, and Bayesian all have their differences. The manual ACH views only the relationship between each individual piece of evidence and hypotheses. The matrix will assist the analyst in evaluating evidence that is most indicative in opposing hypotheses
The automated ACH allows analysts to place evidence in categories. The categories include date, type of source, credibility of source, and the relevance of the source. By sorting the evidence it makes it much easier for the analyst to put all his or her efforts towards the most convincing evidence. The Bayesian ACH places more importance on the relationship between the evidence and hypotheses. It typically has multiple sets of hypotheses, which tend to increase judgments that must be made in the analytical process.
Heuer Jr states that in the third step, analyzing the information, the automated ACH will sort the evidence by categories while the Bayesian ACH will give a mathematical algorithm that shows the analyst a possible answer. Sorting evidence by relevance in categories significantly helps with intelligence analysis. Heuer Jr explains that in the automated version an analyst can compare evidence from a clandestine source or open source can help reveal deception from a source. While the inconsistency and weighted inconsistency scores can produce an accurate estimate, the analyst must use their own judgment in the end. According to Bayesian ACH proponents, they claim that using an Inconsistency Score or Weighted Inconsistency Score is problematic since it relies on incomplete information. A Bayesian inference is ideal for making probabilities bases on judgments from intelligence analysts. However, it is very complex, time consuming, and requires a Bayesian methodologist to help the analyst when it is being conducted. Proponents state that this displays how complex an intelligence problem can be and that one single analyst cannot make a judgment call without help from other sources. In the final step, writing the product, Heuer Jr says each ACH has its benefits and can help the analyst in reinforcing what their research has revealed or shows the analyst an alternative path for additional research.
Critique:
ACH has it benefits and its obstacles. For example, it is very good at helping an analyst see alternative hypotheses and pieces of evidence and weighing that evidence to help make a correct decision. In any intelligence project an analyst must think from different points of views. ACH can help an analyst go step by step and see if their current hypothesis will answer the intelligence requirement. However, ACH can also have its issues. As Heuer Jr explains, when weighing evidence it is easy to be biased, especially when deciding how credible or relevant a piece of evidence is. In addition, software can only help an analyst come to a decision. If an analyst says that ACH told them the answer and base everything off of that, they will likely lose credibility. In the end, ACH can help with intelligence analysis by helping the analyst think of different ideas to address the problem and support their research, but it can also be a problematic if analysts rely too much on it as software cannot answer problems in the intelligence profession.
Source: Heuer Jr, Richards. (October 16, 2005). How Does Analysis of Competing Hypotheses Improve Intelligence Analysis? Pherson Associates. http://www.pherson.org/wp-content/uploads/2013/06/06.-How-Does-ACH-Improve-Analysis_FINAL.pdf
Posts
I do agree with you Jared that ACH is effective tool to use. The process helps assemble and organize the information in a manner designed to facilitate analysis and evaluate the information to make an estimative judgment. I also believe that ACH is helpful for intelligence analysts to overcome, or at least minimize, some of the cognitive limitations. I do find the Bayesian proponents of weighing multiple hypotheses helpful for an analyst to examine.
ReplyDeleteThis is a very informative article, I really like the point made about challenging one's initial mindset, which brings to mind the issue of cognitive bias. With all the imperfections of using the ACH method, at the very least, it gives the analyst the ability to see all the available/possible alternatives. Getting to the point at which a "credible evaluation" of the presented information needs to be made still remains critical and a challenge for the analyst.
ReplyDeleteThank you Pouch and Oddi. ACH can be very beneficial in the intelligence field. As both of you said it helps overcome cognitive bias, if ACH is properly used it can definitely assist the analyst in weighing other options for an answer. In addition, it can also help the analyst pursue another route if it seems more logical. The analyst must realize ACH's limitations to successfully use it.
ReplyDeleteACH is a powerful tool that can help analysts look at a problem in a different way and challenge biases that they have towards a specific problem. However ACH is too limited to be used as a sole methodology in an analytical work. This is due to the vast amount of influence that the analyst has on the information entered into the matrix while also assigning credibility and reliability. ACH should primarily be used a thinking tool to measure how an analyst feels about a problem given specific information. When used in conjunction with other analytic methodologies is where ACH is most useful as it is more difficult for an analyst to spread biases across multiple analytic platforms.
ReplyDeleteI like the use of the original investigation of ACH effectiveness for this blog post. Did Heur say how he would change ACH if he was to rework his model?
ReplyDeleteHeuer states the pros and cons of mainly the manual and Bayesian ACH. Many complaints come from the manual ACH, as it is very time consuming to use and that the Bayesian ACH probability calculation is based off of incomplete information and at times requires a Bayesian methodologist to assist the analyst through the process.
ReplyDeleteHeuer makes a good point when he states that the purpose of ACH is for the analyst to challenge his or her initial mindset. Many researchers often fall victim to Confirmation Bias, the tendency to seek information that conforms with previous beliefs. ACH is a useful thinking tool that allows the analyst to recognize their cognitive biases and challenge any preconceived opinions. It also allows the analyst to reveal and better understand their analytical process. Nevertheless, I agree with your critique that ACH has its limitations. When weighing evidence, it is easy to be biased with regards to relevance and credibility. I personally view ACH as a beneficial thinking tool in the beginning stages of research, helping us look at a problem more broadly.
ReplyDeleteI am in agreement with Ian. ACH is indeed a powerful analytical tool, and better used in tandem with other analytical methodologies, but only when rid of cognitive bias. Being aware of one's cognitive biases can be a difficult procedure on one's own. In my opinion, collaboration can be a powerful combatant of cognitive bias, but it can also act as a double-edged sword. Collaboration allows more insight due to the different ways we view the situations and alternative outcomes presented for us. In fact, it is recommended that analysts with different perspectives collaborate to brainstorm the possibilities. However, collaboration can also lead to other's cognitive biases projecting out to the analyst and/or groupthink.
ReplyDeleteJared, great summary! ACH is very useful tool, but it is just that, a tool. One of many. By itself, it should not be the sole analytic method an analyst uses, but instead provide a general estimate for a limited number of outcomes or at the very least, prompt an analyst to examine other outcomes. Again, ACH is a great tool, but as we've seen this week in the postings, it is imperfect no matter how one approaches the process. It is is very useful in forcing an analyst to confront bias with evidence, but depending on the specific model of ACH used, complications arise with each model.
ReplyDelete