Summary Findings: Argument Mapping (3 Stars Out Of 5)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 12 articles read in advance of (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 1 APR 2009 regarding Argument Mapping
specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Definition:

Argument mapping (AM) is an analytic modifier that can be used to examine the logic behind the development of a particular conclusion and/or hypothesis. The product of AM is a visual representation (typically a box-and-line diagram) of the reasons that support and oppose the claim. Constructing a visual depiction of a complex argument reduces the level of abstraction in evaluating a decision.

Strengths:

Provides Audit Trail

Breaks down prose arguments and visualizes arguments for decision makers.

Shows strengths and weaknesses of argument

Improves critical thinking

Reduces cognitive bias and blindspots

Weaknesses:

Not estimative in nature.

The method does not explicitly seek out contradictory evidence.

The language specific to argument mapping is not uniform from scholar to scholar.

Dependent on the quality of presentation of original argument

The technique does not consider the impact of deliberately deceptive information

How-To:

* Must locate a central claim for a position (this is the conclusion of an argument)

--conclusion indicators include: therefore, thus, so, hence

* extrapolate true and logistical reasons and objections which support or refute the central claim.

--rewrite statements as individual sentences

--Reasons must answer the question: "How do we know that [insert claim] is true/warranted?"

* include all premises for reasons - leave nothing to be implied

--premise indicators include: since, because, for, given that

* With objections, list any possible refutations underneath the objection

* Form a 'warrant' - a statement that justifies the step from a reason to a claim

--warrants reveals an arguer's core implicit beliefs

--used to scrutinize the soundness of an argument

--used to accept or reject controversial arguments

*Use arrows to indicate which premises or co-premises support claims; or premises refute oppositions.

*indicate premises that need to be combined in order to support a conclusion, and premises that are each seperate reasons to believe a conclusion

*Use a color-coding scheme (if possible) to help visualize the arguments

--Supporting claims - green

--opposing claims - red

--nuetral claims - gray or no color

Experience:
We attempted to argument map a one page NY times editorial. The comments below were generated as a result of that exercise.

Argument mapping is difficult to synthesize from prose that is not written in a BLUF (Bottom Line Up Front) format.

The ability to effectively argument map is too dependent on quality of the articles given.

People will disagree on structure of argument and levels of relevance of data.

Argument mapping is not appropriate to use to develop an analytic estimate, rather it can be best implemented as a modifier to validate an already determined estimate.

Argument mapping would be an excellent modifier to test the validity of a conclusion/forecast. Would be an excellent supplement to ACH - suggest conducting an ACH to find a claim that is most likely, then test that claim for validity using Argument Mapping. This will help to reduce cognitive blindspots and biases, as well as test the reasoning behind the conclusion.

Extrapolating the main claim of the argument was easier once all of the premises supporting it were laid out.

The original format of the 'argument" really matters when it come to the ease or difficulty of applying argument mapping to the text.

Cultivating Expertise In Informal Reasoning

http://www.philosophy.unimelb.edu.au/reason/papers/CJEP_van_Gelder.pdf

Summary:

Argument mapping technology was introduced in a quasi-study of informal reasoning of first year university students in Australia. Operating under the premise that informal reasoning "appears at quite an early age and continues to develop through secondary and tertiary education...few people manage to become highly proficient." Informal reasoning was likened to a social tennis player: without dedicated practice and guidance, the athlete will never reach full potential. Thus, after initial training "experience in a domain is not related to the level of expertise."

Van Gelder argues informal reasoning skills are achieved to the extent that one engages in large amounts of deliberate practice...to go beyond ordinary competence. He postulated an increase in informal reasoning practice would have a positive effect on overall reasoning skills. To test his postulation, he utilized argument mapping software as a method to used in conventional logic instruction. The main benefit of using argument maps was to "enhance the quantity and quality of feedback" in reasoning activity.

The results of his study indicate students made substantial gains in informal reasoning skills. Participants gained approximately as much over one semester as they would have over 3-4 years of university study. Also, the amount of gain was positively to amount of practice.

The author does admit the results do not establish that deliberate practice is necessary for advanced expertise in informal reasoning.

Decision Mapping Easier Than Argument Mapping?

timvangelder.com

Summary:
The explores the differences between decision mapping and argument mapping, two very similar techniques with a fundamental difference. Whereas argument mapping is applied to an argument, decision mapping is used to determine "choices between multiple possible actions." As an example of each, the author draws both a decision map and an argument map based on a recent New York Times article by Col. Muammar Qaddafi proposing a peace plan for the Middle East.

After applying both techniques to Col. Qaddafi's proposition (which was the creation of "Isratine" - a joint Isreali/Palestinian state), the author found that the decision map was easier to develop than the argument map. Part of this was due to the nature of the piece being analyzed : an article in someone else's words. "The translation from prose to decision map was much more straightforward than the translation from prose to argument map. In the latter case, there seemed to be far more discretion about how to do it, and hence a much higher level of effort and expertise was required to determine which of the approaches would be 'right' or best. "

The author expresses his surprise that the decision map was easier to develop, although he concedes that the subject of the exercise may have had alot to do with that. If decision mapping is, in fact, generally easier than argument mapping, he states that the following would be true as well:

• Decision mapping should find faster and wider uptake than argument mapping
• From a pedagogical or training perspective, decision mapping should be introduced first, with argument mapping treated as a more advanced subject.

In a comment to the blog, a reader notes that he, too, finds decision mapping to be easier to use than argument mapping. He points out "For some reason argument mapping seems to be so much more dependent on language and context and is perhaps more prone to making mistakes." He explains that it is possible to mis-represent the author of an argument when constructing the argument map due to the tendency to want to "close the gaps automatically even if the original text doesn’t necessarily support it. " He further states that the first step of the decision mapping process is the most crucial, i.e. "sizing up the situation appropriately." The author responds, stating that that first step is critical, although he has no advice as to how to make sure you go about it correctly.

No Computer Program Required: Even Pencil-And-Paper Argument Mapping Improves Critical Thinking Skills

Harrell, Maralee. Teaching Philosophy, Dec2008, Vol. 31 Issue 4, p351-374, 24p

Summary:
The author explains that after she "stumbled across" Tim van Gelder's argument mapping software Reason!Able, she became intrigued as to how little she truly understood her own arguments she made while teaching philosophy at Carnegie Mellon University. According to the author, "I found, to my surprise, that I did not understand these arguments as well as I thought I had, and that the mapping was forcing me to analyze and synthesize in a way that I had never done before." The author explains how, especially in philosophy, arguments are not linear, and by applying argument mapping techniques it is possible to analyze different areas of the argument that one may not have seen previously and identify area that are not as well developed.

The author notes several rules to be used in the development of argument maps. First, look for premise and conclusion indicators. Examples include "since, because, for, given that; and some
common conclusion indicators are: therefore, thus, so, hence." Second, rewrite statements as individual sentences. Third, make sure to include all premises and conclusions; leave nothing to be implied. Lastly, "clearly indicate the difference between premises that need to be combined in order to support a conclusion, and premises that are each separate reasons to believe a conclusion."

The author, after incorporating argument mapping via paper-and-pencil diagrams, saw a marked improvement in her students critical thinking abilities and analysis skills. In order to further test the validity of the method, argument mapping techniques were taught in 5 out of 9 introductory philosophy courses at Carnegie Mellon, whereas those methods were not taught in the remaining 4. The goal was twofold: "The first is that all of our students, no matter how they are taught, are gaining argument analysis skills by taking our introductory course. This is important to know if we are then going to inquire which students gained more. This hypothesis implies that, on average, all students in introductory philosophy will exhibit significant improvement on argument analysis tasks over the course of the semester. The second hypothesis is that the ability to construct argument maps that accurately reflect the text they are analyzing is a considerable aid for improving students’ argument analysis skills (more of an aid that being able to represent an argument some other way). This second hypothesis implies that students who are able to construct argument diagrams and use them during argument analysis tasks should perform better on these tasks than students who do not have this ability." Although some students did use computer software, most used the paper-and-pencil method.

The first hypothesis was confirmed: all students in the study, whether they used argument mapping or not, increased to some degree their argument analysis abilities by virtue of being in the class. The second hypothesis was confirmed as well: students who used argument mapping techniques, did in fact, improve their argument analysis abilities to a greater degree then those that did not employ the technique.

The author concludes by stating that although many tools are available to conduct argument mapping, the method itself is effective no matter what tool is used. In fact, various tools were used during the study at the discretion of the student and instructor. "In our course, instructors and students are free to use any kind of medium they prefer to build argument maps. Some instructors use computer software, while others use the chalk board or overhead slides; similarly some students use one of a variety of drawing programs, while others use just pencils and paper. Our results show that the argument mapping skill, no matter how the maps are produced, is an important part of the gains in argument analysis abilities our students achieve. While on average all of the students in each of the lectures improved their abilities on these tasks over the course of the semester, the most dramatic improvements were made by the students who were able to construct argument maps." Thus, the visual representation of the argument itself was more important than the tool used to develop that image.

Argument Mapping - The Basics

Argument Mapping - The Basics
based on the heuristics and Rationale software developed by Austhink

*Author's Note: Although this guideline does not delve into the pros and cons of argument mapping, it does give a good idea of how to construct an argument map - whether you are using this particular software, or if you are making an argument map with pencil and paper.

Summary:

The "Argument Mapping - The Basics" sheets provide the reader with a outline of understanding for what argument mapping is, the terms used in argument mapping and logic, as well as some important rules of logic that you must keep in mind when structuring an argument map. Argument maps start with a conclusion, which is at the tip of the pyramidal hierarchy, with reasons and objections listed below the conclusion. Reasons can have co-premises, and co-premises can have other reasons to support the claim listed above. Co-premises can also work together to support particular reasoning. Objections are listed to oppose the conclusion or reason and can have rebuttals listed underneath the objections.

Similar to games of strategy (chess, risk, etc.), there appears to be a learning curve with argument mapping. It takes some time to get the 'feel' of the game and to fully understand the rules, but with time, the process become quick and effortless.

Important information within the document:

Definition of Argument Mapping: "Argument mapping is a way to visually show the logical structure of arguments. You break up an argument into its constituent claims, and use lines, boxes, colors and location to indicate the relationships between the various parts. The resulting map allows us to see exactly how each part of an argument is related to every other part."

Other important definitions to know when creating argument maps:

• Argument: a claim and reason(s) to believe that that claim is true.
• Simple argument: the building block of all arguments, consisting of one claim and one reason (with two or more co-premises).
  
• Complex argument: has several simple arguments linked together (the diagram below illustrates a complex argument)
  
• Conclusion: the main point an argument is trying to prove, usually a belief. Also called the position, the main claim, the issue at hand.
• Reason: evidence given to support the conclusion.
• Co-premise: the subset of a reason. Every reason has at least two co-premises, and each of these co-premises must be true for the reason to support the claim.
• Objection: a ‘reason’ that a claim is false; evidence against a claim
• Rebuttal: an objection to an objection.

Syntax of an Argument Map:


Of note:

• Arguments can have many claims, many reasons, many objections and rebuttals, but only one conclusion.
• Distinguish a claim with a single reason (made up of two co-premises) from a claim with two independent reasons.
• The exact structure of an argument is very important. For example, if side A has two good reasons to conclude something, and their opponent (side B) thinks one of those reasons is bad, then A’s conclusion may still be true/warranted if the remaining, unobjected-to reason is convincing.
• An argument map can represent a debate by showing exactly where two sides disagree on the issue.
  
• Argument maps show the structure of the argument/debate – every box is not necessarily true, but the first step is to understand the structure of the argument.

Rules within each box:

• Declarative Sentence: Each box should have a full sentence (not a phrase) and should be declaring something, taking a position (whether it is true or false).
• No Reasoning: No box should have reasoning going on inside it, only single claims. The reasoning is represented by the arrows and locations in the map. Look for words that indicate reasoning (e.g. because) and translate the reasoning into the map.
• Two Terms: Each box can only have two main terms, so that each box is either true or false, not both. If you have more than two terms in a single box, separate them into multiple boxes.

Rules within each simple argument:

• Assertibility Question: All reasons for claims must answer the question: “How do we know that [insert specific claim here] is true/warranted?” You are asking what evidence allows one to assert that the claim is true. Every claim box should have a reason box below it that answers this question.
• Holding Hands: Applied horizontally within each simple argument. Within each reason, a term stated in one co-premise must be mentioned in one of the other co-premises in that same reason (if it is not in the claim above it – see the Rabbit Rule below). The terms must ‘hold hands’ within a single reason if they are not already accounted for by the Rabbit Rule.
• Rabbit Rule: Applied vertically, between a claim and each of its reasons, and is combined with the Holding Hands rule. “You can’t pull a rabbit out of a hat.” Using these two rules for each simple argument, you make sure that every term mentioned in each box is found in one of the others.
  

Argument Mapping (Efebia)

Summary
Argument mapping is a logical sequencing method that employs box-and-line diagrams to "map" a course of possible decisions for a given "argument" and the ramifications of each option. The purpose of the map is to provide a visual depiction of the relationships between the overall contention and the logical (evidential) claims supporting or opposing it. Upon examination of the supporting and opposing claims, an argument map should serve as a useful tool for use in the decision making process, by reducing the complexity of a dilemma, conflict, or argument. Since argument mapping distinguishes itself from other box-and-line methods by utilizing different shapes, colors, and positioning, it appeals to the strong visual comprehension abilities of humans, allowing for improved processing and understanding, ultimately simplifying the complexity of an argument.

Basic Construction of an Argument Map
The components of an argument map are listed below with a brief explanation. Those pieces of the map are identified in the example below (click on the map to link to the source and to access a higher resolution map).

• The main claim: referred to in this article as the "position" or "contention" -is the hypothesis to be logically examined. The argument map's purpose is to help the decision maker accept or reject this hypothesis.
  
• Reason: a positive claim, or one that directly supports the main claim. Reasons not only support the main position, they may also support another reason.
• Objection: a negative claim, or one that opposes the overall main claim.
• Rebuttal: opposes an objection directly above it; or, "objection against an objection".

Strengths of Argument Mapping
According to the article, by using colors, shapes, etc. in argument mapping, the mind can better process the complexity and abstract nature of a difficult argument. Additionally, the logical structure of argument mapping is easier to present than traditional prose, which requires the reader to deconstruct the argument for himself, demanding a large investment in time and cognition.

The article also cites "extensive research" conducted by the University of Melbourne to gauge the effectiveness of argument mapping in the context of critical thinking. The study compared the achieved critical thinking abilities of students who utilized the Reason!able (Rationale) argument mapping tool versus those who used traditional prose. The study determined that, in a 12-week course, students who used argument mapping gained 12 IQ points.

Author's Comment:
Since this article may have commercial implications (as it names a particular brand of software), it is obviously a strong advocate for argument mapping, and does not approach the method with much objectivity. While there may be some obvious disadvantages to using argument mapping in certain situations, none are provided here. Moreover, the only other method this article discussed was traditional prose, which is not an analytic method by itself, but a medium of production or dissemination. Despite a mere mention of conceptual modeling and flow charts, there was no substantive, formal comparative discussion on them.

Argument Mapping and Analysis of Competing Hypotheses (AM vs ACH)

Summary

This article presents a comparative assessment of argument mapping (AM) and Analysis of Competing Hypotheses (ACH) as analytic diagramming techniques. While the paper focuses primarily on the differing techniques in diagramming a decision making pathway, the author considers both as "complementary analytical frameworks".

Comparing the Techniques

Both techniques involve analyzing a given proposition (hypotheses in ACH; conclusions in AM), evaluating the evidence (evidence in ACH; reasons and objections in AM), and diagramming a visual representation of the relationships (matrices in ACH; box-and-line diagrams in AM). In ACH, the letters C, I, and N represent positive (consistent), negative (inconsistent), or nuetral relationships, respectively. In AM, arrows indicate a relationship between the reasons supporting or opposing the conclusion; color indicates whether the evidence is a reason (green) or an objection (red) for the argument. When a piece of evidence is neither a reason nor an objection (neutral or irrelevant), the creator of the AM may choose either to eliminate it from the map or to make it a different color (typically gray).

Multiple Propositions
Due to the manner in which each of these methods handles multiple propositions, ACH gets the advantage over AM. ACH incorporates multiple propositions simply by adding another column in the matrix. However, adding any additional propositions to an AM may result in a cluttered diagram with crossed lines and complicated pathways. To resolve this issue, it may be more appropriate to duplicate pieces of evidence, or even create an additional map, making it less efficient.

Multi-tiered Evidence
The paper uses an example from Psychology of Intelligence Analysis to demonstrate a situation in which AM may be a more efficient method. The chosen proposition is "Iraq will not retaliate forUS bombing of its intelligence headquarters," and the piece of evidence in question is "Saddam [has made a] public statement of intent not to retaliate". Often, as cited in this example, an additional piece of evidence may be necessary to establish a piece of evidence. When using an ACH, the analyst may need to construct another matrix to validate evidence. The AM technique allows for the addition another level of evidence within the same map. The paper refers to the multi-tiered evidence as "granularity". Granularity allows evidence of differing levels of abstraction to be present on the same map; "the higher the level on map, the more general or abstract the reason or objection".

Assumptions
AM also allows the user to add assumptions or warrants alongside the pieces of evidence. Incorporating an assumption into an ACH would force the user either to add a justification with the conclusion in its cell in the matrix, or combine multiple pieces of evidence together.

Enhancing Deliberation Through Computer Supported Argument Mapping

Tim van Gelder
Department of Philosophy, University of Melbourne, Australia; and Austhink


Summary
Tim van Gelder defines deliberation as "a form of thinking in which we decide where we stand on some claim in light of the relevant arguments." Although this is a common and important process, it is complicated and often conducted poorly. Gelder contends that deliberation can be improved by mapping out arguments, especially when the methodology utilizes the new computer tools available. An argument map is a presentation of reasoning in which the evidential relationships among claims are made wholly explicit using graphical or other non-verbal techniques. Argument mapping is producing such maps.

This fairly minimal or broad definition recommended by Gelder allows for enormous variety in argument maps. The point of an argument map is to present complex reasoning in a clear and unambiguous way, and mappers should use whatever resources work best. Currently, argument maps are mostly comprised of "box and arrow" diagrams. With technology expanding, other presentations are likely to count as argument mapping. For example, somebody may develop a way to present arguments in virtual 3D or through a virtual reality environment.

According to Gelder, at least four main factors explain the superiority of argument maps. These points concern the limitations of prose which are partly or wholly overcome by argument maps. 1) In prose, the reader has to figure out what the relationships among the claims are. In an argument diagram, in contrast, all relationships are made completely explicit using simple visual conventions. In practice, this relieves a huge burden. Readers can devote their mental energy to thinking about the argument itself rather than trying to figure out what the argument is. 2) Prose is a monochrome stream of words, sentences, and paragraphs. Prose does not use any color, shape, line, or position in space to convey information about the structure of the argument. We know, however, that our brains can process huge amounts of color, shape, and space information very quickly. In an argument map, color can be used to indicate in a matter of milliseconds whether a claim is being presented as reason or an objection. 3) Prose is sequential in nature. However, arguments are fundamentally not sequential. Arguments are more than just one thing after another; they are more complicated. 4) Using diagrams, we can to some extent take advantage of the way humans learn and understand. "We can place all the reasons over here and all the objections over there, or we can make stronger reasons bigger, or place them underneath (supporting) the conclusion."

Until now, argument maps have not really taken off as a practical tool for argument deliberation. Creating these diagrams by hand can be quite difficult. However, new computer software (both free and commercial) is making this method easier. New argument mapping pieces of software include Araucaria, Athena, and Reason!Able.

Argument Maps Improve Critical Thinking

Charles R. Twardy
Revised draft for publication in Teaching Philosophy
School of Computer Science and Software Engineering
Monash University, Australia

Summary
When Charles R. Twardy, a professor at Monash University, first heard about Tim van Gelder's Reason!Able argument mapping software, Twardy was quite skeptical about the effectiveness of the methodology. Rumors circulated that the new software had the ability to drastically increase the quality of critical thinking by van Gelder's students. Twardy contacted van Gelder and the two professors agreed that Twardy should visit van Gelder's university and teach one of his classes to see if students' critical thinking skills really do improve with the Reason!Able software (argument mapping) or whether the students benefit from the "founder effect."

The Reason!Able software for argument mapping amazed Twardy. He saw a significant improvement in the abilities of his students to think critically about arguments after taking a course on the Reason!Able software and the argument mapping methodology. Twardy concluded that "Computer-based argument mapping greatly enhances student critical thinking, more than tripling absolute gains made by other methods." (The gains, or scores, Twardy is referring to are those from the California Critical Thinking Skills Test)

The most significant advantage that argument mapping provides students is the ability to show precisely how students make errors in their reasoning, making it much easier for them to fix their errors. Specifically, argument maps help us to understand how arguments are structured. Typically, we do not make the distinction between two claims forming part of a single reason or whether they are parts of separate reasons. Prose does not force students to know the structure of arguments. Even if you understand an argument, you may not understand the argument's structure.

The second benefit to argument mapping is the methodology's versatility. Argument mapping is a general skill that can be applied to all kinds of arguments.

The major negative to Reason!Able and argument mapping is that users really need a class and significant practice to master the skill. Almost anyone who is asked to map a two-paragraph argument fails to do so correctly. Twardy argues that "practice is clearly important; argument mapping without practice would not much improve critical thinking."

Enhancing our Grasp of Complex Arguments

Enhancing our Grasp of Complex Arguments
By Paul Monk and Tim van Gelder
This paper was presented by Paul Monk as a plenary address to the 2004 Fenner Conference on the Environment, Australian Academy of Science, Canberra, 24 May 2004

Summary:

What argument mapping is used for:

• To structure, communicate, and correct arguments of any degree of complexity
• To govern deliberation, keeping it on task, target use of evidence, specify disagreements, and make the process more efficient
  

Verbiage tends to make people miss what is being said and asked and encourages people to grasp tightly to their own thoughts. Monk and van Gelder posit that the use of only language, writing processes, and mental cues are too primitive to completely understand the complex arguments that people are now faced with. They continue by stating, “We conduct complex arguments as if a combination of holistic apprehension, intuitive judgment and natural language were sufficient for handling them [arguments]. None of us, I think, would consciously make that claim. We do what we do by tradition and by default, not because we have thought through why we do it, how it works and whether it serves us well.”

Playing the game of tic-tac-toe (on a 4x4 grid or larger) without using actual gridlines is used to illustrate the point that our working memory struggles without the presence of a visual aid (the grid). Cognitive blind spots and biases, the methods used to record and communicate arguments, and separation of disciplines due to different idiolects all accentuate the problem of our limited working memory.

Just as maps and charts allow us to navigate land and sea with more ease than an oral explanation, a map can help us visualize and navigate through problems and arguments. To map an argument, you must start with a proposition, or chief contention – this contention is entered into a white box and placed at the top of an argument map. Supporting claims are color-coded green, while objections are coded red. Claims are organized in a pyramidal hierarchy to maximize the appearance of evidential and logical relationships. The first set of claims (top level) begs the question “what are the distinct arguments provided for the main point (the chief contention)?” Subsequent levels are asked, “Do they support all of these primary arguments with further evidence? [and] Do they countenance any objections to their argument and rebut them?

The authors use the article Coalition of the Willing? Make That War Criminals, which discusses whether or not a preemptive strike on Iraq would constitute a crime against humanity, to demonstrate how argument mapping is useful. (See Image Below)


Advantages of argument mapping over prose:

1. It makes explicit logical relationships that the linearity and abstractness of prose cannot help but obscure.
2. The map offers an instant and effortless scan-ability of the overall structure of the argument, which you simply cannot derive from prose.
3. There is an ease of movement from the detail to the overview that is far more difficult in the case of prose.
   
4. There are unambiguous visual clues as to the significance that particular details have, due to the hierarchical ordering of the structure, the color-coding of the individual boxes and the inferential relations between boxes.
5. A map offers a visual clarity as to the limits of a debate, whereas prose obscures these limits or labors to spell them out.
6. The cognitive burden imposed on us by the task of analyzing a piece of prose is drastically reduced in the case of a map, for the same reasons that it is reduced in moving from a prose description of London to a map.
7. For any given proposition, all claims are integrated into a single structure, instead of consisting of various component parts, which then have to be assembled by whoever happens to be trying to comprehend the argument in question.
   

*Author’s Note: Tim van Gelder has done extensive research in the field of argument mapping and is the leading mind behind Reason!Able, a computer program designed for argument mapping. Reason!Able has now evolved into Rationale. See Video Below.