Facilitating Problem Solving: A Case Study Using the Devil's Advocacy Technique

By: Ryan T. Hartwig

Summary:

This article presents a case study of the devil's advocate (DA) method of analysis. The purpose of this study is to analyze the efficacy of DA in facilitating group problem-solving and decision-making in a real-world situation. Ryan T. Hartwig, the researcher, begins by briefly explaining that the process of DA is played out when, "one group member or a subgroup critiques a group's (or another subgroup's) plan by raising questions about the plan's assumptions and consequences, but does not offer a counter-plan". The participants in this study are six staff members of the Life Planning Center (LPC) at a university located in Denver, Colorado. Hartwig explains that the LPC is newly-staffed, and therefore is experiencing complications in functioning within the university. The challenge bestowed upon the six staff members, then, is to utilize DA in developing a plan on how to motivate their university's academic administrators to publish accurate course rotations in a timely manner.   

Hartwig's DA method is similar to the method that he provides in Figure 1 below, with a couple exceptions. Before splitting the groups into two subgroups,  Hartwig ties in the first two steps of Gouran and Hirokawa's (1999) functional decision-making technique by defining the problem and establishing criteria for an acceptable solution with the group as a whole. The second way Hartwig strays away from the process in Figure 1 is by omitting Round Two and the Subsequent Rounds altogether. Instead, Hartwig has both groups work together to produce a final recommendation. This process lasted about 95 minutes. 
 
 

Findings:

After the process was over, each participant took a survey about both Hartwig and the DA process. The survey answers were assessed using a 5-point Likert scale.  The highest scores pertaining to the DA process, both averaging 4.5, dealt with DA's ability to facilitate the groups' abilities to make recommendations and critiques, and in revealing new recommendations and assumptions that were not thought of previously. The lowest score, averaging 3.33, came from the group's opinions on the recommendations and assumptions that the groups brought up during DA. Overall, the participants viewed that DA's major strengths are that it allows participants to think outside of the box, and that it enables discussion through conflict. The participants found DA's major weaknesses to be that it relies on opinions that participants may refuse to budge from, and that it is difficult for some participants to handle discussions revolved around conflict. 

Hartwig concluded that this method of DA stimulated good group discussion, generated good ideas, and provided the participants with useful recommendations for their problem. Hartwig recommends that DA should be paired with Gouran and Hirokawa's (1996) functional theory of group decision-making, or a similar process.

Critiques:

Both Hartwig and the participants mentioned that this use of DA was not extensive enough, therefore, it is difficult to gauge if the method could have been more productive given more time. This plays into the fact that Hartwig did not truly utilize his own definition of DA, since he changed the steps after step one. Hartwig also divulged his own recommendations to the problem during DA, which could have tampered with the outcome of the process. Based off of these critiques, it is hard to say if this case study qualifies as a true analysis of DA. 

Source: Hartwig, R. (2010). Facilitating problem solving: A case study using the devil's advocacy technique. Group Facilitation: A Research and Applications Journal, 10, 17-31. Retrieved from http://tinyurl.com/ja8jkhe  

Innovation Roles: From Souls of Fire to Devil's Advocates

Summary:

This article discusses the findings of a study which aims to see if organizational members in various innovative roles differ significantly with respect to their perceptions about pros and cons of innovation and their levels of innovation-related communication. The purpose of the study is to determine how members at various positions along the chain of command within an organization impact and influence the innovative process; either positively or negatively. The author explains the traditionally accepted roles in the organizational innovative process as being that of “the idea generator”, “the idea champion”, and “the orchestrator”. The idea generator is typically a lower level member within an organization who has firsthand experience with the problem that requires innovation. This role typically requires support from employees with more authoritative power. These employees are the idea champions who greenlight innovation efforts such as implementation and experimentation by providing various forms of support. The third role is that of the orchestrator who is typically a high ranking member of the organization whose job is to navigate the innovation through the political turmoil that often results when changes take place within a workplace.

The study itself was conducted by the Cancer Information Service (CIS) to see if they could find a more efficient way to spread the word about the dangers of cancer. The previous method of informing the public was a 1-800 number citizens could call to get information. The proposed innovation was a trial run of calling women in lower income minority neighborhoods urging them to get mammograms. The survey following the trial asked various members at all stages of the implementation process and at all ranks within the organization to voice their opinion on the pros or cons of the innovation. 

Findings:

The study concluded that idea generators typically have more “buy in” than other organizational members as expected. However, idea champions did not have as high a pro rating for new innovations as was expected. This was thought to be due to a number of external factors including negative perceptions of cold calling customers and low job satisfaction with phone operators. Orchestrators of the innovation did find it had more pros than cons. These findings show that those who have the power to implement organizational innovation also have the power to challenge it, if they chose to do so. 

Conclusions:

The author acknowledges the mixed results from the expected outcome that all members who take on roles in the organizational innovation process would support it. The author then suggests the implementation of a devil’s advocate into the organizational roles for those who object to innovation for legitimate reasons. This would satisfy the criticism that past innovation role research has a managerial or pro-innovation bias. The author raises a series of questions designed to direct the devil’s advocate in any cases of organizational innovation processes. Such questions include what is the role of the devil’s advocate on the team? Who appoints the devil’s advocate to their position? How is the devil’s advocate looked upon by other members of the organization? And finally, what is the role of power in determining the effectiveness of a devil’s advocate? The author believes these core questions must be answered and examined through future studies to determine the effectiveness of a devil’s advocate before implementing them as a character role in the organizational innovation process.

 

Critique:

This study, while it didn’t use the devil’s advocate methodology directly, shows instances where it can be useful in understanding innovation biases. For example, when the findings of the study showed that idea champions did not follow the assumed path of being proponents for innovation, a number of possible reasons for why this could have been the case were suggested. Had a devil’s advocate been incorporated it is likely they would have voiced similar concerns, thus eliminating speculation for an outcome. In general, the study itself lacks depth in attempting to identify if members of an organization at various ranks have more or less aptitude to support an innovation. It would seem apparent that those directly responsible for creating the new idea and seeing it through to completion would have more “buy in” than those who are not directly involved in the process. Similarly, it makes sense that members of various ranks would have differing opinions on the usefulness of an innovation based on how that innovation impacts them directly. An innovation that saves the company money in the long run but intensifies the workload of the lower ranking members will undoubtedly be seen differently at various levels. It shouldn’t take a devil’s advocate to come up with that line of reasoning. 

Citation:

Meyer, Marcy. (2000). Innovation Roles: from Souls of Fire to Devil’s Advocates. Journal of Business Communication, October 2000; vol. 37, 4: pp. 328-347. Retrieved from http://tinyurl.com/j949aow on September 8, 2016.  

 

Summary of Findings: SWOT (2.5 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the  articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in September 2016 regarding SWOT as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

SWOT is an analytic modifier that combines Strengths, Weaknesses, Opportunities, and Threats into one cohesive analysis.  SWOT is marginally effective in looking at both internal and external environments across the previously set parameters. The analysis focuses on all of these factors as viewed from the standpoint of a specific organization.

Strengths:

• Useful for team-collaboration (Particularly if used with the nominal group technique)
• It is easy to use
• Can provide a better understanding of an outside organization
• Can be used as an analytic modifier with other methods of analysis

Weaknesses:

• The results are not easily replicated because information imputed can be highly subjective.
• Because SWOT is more of a modifier and not a method, its use as a forecasting tool on its own is very minimal.
• As an intelligence tool, it is too often used for internal inspection and not external threats/problems.
• SWOT needs to be paired with a second, more robust method in order to enhance its capability as a forecasting tool.
• Often misunderstood; the definitions for each category are not well defined and its intent is vague.

How-To:

SWOT is a moderately useful modifier when thinking externally about another organization or individual. SWOT is effectively a brainstorming technique that is simple to conduct:

1. Define the external organization or individual in question that requires analysis.
2. Identify and list the known or perceived Strengths of the topic. (A solid number to shoot for is between 5-10)  
3. Do the same thing for the topic’s Weaknesses.
4. Then list any potential Opportunities the topic has to improve their current standings. (Think what actions could they take in the future to better their situation.)
5. Identify any potential Threats to the topic that could cause them to deteriorate or miss opportunities.
6. Use the completed list to create strategies that avoid the topics’ strengths and opportunities and aim to take advantage of their weaknesses and threats.

Personal Application of Technique:

In order for SWOT to be appropriately applied for intelligence work, the analysis needs to be focused on a specific external environment. Team members conducting the analysis role-play as the organization or entity in which they wish to study and work from what knowledge they have on that specific organization. The obvious issue that appears from using this modifier is that understandings may be heavily skewed from lack of knowledge or bias creating an inaccurate assessment. Accuracy is also limited because feelings or known “facts” about the subject of the study may change dramatically from day to day meaning that the analysis is not adequately replicable.

For this exercise, the two groups in our class collaborated on a joint effort to utilize SWOT. One team led the demonstration and began by informing the other group about the topic for the SWOT analysis, which was Mercyhurst University. The leading team used nominal group technique (NGT) and had the other team independently brainstorm ideas that would fit into the different categories for approximately three minutes. After this was completed, ideas were sourced from everyone and put on the board, one category at a time. Once all categories were completed, the leading team used the remaining time to combine the categories in an attempt to create actionable strategies. For example, factors from the strengths category were combined with some from the opportunities category to create a likely actionable strategy for Mercyhurst University.

Much of the extant research on this modifier suggests that the tool should not be limited to the technique alone, however, it should be supplemented by additional methodologies to enhance the SWOTs overall observation and insight capabilities.  Additionally, the research shows that SWOT does not necessarily provide adequate analytic coverage due to much of the specific areas of analysis being ambiguous and non-replicable.

*SWOT analysis is a modifying technique that can be used mostly as a thinking tool. In order to develop a form of forecasting capability with SWOT it must be attached to a second methodology that can give insight toward a forecast. Often times SWOT is something humans just do out of habit due to no other method coming to mind. For this exercise the two groups were split between one researching articles concerning SWOT, while the other conducted a SWOT exercise in class. From both it became apparent that particularly in the application of intelligence and capabilities to influence forecasting SWOT holds little to no strength by itself.

For Further Information:

Wikipedia: https://en.wikipedia.org/wiki/SWOT_analysis

Mindtools: https://www.mindtools.com/pages/article/newTMC_05.htm

BusinessNewsDaily: http://www.businessnewsdaily.com/4245-swot-analysis.html

Evaluation Study on SWOT: http://www.ejbss.com/Data/Sites/1/vol2no1april2013/ejbss-1235-13-aconceptualframeworkonevaluating.pdf

Evaluation Study on SWOT by Finnegan: https://www.scribd.com/doc/31990392/Evaluating-SWOT-s-Value-In-Creating-Actionable-Strategic-Intelligence