Summary of Findings (White Team): Delphi Technique (3out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 16 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst College on 22 April 2010 regarding Delphi Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Delphi is a method for a structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem. It was created in the 1950's by the Rand Corporation to increase efficiency and to bring together Subject Matter Experts to assess and create possible outcomes to a problem. It is a process conducted through the use of multiple rounds to introduce new ideas and come to a consensus on those new ideas along with existing ideas.

Strengths:

• Its ability to ascertain expert opinions and potential policy options, including new alternatives.
• Participants can be located all over the world.
• Depending on the goal of the study, consensus on a forecast may be reached.
• Anonymity improves participants ability to judge the presented options, not just presentation style.
• Forecasts can be judged against the later reality of the situation.

Weaknesses:

• Participants must trust the methodology and moderator or facilitator for this to work.
• Moderator or facilitator should be objective.
• Can lead to Groupthink.
• Susceptible to the anchoring bias.

How To:

Pick a facilitation leader: The facilitator is an expert in research data collection, and is not a stakeholder.

Select a panel of experts: The panelist should have an intimate knowledge of the projects.

A questionnaire is sent to experts.

Each expert gives his / her answers to the questions in an independent and anonymous way.

The moderator summarizes the responses to the first questionnaire and develops a feedback report along with the second set of questionnaires for the panelists

panelists independently evaluate earlier responses and vote on the second questionnaire

The set of responses is then sent back to experts and they are asked if they wish to revise the initial predictions.

The process is reiterated until a degree of consensus is reached by experts.

Some variations to this very basic method include:

• The number of iterations (the more rounds, the closer the consensus likely to be reached)
• The method of selection and size of the panel
• The scoring system and the rules used to aggregate the judgments of the panelists
• The extent of anonymity afforded to the panelists
• How consensus is defined and how disagreements are dealt with

Application:

We decided to apply the Delphi technique to expected completion of the Master of Science of Applied Intelligence program here at Mercyhurst College. At graduation, every student is allowed to walk across the stage to be recognized, however, not every student may have completed the degree requirements necessary to receive his or her diploma. Based on statistics from recent years, our class attempted to estimate how many out of 21 students in the class of 2011 would actually receive their diplomas on graduation day. In our graduating class, currently 20 students are scheduled to write a thesis and one student is scheduled to take a comprehensive exam. There were three rounds of anonymous voting:

• Round 1 - Average:14.5; High: 19; Low: 7
• Round 2 - Average: 13.08; High: 17; Low: 7
• Round 3 - Average: 13.16; High: 17; Low: 10

After round 1 there was a 1-minute discussion to articulate why each of us voted the way we did. After round 2 we were presented with previous classes' graduating statistics which was designed to provide background information that we could use to adjust our individual estimates. After round 3, we arrived at our final conclusion as shown above.