Pragmatic Research Design: an Illustration of the Use of the Delphi Technique

This study conducted by two scholars at Rhodes University in South Africa used the Delphi Technique in hopes of forecasting the educational needs that will have to be met to prepare students to be entrepreneurs over the next 20-40 years. This paper does not walk readers through the results of the study, but rather the practical challenges in definitions and the organization of applying the Delphi Technique.

Definitions:
Based on the author’s review of the literature, they identified five main characteristics which define the technique:

1. Its focus on researching the future or things about which little is known,
2. Reliance on the use of expert opinion,
3. Utilizing remote group processes,
4. The adoption of an iterative research process, and
5. The creation of a consensus of opinion.

The authors also identify three versions of the Delphi Technique:

• Numeric – aims to specify a single or minimum range of numeric estimates through the use of summary statistics.
• Policy - on the exploration, generation and definition of several alternatives and the arguments for and against each of these alternatives.
• Historic - aims to explain the range of issues that fostered a specific decision, identification of several scenarios that could have led to the resolution of a past problem.

Entrepreneurship was defined as person who provides innovation in an economy, not owners of micro-businesses in saturated markets. The foundation of their study is based on a review of literature that suggests entrepreneurship can be formed through education.

The Experiment:
To comprehensively forecast the answer to their question, they asked three separate questions. The questions were the following:

1. What sector of the South African economy will most likely offer the greatest potential for entrepreneurial opportunities in the next 25 to 40 years?
2. What qualities are needed by graduates to equip them to be innovative entrepreneurs in the future?
3. What should Higher Education in South Africa do to prepare/develop students to constructively participate in the future economy as innovative entrepreneurs?

Three separate panels were created because each question requires answers from a different set of experts.

1. The panel for the first question was referred to members of government departments and research councils.
2. The panel for the second question was referred to endowed Chairs in the area of entrepreneurship.
3. The panel for the third question was referred to alumni of entrepreneurship programs and educationalists and academics in these programs.

Essential Components Of Curricular Learning Communities In Higher Education

The purpose of this study was to identify the essential components of curricular learning communities in higher education. A panel of experts participated in a four-round Delphi process designed to identify these essential features. The writer used a modified Delphi process to first elicit and then rate the importance of characteristics of curricular learning communities in higher education.

Delphi Advantages:
-The Delphi technique offers the advantage of group response without the attendant disadvantages sometimes experienced with group problem solving or decision-making.
-Expert participants are more likely to generate reasoned, independent, and well-considered opinions in the absence of exposure to the "persuasively stated opinions of others". Because the experts do not ever participate in a face-to-face discussion, there is no danger of one or more individuals’ opinions being swayed by a more dominant or more experienced individual.
-Efficiency and flexibility, especially in light of modern communication technologies such as e-mail and the Internet. Experts may be drawn from a wide geographic area, and the participants’ commitment in terms of time and money invested is minimal.
-Delphi method has been shown to be an effective way to conduct research when the responses being sought are value judgments rather than factual information. Although it is more difficult to assess the "correctness" of value judgments, it is generally agreed upon that value judgments are not all equal but can in fact be more "right" or more "wrong."

Delphi Limitations:-Delphi should not be used when any of the following three critical conditions are not present: adequate time, participant skill in written communication, and high participant motivation. It is estimated that a minimum of 45 days is required to carry out a Delphi study.
-Participants must be knowledgeable and able to clearly communicate their ideas. A high degree of motivation is needed to offset the tendency for participant dropout as the study progresses. Because there is no direct contact between participants, those who are not highly motivated and interested in the subject at hand may feel isolated or detached from the process.
-Another is the problem of bias in Delphi studies that can occur from poorly worded or leading questions or selective interpretation of the results.

Instrument Design and Implementation
Round One: Initial Survey:
The first round in the current study consisted of a brief survey, designed to collect some demographic data on the participants, and one open-ended question.

Round Two: Questionnaire One:
A list of 79 features was compiled from the information obtained in the initial survey. Obvious repetitions were eliminated, though items that were similar but not exactly the same were maintained. Items were sorted into four categories: Curricular Features, Pedagogical Features, Structural Features, and Environmental Features. Participants were asked to rate each feature on a Likert-type scale, identifying each feature as an "essential" (5), "very important" (4), "moderately important" (3), "slightly important" (2), or "not important"(1) characteristic of a curricular learning community.

Round Three: Questionnaire Two:
Questionnaire Two listed only the features that had received a mean rating of 4.0 or higher in the previous round. Once again the items were placed into the four categories of Curricular, Pedagogical, Structural, and Environmental.

Round Four: Questionnaire Three:
The third and final questionnaire listed the forty features that received the highest rating (determined by mean and mode) on the previous questionnaire. Panelists were given the following information: ranking of the items from first and second questionnaire, mean score of the items from both rounds, and the number of times each item was selected as one of the three to five most important items.

In this round, panelists were asked to assign a total of 100 value points to the forty items. At the end of this questionnaire, participants were asked to answer the another open-ended question.

Summary of Data Collection and Analysis Procedures:

The following table outlines the four-round Delphi procedure that was followed in this study: (see link for table)

Link: http://www.winona.edu/advising/lcchapter3.htm