Summary
The Delphi method structures and facilitates group
communication that focus, upon a complex problem so that, over a series of
iterations, a group consensus can be achieved about some future direction. It
has five major characteristics:
- The sample consists of a "panel" of carefully selected experts representing a broad spectrum of opinion on the topic or issue being examined.
- Participants are usually anonymous.
- The "moderator" (i.e. researcher) constructs a series of structured questionnaires and feedback reports for the panel over the course of the Delphi.
- It is an iterative process often involving three to four iterations or “rounds” of questionnaires and feedback reports.
- There is an output typically in form of a research report with the Delphi results, the forecasts, policy and program options with their strengths and weaknesses, recommendations to senior management and, possibly, action plans for developing and implementing the policies and programs.
It has some advantages over some other group decision-making
techniques like the nominal group technique (NGT) and interacting group method
(IGM). First, panel members are not swayed by group pressures or vocal members
as can easily happen with NGT and IGM. Second, interpersonal conflicts and
communication problems are virtually nonexistent because panel members do not
interact. Third, travel costs and the problem of coordination to get everyone
at the same place at the same time are not factors. The Delphi method consists
of four key planning and execution activities:
- Problem definition: It is necessary to define the nature and scope of the problem, expected outcomes of the study and the appropriateness of the Delphi method.
- Panel Selection: The Delphi method requires a panel of subject-matter experts (SMEs). The criteria for determining who qualifies as a SME may rest not only on knowledge, but could include criteria such as personal experiences or being stakeholders. Besides, it is important to inform prospective panel members that their commitment to participate would involve several rounds of questionnaires and feedback possibly extending over a period of months. On the hand, Panel selection might not be random because, in some research fields, there might be very few SMEs; thus, one might select all known SMEs.
- Determining the panel size: While there is no one sample size advocated for Delphi studies, rules-of-thumb suggest that 15-30 carefully selected SMEs could be used for a heterogeneous population and as few as five to ten for a homogeneous population. The careful selection of SMEs is a key factor in the Delphi method that enables a researcher confidently to use a small panel. That is not to say that large panels are never used. Indeed, some Delphi studies have used large panels numbering over 100 members.
- Conducting the Delphi rounds: A Delphi study usually involves three to four rounds or iterations, not just a one-time effort; thus, the moderator is able to set up Round 1 according to some strategy knowing that another two to three rounds could be conducted to achieve consensus or other goals. Although three to four rounds are typically used, the moderator should stop the rounds when the criteria for consensus are achieved, when results become repetitive, or when an impasse is reached. Following the final round, the moderator prepares a comprehensive report and distributes it or a short version to all members.
Critique
This paper describes the characteristics of the Delphi method, including criticisms of the method, and steps in conducting a Delphi study as well as pitfalls to avoid. It is a good overview in order to understand the basics of the method. However, the author does not mention an important weakness of the Delphi method. From my point of view, this method is heavily contingent upon the researcher’s ability. If the researcher have enough knowledge and ability to conduct this method, it can produce valuable results. Otherwise, it is just a fruitless time-consumption.
Source:
Robert Loo, (2002) "The Delphi method: a powerful tool for strategic management", Policing: An International Journal of Police Strategies & Management, Vol. 25 Iss: 4, pp.762 – 769. Retrieved from http://search.proquest.com.ezproxy.mercyhurst.edu/docview/211297480/A90AEE4047D44C01PQ/1?accountid=27687
How do you infer that the success of the method heavily relies on the experimenter? If that so ( relies on the experimenter), then why does the experimenter need expert opinion? Your critique doesn't specify the phase that the method needs conductors' ability; in collating the survey results and understanding them or in general?
ReplyDeleteResearcher still needs the expert opinion. That is not my point. I stressed the importance of researcher's ability. In order to provide anonymity, researcher is the only one who controls every single step of the method. Therefore, researcher's ability to manage the process becomes crucial. If researcher fails, the method has no chance to succeed. I believe this issue is a weakness of the Delphi method. However, many articles including this one, are not addressing this weakness.
ReplyDeleteI think in addition to your point, the researcher needs the expert opinions to develop the consensus, where the final product of the Delphi method is an accumulation and agreement of expert opinions, which will usually be a better estimate than the researcher, or any of the experts, would have been able to develop alone.
DeleteI agree with you. A pressure-free consensus created by the experts is the most powerful aspect of this method.
DeleteMy concern with the Delphi method is the way the questions are worded/formatted. Would it be beneficial for the researcher to apply other analytic techniques prior to finalizing their use case?
DeleteMany articles about the Delphi method suggest triangulation which means using some other analytic techniques in addition to the Delphi method. Therefore, your concern is very meaningful to test the results of the method.
DeleteOne thing I noticed in your article that was not present in the other ones I have read so far was the benefit of low travel costs and not needing to coordinate the schedules of all the busy participants. Although seemingly insignificant, this benefit allows for a larger sample size which corresponds with more accurate information.
ReplyDeleteI agree completely with Devon. The switch from pen and paper to electronic methods of conducting a Delphi must have made the methodology much more accessible for researchers. The reduced time and costs of conducting the survey make it an increasingly valuable tool. I would imagine it has already been done, but Delphi specific programs could offer the researcher a highly customizable format with which they could ask questions.
Delete