By:
Summary:
In December 2011, the NSA released a document explaining
how the Delphi technique could minimize the subjective elements in the decision
making process. The document went on to explain the technique and the
weaknesses it presents within the intelligence community. The Delphi technique was developed to make the
“most effective use of informed intuitive judgment by creating conditions under
which a group of experts can perform most ably and their answers can be
combined into a single group opinion.”
This technique utilizes anonymous responses, controlled
feedback, and a statistical derivation of responses. Participants are questioned individually to avoid
problems that face-to-face conversations can have such as: dominating personalities,
group members fearing abandoning the overall public opinion, groups making lots
of noise that do not contribute to the issue at hand, the bandwagon effect, and
the idea that experts are hard to get together in one place at one time. According
to RAND, anonymous responses do produce better results than face-to-face group
discussions.
After responding to the first set of questions,
participants are asked to respond again using the information provided by the
other responses. This cycle can continue for several iterations. Participants
are sometimes allowed to ask further questions or give reasons for additional
data. By utilizing this technique, participants can produce new ideas which are
then disseminated among the group and respondents can get a feel for the majority’s
opinion. This technique aggregates every opinion to make sure it is
contributing to the group’s final opinion. In addition to the normal Delphi
technique, the document suggests to rank experts and weigh their opinions.
The article also goes on to conclude that the Delphi technique
needs to have the right type of experts to the answer questions. The
intelligence community has two issues regarding this, first: it is not normal
for experts using the Delphi technique to carry technical data to back up their
judgments; and second, the structure of the intelligence community suggests
that “persons with the requisite technical knowledge are usually not on a level
to make judgments, while those at the appropriate levels often must depend on
others for data.” Instead, the intelligence community is made up of specialists
that may not be capable of judging all facets of a complex problem.
Critique:
In the field of intelligence, validating analytical techniques
such as the Delphi technique is significant. It can help find the consensus of
the community which can help determine policy decisions. In addition to the
weaknesses stated above, the Delphi technique can also ignore facts and cause bias.
Although the concept of “two heads are better than one” makes logical sense,
the idea that the coordinator can alter the feedback to cause bias makes this
technique very subjective.
Source:
National Security Agency. (2011). The Delphi Technique (NSA DOCID: 3928741). https://www.nsa.gov/public_info/_files/cryptologic_spectrum/The_Delphi_Technique.pdf
Posts
Joy,
ReplyDeleteA modifier the article suggests is ranking experts and weighing their judgments. What are your thoughts on this modifier, does it make the technique more robust or does it take away from the intended use of Delphi?
Ricardo,
ReplyDeleteI think it would depend on the situation. Ranking experts and weighing judgments could cause the technique to produce an entirely different answer, which leads to biases.