The Delphi Technique in Project Management

Abstract

According to the definition of ISO 21500:2012 a Project is: “a unique set of processes consisting of coordinated and controlled activities with start and end dates, performed to achieve project objectives. Achievement of the project objectives requires the provision of deliverables conforming to specific requirements”.

The Delphi technique is used mostly for reaching consensus among experts, regarding the decision making process. Moreover, as a project manager is crucial to think about the impact that possible future events can have on your projects, this technique helps also in forecasting the future with some grade of certainty, estimating likelihood and outcomes.

The iterative process consist of anonymous exchange of views, assumptions and prevision given to a facilitator that will write a conclusive report. The group will read the report of the previous meeting, update their discussion and thoughts for the new report. The process continue in this way until the consensus is reached among all the participants. The anonymity assure the free expression of opinion and encourage openness. The process will start from a broad perspective than in the end will scope the core problems and questions to assure consensus, 3 phases has to be followed:

• Brainstorming: experts list the relevant factors and remove the duplicates
• Narrowing down: differentiate in panels and factors selection
• Ranking: rank the factors on own panel, calculate average, assess consensus

This method can be reiterate many times based on the analysis and the ranking of different factors list on panels, controlling every time the feedbacks (it can be done also online with virtual teams). It is used also to identify risk and opportunities, learning from mistakes, promoting a brainstorming session or to create a Work Breakdown Structure.

Big Idea

Brief history introduction

---

Delphi method was developed in 1953 by Olaf Helmer and Norman Dalkey of the Rand Corporation as an instrument to forecast the impact of technology on warfare. The name refers to the Delphi Oracle, a priestess at the Apollo temple in ancient Greece.

Definitions

---

"A method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem".

“Technique to arrive at a group position regarding an issue under investigation. The Delphi method consists of a series of repeated interrogations, usually by means of questionnaires, of a group of individuals whose opinions or judgments are of interest. After the initial interrogation of each individual, each subsequent interrogation is accompanied by information regarding the preceding round of replies,usually presented anonymously. The individual is thus encouraged to reconsider and, if appropriate, to change his previous reply in light of the replies of other members of the group. After two or three rounds, the group position is determined by averaging”.

General Concept

---

It is an essential project management technique used to gather information, the goal is to reduce the amount of different responses and reach the consensus among the experts (stated by some studies to be between 70% and 51%), ,this method is recognized, adopted and is it still used nowadays. Treating uncertainty is a central issue in project management, an entire literature about risk management can be found if interested to have more arguments. “Project management is about reducing uncertainty step by step until it has been eliminated completely, for better or worse, and we achieved a particular result” . This method tries to break the stereotype that says: “When you get three experts together, you'll often end up with four different opinions.” Every project has his own characteristics and needs, consequently there is often not just one-size-fits all solution, we have to narrow down in all of his part to find the customized solution for the specific case. The participants in the process do not need meet, and sometimes they even do not know who are the other participants, only the facilitator knows everything and manages the information flow in order to avoid possible repercussions; the anonymity helps avoiding personal issues between member and the so-called “groupthink”. The phenomenon of groupthink can be really dangerous in an organisation, is marked by the consensus of opinion without thinking critically and evaluating other alternatives and consequences. The core point of this mistake is to do not break the status quo of the group, avoiding conflicts and creativity, this will reduce the pressure of the group toward conformity.

The structure of this technique give time to the participants to think properly about their opinion and responses, being so able to contribute with full hand to the problem to be solved or in the discussion. Moreover, the iterative nature of this approach gives the opportunity to refine, narrow down and test the arguments of the members (being able also to tone down some possible hard statements).

fig here

Since are scheduled multiple rounds of questions in the panel the thoughts of the groups are really investigated and studied, but the communication has to be facilitated in every moment. Also, there is the opportunity to aggregate different opinions from experts without the need to have a physical meeting, it is different also from other methods because it is flexible and there are incorporated qualitative and quantitative approaches in it. This technique is used in the prediction of the future as Helmer, one of the inventors , said: “The future is no longer viewed as unique, unforeseeable and inevitable; there are, instead, a multitude of possible futures, with associated probabilities that can be estimated and to some extend, manipulated”, so one of the main aim with this method is the anticipation and recognition of needs.

fig here

Respondents usually rate their agreement on a specific statement or sentence with a five point scale, for instance, (1) means “strongly disagree” and (5) “strongly agree”, in the end the topics with 4 as the average or higher and with standard deviation below 1 are considered areas in which the agreement is found (the so-called: “items of consensus”). There is not the purpose to arrive always at the unanimity among all the experts, conversely a spread of opinions in the end is a common outcome (limitation and risk).Typical evidences shows that the opinion is converging during the phases and the average response moves in the direction of the true reply; this is mainly due to three different factors: social pressure, re-thinking of the problem and the movement of information during the feedback session.

In the decision making process mainly three kind of information are playing a crucial role: knowledge, speculation and opinion (see picture below for an example, but an entire set of data can be seen in this paper: , for more detail).

The main suggestion to success using this technique are: searching for endorsement of an important person during the activity to sustain and keep high the involvement of all the participants, for example calling the participants after the invitation or think about incentives for each round. Moreover, some initial requirements are needed as: high level of effort among the participants, high skill level in writing the questionnaires and plenty of time available to conduct the process (i.e. 6 weeks). Empirical studies reveal that “Delphi Method is superior to conventional uses of groups in problem solving or forecasting” .

Advantages

---

Advantages:

• Versatile technique, lot of different applications
• Group experts from different sectors/backgrounds (wide range)
• No location limits, not so expensive in this sense
• Promote reflection, evaluation and involvement
• Recognition of the contribution of each participant
• Easy analysis of data
• Anonymity: free thoughts
• ”It´s adaptability to diverse data collection strategies, decreased peer

pressure secondary to anonymity and the ease of condensing opinions of many and varied experts into a few precise statements”

• Overcome the problem of few individual dominating discussions

Steps

---

Ten steps for the Delphi method:

• 1)Formation of a Delphi team on a subject
• 2)Selection of expert panels
• 3)Development of the first questionnaire
• 4)Testing the questionnaire
• 5)Transmission to the panel lists
• 6)Analysis of responses
• 7)Preparation of second round
• 8)Transmission of second round questionnaires to the panel lists
• 9)Analysis of the second round (iteration)
• 10)Preparation and presentation of report

fig here

Types of Delphi

---

Types of Delphi:

• Classical Delphi: five features: anonymity, iteration, feedback, statistical group response and response stability among experts on a specific issue.
• Policy Delphi: the aim is generate policy using a structured dialogue, it is possible to have several divergent opinions from experts; the features are: answer questions individually but together in the meeting, iteration, feedback, polarised group response and conflicts.
• Decision Delphi: used for reach and high grade of involvement among decision makers on social issues, from different hierarchy positions. Here the names are mentioned, but the responses to the questionnaires are anonymous.

Nominal Group Technique

---

A modified Delphi method is called “The nominal group technique” (developed by Delbecq and VandeVen, 1971) the main difference is that the meetings are done face to face, starting with an open-ended question, aiming at a group discussion to reach consensus. Ideas and thoughts about the topic are shared and recorded, each person evaluate the ideas anonymously and individually votes for the best ones keeping the anonymity, finally a report with the responses is prepared. NGT is more structured than the Brainstorming technique, obtaining multiple inputs from experts on precise issues, is used mostly in educational contexts, this specific tool is helpful because researches shows that individuals produce more ideas when working alone and they do it without sacrificing the quality. As it is for the Delphi method the facilitator here has to be an expert in the field or a person with credibility among the participants. This technique can be useful when there are little evidences but some decisions needs to be made (as it is for Delphi), or to rank priorities during a group meeting.

fig here

Strengths and Weaknesses

---

Strengths:

• Less problems in group interaction
• Common thoughts periods allowed
• Listen also to minor views
• Both qualitative and quantitative data produced

Weaknesses:

• Risk of false consensus
• Problem of selection biases
• It require as a basis a strong facilitator

Applications

When can we use it? When there is the need to explore an issue with a group of people or when you want to move a group towards consensus or when a particular problem does not permit the use of an analytical tool, when the participants are not able to meet physically or they are too many. This method can be used for several reasons: helping the decision making process as a group, to gain inputs from experts, to define some needs, to identify results, to prioritize some causes, to suggest solutions, to define future goals, to anticipate outcomes, for planning and scheduling, for problem solving, for data collection, to identify dimensions of a problem or to forecast. Delphi Method can be done with a small or big group (hundreds of people), but the common way schedule nearly 20 people. It is useful especially when there are many experts to contact, meaning that the consensus is difficult to reach and when the experts are located in different in different parts of the world.

The first step is to select a group of experts based on the topic that has to be analyzed, later a questionnaire is sent to all the participants with instructions for each topic and on how to write opinions, experience and personal researches. These questionnaires are then sent to the facilitator that puts together all the comments and send a copy to each participant. At the end of every session all the questionnaire are sent again to the facilitator who will decide if there is the need to do another round or not, the repetition of this procedure can be done as many time as necessary to achieve the final consensus.

Other common applications

---

Other common applications in projects are:

• In time management to estimate the duration of activities
• In cost management to estimate the different costs
• In risk management to identify the risks
• In scope management to collect various requirements
• Government planning
• Business and industry
• Culture, family and behavior
• Education and training
• Communications
• International Security

Limitations

The limitations of this method are that the responses can take a lot of time and so also the rate of discussion can be low, sometimes the interaction is not right as it is in a face to face meeting. Finally, it is possible that the information received from the experts are not going to create new value.

Disadvantages

---

Disadvantages:

• Facilitator´s fail (i.e. wrong expert/questions selection)
• Long time needed if made via mail or post (time consuming)
• Participants involvement is a requirement (quality of the responses decrease with the passing of time)
• Individual point of view (combine with other tools for a more complete overview)
• Anonymity may lead to lack of responsibility/accountability
• Labour intensive
• ”Conceptual and methodological inadequacies, potential for sloppy

execution, crudely designed questionnaires, poor choice of experts, unreliable result analysis, limited value of feedback and consensus, and instability of responses among consecutive Delphi rounds.”

• Little research in assessing the reliability of this method
• Risk of “groupthink” phenomenon