The Delphi Technique in Project Management

Revision as of 09:33, 22 September 2016

According to the definition of ISO 21500:2012 a Project is: “A unique set of processes consisting of coordinated and controlled activities with start and end dates, performed to achieve project objectives. Achievement of the project objectives requires the provision of deliverables conforming to specific requirements”.^{[1] [2]} The Delphi technique is used mostly for reaching consensus among experts, regarding the decision making process. Moreover, as a project manager is crucial to think about the impact that possible future events can have on your projects. This technique helps also in forecasting the future with some grade of certainty, estimating likelihood and outcomes. The iterative process consist of anonymous exchange of views, assumptions and prevision given to a facilitator, that will write a conclusive report. The group will read the report of the previous meeting, update their discussion and thoughts for the new report. The process continue in this way until the consensus is reached among all the participants. The anonymity assure the free expression of opinion and encourage openness. The process will start from a broad perspective than in the end will scope the core problems and questions to assure consensus; 3 phases has to be followed:

• Brainstorming: experts list the relevant factors and remove the duplicates.
• Narrowing down: differentiate in panels and factors selection.
• Ranking: rank the factors on own panel, calculate average, assess consensus.

This method can be reiterate many times based on the analysis and the ranking of different factors list on panels; controlling every time the feedback (it can be done also online with virtual teams). It is used also to identify risk and opportunities, learning from mistakes, promoting a brainstorming session or creating a Work Breakdown Structure.

This article will have the following structure: the general idea will be presented, related to some definitions, advantages, steps to follow and strengths and weaknesses. Finally, applications and limitations will be explored, using a study case that will be presented, highlighting a real application of this procedure.

Big Idea

Brief history introduction

Delphi method was developed in 1953 by Olaf Helmer and Norman Dalkey of RAND Corporation as instrument to forecast the impact of technology on warfare. The name refers to the Delphi Oracle, a priestess at the Apollo temple in ancient Greece.

Definitions

"A method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem".^[3]

“Technique to arrive at a group position regarding an issue under investigation. The Delphi method consists of a series of repeated interrogations, usually by means of questionnaires, of a group of individuals whose opinions or judgments are of interest. After the initial interrogation of each individual, each subsequent interrogation is accompanied by information regarding the preceding round of replies, usually presented anonymously. The individual is thus encouraged to reconsider and, if appropriate, to change his previous reply in light of the replies of other members of the group. After two or three rounds, the group position is determined by averaging”. ^[4]

General Concept

It is an essential project management technique used to gather information, the goal is to reduce the amount of different responses and reach the consensus among the experts (stated by some studies to be between 70% and 51%)^[5],^[6]. This method is recognized, adopted and is it still used nowadays. Treating uncertainty is a central issue in project management, an entire literature about risk management can be found if interested to have more explanations: “Project management is about reducing uncertainty step by step until it has been eliminated completely, for better or worse, and we achieved a particular result” ^[1]. This method tries to break the stereotype that says: “When you get three experts together, you'll often end up with four different opinions.” Every project has his own characteristics and needs, consequently there is often not just one-size-fits all solution, we have to narrow down all parts to find the customized solution for the specific case. The participants in the process do not need meet, sometimes they even do not know who are the other participants, only the facilitator knows everything and manages the information flow in order to avoid possible repercussions. The anonymity helps avoiding personal issues between member and the so-called “groupthink”. The phenomenon of groupthink can be really dangerous in an organisation, is marked by the consensus of opinion without thinking critically and evaluating other alternatives and consequences. The core point of this mistake is to do not break the status quo of the group, avoiding conflicts and creativity, this will reduce the pressure of the group toward conformity.

The structure of this technique give time to the participants to think properly about their opinion and responses, being so able to contribute with full hand at problem to be solved or in the discussion. Moreover, the iterative nature of this approach gives the opportunity to refine, narrow down and test the arguments of the members (being able also to tone down some possible hard statements).

Figure 1^[7],^[8],^[9]: Example of Structure of Questionnaires

Since are scheduled multiple rounds of questions in the panel, the thoughts of the groups are really investigated and studied, the communication has to be facilitated in every moment. Also, there is the opportunity to aggregate different opinions from experts without the need to have a physical meeting; it is different also from other methods because it is flexible and there are incorporated qualitative and quantitative approaches in it. This technique is used in the prediction of the future as Helmer, one of the inventors , said: “The future is no longer viewed as unique, unforeseeable and inevitable; there are, instead, a multitude of possible futures, with associated probabilities that can be estimated and to some extend, manipulated”^[10], one of the main aim with this method is the anticipation and recognition of needs.

According to (^[11][12][10]) the main feature of consensus are: (1) Anonymity, in order to avoid some dominant character and implemented using private questionnaires; (2) Iteration, the structure with rounds allows the participants to change their opinions along the process; (3) Controlled feedback, showing the different responses of the individuals inside the group; (4) Statistical group response, measuring and judging the summary of all the responses, having much more detailed information.

Respondents usually rate their agreement on a specific statement or sentence with a five point scale, for instance, (1) means “strongly disagree” and (5) “strongly agree”. In the end the topics with 4 as average (or higher) and with standard deviation below 1 are considered areas in which the agreement is found (the so-called: “items of consensus”). There is not the purpose to arrive always at the unanimity among all the experts, conversely a spread of opinions in the end is a common outcome (limitation and risk).Typical evidences shows that the opinion is converging during the phases and the average response moves in the direction of the true reply; this is mainly due to three different factors: social pressure, re-thinking of the problem and movement of information during the feedback session. ^[7] In the decision making process mainly three kind of information are playing a crucial role: knowledge, speculation and opinion (an entire set of data can be seen in : ^[8]).

The main suggestion to success using this technique are: searching for endorsement of an important person during the activity to sustain and keep high the involvement of all the participants. For example, calling the participants after the invitation or think about incentives for each round. Moreover, certain initial requirements are needed as: high level of effort among the participants, high skill level in writing the questionnaires and plenty of time available to conduct the process (i.e. 6 weeks). Empirical studies reveal that “Delphi Method is superior to conventional uses of groups in problem solving or forecasting” ^[13].

Advantages

Advantages:^[9],^[10]

• Versatile technique, lot of different applications
• Group experts from different sectors/backgrounds
• No location limits, not so expensive
• Promote reflection, evaluation and involvement
• Recognition of the contribution of each participant
• Easy analysis of data
• Anonymity: free thoughts
• ”It´s adaptability to diverse data collection strategies, decreased peer pressure secondary to anonymity and the ease of condensing opinions of many and varied experts into a few precise statements”^[14]
• Overcome the problem of few individual dominating discussions

Steps

Generally, ten steps for the Delphi method are indicated:^[7],^[8]

• 1) Formation of a team on a subject
• 2) Selection of expert panels
• 3) Development of the first questionnaire
• 4) Testing the questionnaire
• 5) Transmission to the panel lists
• 6) Analysis of responses
• 7) Preparation of second round
• 8) Transmission of second round questionnaires to the panel lists
• 9) Analysis of the second round (iteration)
• 10) Preparation and presentation of report

Figure 2^[11]: Delphi Method Procedure Overview

Types of Delphi

Types of Delphi: ^[9]

• Classical Delphi: 5 features: Anonymity, Iteration, Feedback, Statistical group response and Response stability among experts on a specific issue.
• Policy Delphi: the aim is generate policy using a structured dialogue, it is possible to have several divergent opinions from experts. The features are: Answer questions individually but together in the meeting, Iteration, Feedback, Polarized group response and conflicts.
• Decision Delphi: used to reach an high grade of involvement among decision makers on social issues, from different hierarchy positions. Here the names are mentioned, but the responses to the questionnaires are anonymous.

Strengths and Weaknesses

Strengths:^[10],^[11]

• Less problems in group interaction
• Common thoughts periods allowed
• Listen also to minor views
• Both qualitative and quantitative data produced

Weaknesses:

• Risk of false consensus
• Problem of selection biases
• It require a strong/able facilitator

Nominal Group Technique

A modified Delphi method is called “The nominal group technique”^[11], the main difference is that the meetings are done face to face, (starting with an open-ended question) aiming at a group discussion to reach consensus. Ideas and thoughts about the topic are shared and recorded, each person evaluate the ideas anonymously and individually votes for the best ones keeping the anonymity; finally a report with the responses is prepared. NGT is more structured than the Brainstorming technique, obtaining multiple inputs from experts on precise issues. Used mostly in educational contexts, this specific tool is helpful because researches shows that individuals produce more ideas when working alone and they do it without sacrificing the quality. As it is for Delphi, the facilitator here has to be an expert in the field or a person with credibility among the participants. This technique can be useful when there are little evidences but some decisions needs to be made, or to rank priorities during a group meeting.

Figure 3^[11]: Stages of NGT technique

Applications

The most common types of applications are: when there is the need to explore an issue with a group of people, when you want to move a group towards consensus, when a particular problem does not permit the use of an analytical tool, when the participants are not able to meet physically or they are too many. This method can be used for several reasons: helping the decision making process as a group, to gain inputs from experts, to define some needs, to identify results, to prioritize some causes, to suggest solutions, to define future goals, to anticipate outcomes, to plan and schedule, for problem solving, for data collection, to identify dimensions of a problem or to forecast. Delphi Method can be done with small or big group (hundreds of people), but the common way schedule nearly 20 people. It is useful especially when there are many experts to contact, meaning that the consensus is difficult to reach and when the experts are located in different parts of the world. ^[3]

The first step is to select a group of experts based on the topic that has to be analyzed, later a questionnaire is sent to all the participants with instructions for each topic and on how to write opinions, experience and personal researches. These questionnaires are then sent to the facilitator that puts together all the comments and send a copy to each participant. At the end of every session all the questionnaire are sent again to the facilitator who will decide if there is the need of another round or not, the iteration can be done as many time as necessary to achieve the final consensus.

Other common applications

Other common applications in projects are:^[9],^[3]

• In time management to estimate the duration of activities
• In cost management to estimate the different costs
• In risk management to identify the risks
• In scope management to collect various requirements
• Government planning
• Business and industry
• Culture, family and behavior
• Education and training
• Communications
• International Security

Limitations

The limitations of this method are that the responses can take a lot of time. Also the rate of discussion can be low, sometimes the interaction is not right as it is in a face to face meeting. Finally, it is possible that the information received from the experts are not going to create new value.

Disadvantages

Disadvantages:^[9],^[10]

• Facilitator´s fail (wrong expert or questions selection)
• Long time needed using mail or post
• Participants involvement is a requirement (quality of the responses usually decrease with the passing of time)
• Individual point of view
• Anonymity may lead to lack of responsibility/accountability
• Labour intensive
• ”Conceptual and methodological inadequacies, potential for sloppy execution, crudely designed questionnaires, poor choice of experts, unreliable result analysis, limited value of feedback and consensus, and instability of responses among consecutive Delphi rounds.” ^[15]
• Little research in assessing the reliability of this method
• Risk of “groupthink” phenomenon

Study Case

“The Future of the academic library and the academic librarian. A Delphi study reloaded”. ^[16] This case is a nice example of the application of Delphi method in a project. In 1999 in Chania, Greece was done a study on a project to identify academic library and librarian role, use in the present and future scenario inside an educational program, in 2005 this work was revised, due to the changing that are characterizing the world, especially in this case the improvement of the technology. These factors all together contributed to change the image of library and librarian through these years and so a new evaluation of this process was needed. The aims were to identify both the results in the management of the previous project goals/purposes and also focusing on the improvements to develop in order to maintain this project alive. So it was done both for forecasting new issues that can be crucial for the project and also for evaluating with use of experts (in this case panel made by librarians and insiders), connection of a real case with the theory previously explained, comparing also the old forecast made with the previous situation and the new one.

Steps

The participants were 36 coming from 20 different countries (online meeting application) in 4 continents. The steps used in this specific case (accordingly with the theory) were: ^[16]

• Formation of the panel of professionals
• Development of the first questionnaire
• Distribution of the first questionnaire
• Analysis of the first round responses
• Preparation of the second round etc.
• Clarification of some points through the third round
• Final analysis and presentation of results

Rounds and Questionnaires

Mainly because this method is time demanding they did not have the opportunity to test it before effectively starting, this meant some misunderstandings in the beginning, but it went better and better with the passing of rounds. In the first round the questions were about the differences between 2005 and future (2015) and rate the things decided in the past (1999). In the second and third round the focus was on librarians and academic libraries among four categories: 1) internet as a competitor, 2) local vs. remote access, 3) printed vs. electronic media, 4) staff and user training. Examples of the questionnaires (rounds 1, 2, 3) can be found in the picture below (for more detail check the references):

Figure 4^[16]: Part of the questionnaire round 1

Figure 5^[16]: Part of the questionnaire round 2 and 3

Example of First Round Results

To have an idea and a better understanding of this technique there will be available also some results (even if they are just a little part, for more detail check the references), for instance during the first round the participants were asked to reply to four topics about present and future; using a ranking scale between 5 and 0, where 5 means “really important” and 0 “not important at all”, applied to this data mean and median were calculated. An example can be seen in the graph below:

Figure 6^[16]: Data example of topic1

Figure 7^[16]: Data example of topic2

Comparison between present and future

For every topic some consequent observations and remarks were made and analyzed in order to improve; moreover, the comparison between the present (2005) and the future (2015) was underlined using the median values of the ranking (see below), together with the relative observations and remarks:

Figure 8^[16]: Present and Future comparison1

Figure 9^[16]: Present and Future comparison2

The second and third round were done and analyzed in a similar way based on the four areas underlined before, together with some open questions to have some personal opinions from the experts and consequent presentation of results.

Conclusions

Some main conclusions at the end of this study case were done:

• Library will still exist in the future (2015)
• Their main activities will change slightly compared with the actual state
• Libraries will need to improve in the implementation of IT and have more skilled employees
• Libraries will be used also for social scope and for studying
• Librarian will be a value added to the science found on internet.

Other cases can be seen in ^[17], ^[18],^[8],^[9] if interested.

Annotated Bibliography

Brief explanation of some of the references used:

^[1] General overview of Projects, Program and Portfolio. Used also as a starting point for the Delphi method mention.

^[9] Literature review of Delphi, good examples of advantages, disadvantages and different variant of this technique.

^[16] Case Study used as an example of the real life application of the Delphi, took also some data and picture of their statistical results.

^[10],^[7],^[8] Pioneer contributions in this field in RAND corporation about Delphi with case studies and data available.

Word count 2984.

References

1. ↑ ^{1.0 1.1 1.2} Geraldi J., Thuesen C., Oehmen J., How to do Projects, Version 0.5
2. ↑ Maylor H., Project Management, 4th edition, Pearson Education Limited 2010
3. ↑ ^{3.0 3.1 3.2} Linstone, H.A. and Turoff, M. (eds.) (1975 and 2002 reviewed version) The Delphi Method Techniques and Applications.
4. ↑ Web Dictionary of Cybernetics and Systems http://pespmc1.vub.ac.be/ASC/indexASC.html; IIASA.
5. ↑ Broomfield D, Humphris GM. Using the Delphi technique to identify the cancer education requirements of general practitioners. Med Educ 2001;35:928-937.
6. ↑ McKenna, H, Hasson, F, & Smith M. A Delphi survey of midwives and midwifery students to identify non-midwifery duties Midwifery 2002; 18 (4): 314-322.
7. ↑ ^{7.0 7.1 7.2 7.3} Dalkey N.C., Delphi, 1967, The RAND Corporation, Santa Monica, California
8. ↑ ^{8.0 8.1 8.2 8.3 8.4} Dalkey N.C., 1969, The delphi method: an experimental study of group opinion, prepared for united states air force project RAND
9. ↑ ^{9.0 9.1 9.2 9.3 9.4 9.5 9.6} Hanafin S., Review of literature on the Delphi Technique, March 2004
10. ↑ ^{10.0 10.1 10.2 10.3 10.4 10.5} Helmer O., 1967, Analysis of the future: The Delphi method, The RAND Corporation, Santa Monica, California
11. ↑ ^{11.0 11.1 11.2 11.3 11.4} Bishop C., Pitchforth E, Russell E., van Teijlingen E., Delphi method and nominal group techniques in family planning and reproductive health research
12. ↑ Williams PL, Webb C. The Delphi Technique: a methodological discussion. J Adv Nurs 1994;19:180-6.
13. ↑ Gordon T.J., Helmer O., 1964, Report on a long-range forecasting study
14. ↑ Snyder-Halpern, R. (2002) ‘Indicators of organizational readiness for clinical information technology/systems innovation: a Delphi study’, International Journal of Medical Informatics 63 179-204.
15. ↑ Gupta, U.G. and Clarke, R.E. (1996) ‘Theory and Applications of the Delphi Technique: A bibliography (1975-1994)
16. ↑ ^{16.0 16.1 16.2 16.3 16.4 16.5 16.6 16.7 16.8} Feret B., Marcinek M., 2005, “The Future of the academic library and the academic librarian. A Delphi study reloaded”, Purdue University.
17. ↑ Giannarakis G., Litinas N., Theotokas I., 2010, "A Delphi Study to identify corporate social responsibility indicators: The case of Greek telecommunication sector"
18. ↑ Hsu C.C., Sandford B.A., 2007, The Delphi technique: Making sense of consensus, Practical assessment, research & evaluation.