Delphi Method (expert for identification)
The Delphi method is a technique used, primarily, to make quantitative assessments basing the anonymity as one of the main features compared with other measure methods. Normally is referred as a ‘quasi-anonymity’ because the respondents know one another, but their opinions and judgments remain in complete anonymity always. The technique consists in the structuring of a process of communication through experts to solve complex problems by series of questionnaires per rounds. The name was coined by the philosopher Kaplan, in honour of the Oracle of Delphi, at the beginning of the Cold War to obtain the most reliable consensus of opinion. This is characterized as a method for structuring a group communication process, making it more effective under the premise of a group of judgments is more valid than individual ones for dealing complex problems.
Organizations and companies use it when they want to tackle significant decision-making that will set the future guidelines. The results of this analysis have direct implications for the development or revision of a lot of aspects inside the companies. Furthermore, there are relevant applications of this method as forecasting, issue identification, prioritization or concept and framework development that makes it unique and a very powerful tool. The method accuracy can be evaluated by comparing it with other methods of measure with direct interaction, structured and unstructured. After the application of this method, other supplementary analysis to take into account may be the distributional estimates, the individual skills learning and different forms of feedback, as ask the subjects to write their reasons for the answers. Finally, the ethical considerations that have this method are delicate because the participants do not meet face to face, so they can react to ideas unbiased by the participants' identities. This article explains the Delphi methodology and their principles aspects and applications.
Contents |
Introduction
The Delphi method is a technique used to make quantitative assessments to solve complex problems by series of questionnaires made to experts in the fields of study. Companies consider it, mainly, when they have to tackle important decision-making that will set the future orientations. The results obtained in the analysis have direct implications for the evolution of their procedures, policies or programs (Loo 2002).
History
The first experiment using the method was implemented for the improvement in the scores of horse races. The name was coined by the philosopher Kaplan, in honour of the Oracle of Delphi, at the beginning of the Cold War to obtain the most reliable consensus of opinion (Loo 2002). All this work was secret because of its warlike character. Then, in the early 1960s, some information about the method commenced being published in literature (Woudenberg 1991). The Delphi method has been used by many different disciplines during the years trying to get the maximum accuracy in every study field and continues these days being a powerful tool in development and improvement.
Objective of the method
The main objective of this technique is to obtain the most reliable predicted future based on the consensus of a group of experts in the study field (Okoli & Pawlowski, 2004).
Techniques
Delphi method exists in two different forms. Commonly, is used the paper-and-pencil version (called ‘Delphi Exercise’). In this situation, a small team design and send a questionnaire which is sent and answered to the respondent group. After this questionnaire is returned to the developers and they summarize the results. Based on them, is been developed a new questionnaire for the respondents and successively. The respondents have one opportunity, at least, to reevaluate their answers. Moreover, there is another one called ‘Delphi Conference’ that is increasing in popularity over time. This technique replaces the team by a computer which has been programmed to compute the results of the group of study. Comparing both techniques, there are advantages and disadvantages e.g. the ‘Delphi conference’ eliminates the delay caused in the summarization of each round. However, to apply it is required that the communication characteristics be perfectly defined to avoid possible compilation errors or misunderstanding (Linstone & Turoff, 1976).
Main Characteristics
The key characteristics of the Delphi method can be defined as seven different (McKenna 1994):
- The anonymity has to be guaranteed for the participants.
- The usage of a group of experts for obtaining data.
- The usage of two or more rounds between making a summary of the results of the previous round communicated to and evaluated by panel members.
- The participants or respondents can meet in face-to-face discussion.
- Systematic emergence of a concurrence of judgment or opinion.
- The usage of sequential interviews and/or questionnaires.
- The usage of frequency distributions to identify agreement patterns.
Additionally, Robert Loo (Loo 2002) express his point of view of the characteristics as five main points that can be summarized in the seven presented before and the necessity of at least three or four iterations of questionnaires and feedbacks.
Methodology
The methodology of this technique, can be divided in 3 simple points (Okoli and Pawlowski 2004):
- Selection of the Delphi methodology: make the schedule and organize which technique is going to be used as the number of rounds and participants is going to be. Delphi is very desirable because do not require that the
experts meet personally.
- Procedure for selecting experts: Being ensure of the identification of the relevant experts and invite them to participate in the study. This task is especially critical for the further development of the method because
the group of decision making requires qualified experts that have deep understanding of the issues of study. The procedure consist in five steps:
o Prepare a Knowledge Resource Nomination Worksheet.
o Populating the Knowledge Resource Nomination Worksheet with names.
o First-round contacts - Nominations for additional experts.
o Ranking experts by qualification.
o Invite the experts to the study.
- Data collection and analysis method: The mechanism for administering the questionnaires can be by any communication platform (e.g. Email). For administrate them, are involved three steps:
o Brainstorming the important factors.
o List the most important ones.
o Rank the list to let the participants know.
Furthermore, in Figure 1 can be seen the simple theoretical structure of the complete process from the application requirements until the group response (data collection) (Mirzaaghabeik and Vosoughifar 2016).
Figure 1. Theoretical framework of Delphi qualitative in research (image adapted from (Mirzaaghabeik and Vosoughifar 2016) for the article).
Number of participants
The sample size is a practical consideration when the research is faced. For the number of participants, there are not fast or hard rules, but it has to be considered a number of factors (Skulmoski, Hartman, and Krahn 2007): - Heterogeneity of the sample: when is a homogeneous group, a smaller sample of between ten to fifteen is sufficient to obtain significant results. However, if disparate groups are involved, the size of the sample increases
until hundred of participants. Heterogeneous samples can increase the difficulty and the complexity of collecting data, conducting the analysis and verify the results.
- Decision quality: There is a reduction in group error as size increases. - Internal or external verification: As the sample increases, the results obtained will be more convincingly can be to verify.
Number of rounds
The number of rounds is variable and depends mainly on the research purpose. In overall, with two or three iterations is sufficient for obtaining a good result, but depends directly on the homogeneity of the sample. If the work sample is heterogeneous, there will be required three or more rounds and, on the other hand, if the sample is homogeneous, fewer than three rounds may be enough to reach the consensus (Skulmoski, Hartman, and Krahn 2007).
Applications
Delphi has been approved rapidly with hundreds of studies worldwide. Furthermore, the applications of this method have grown in a lot of different ways e.g. social phenomena as human values and attitudes or quality of life (Sackman 1974). Okoli et al. listed examples of the method applications dividing them into two main applications: Forecasting and Issue (identification/prioritization) and Concept/ Framework Development. Table 1 reflects the method applications in information systems research. In the research process, the applications of the Delphi method are very diverse, it can stand out among them the specification of research questions, the selection of variables of interest/generation of propositions, the preliminary identification of causal relationships, the identification of the research topic or the definition of constructs and creation of a common language for discourse (Okoli & Pawlowski 2004).
| Application | Example studies |
|---|---|
| Forecasting and issue
Identification / Prioritization |
Identify the most critical issues in the coming 3-5 years
Forecast changing in the international business environment over the next decade and the changes impact Identify critical issues in IS in the coming 5 years Identify a prioritized list of international data communications activities vital to multinational corporations in managing information exchanges for control and implementation of global business strategies Forecast of the role of the systems in the 21st century |
| Concept / Framework
Development |
Develop a framework of the main areas of the IS field
Develop a descriptive framework of elemental knowledge manipulation activities Develop a capability-based typology of information technologies within the financial services industries Develop a conceptual taxonomy of organizational design actions-mechanisms to enhance technology users’ propensity to innovate in information technology Develop a ranked list of common risk factors for software projects as a foundation for theory building about IS project risk management |
Accuracy of the method
The accuracy of the Delphi method can be measured by the comparison between different judgment methods. In most studies, no statistical comparison between methods is made. So as is shown in Table 2, and based on bibliography (Woudenberg 1991), the judgment methods can be put on a scale of accuracy so that the less accurate is at the left and the most accurate at the right part.
| Year | Africa | Americas | Asia & Pacific | Europe |
|---|---|---|---|---|
| 2014 | 2,300 | 8,950 | 9,325 | 4,200 |
| 2015 | 2,725 | 9,200 | 8,850 | 4,775 |
Supplementary analysis
Lorem ipsum
Ethical considerations
Lorem ipsum
Annotated bibliography
Lorem ipsum
