Risk Quantification and Methods

Many projects fail to complete in original cost and time estimates due to inadequate risk quantification. Risk quantification is a process to evaluate identified risks to produce data that can be used in deciding a response to corresponding risks. It is a 2nd step of project risk management, after risk identification and before risk response development and risk response control according to PMBOK standard. The objective of project risk quantification is to prepare contingencies in terms of costs, time, or human resources and prioritize them. PMBOK, ISO 31000, and PRINCE2 provide principles and processes for effective risk management. Risks are quantified by using either expert intuitions or statistical tools. Five techniques as proposed by PMBOK standard for risk quantification have been reviewed in this article. These tools provide various advantages for risk quantification but also have their limitations. These limitations as well as the challenges and limitation of the risk quantification process, are important to consider to ensure effective risk management. The process of risk quantification is an important step of the risk management process and therefore, important to ensuring the success of a project.

Introduction

Definition

PMBOK describes risk quantification as "evaluating risks and risk interactions to assess the range of possible outcomes". In general, "risk quantification is a process of evaluating the risks that have been identified and developing the data that will be needed for making decisions as to what should be done about them" [7].

Inputs and Outputs of Risk Quantification

In risk quantification process of a project, there are inputs that should be considered with delegate care and as a result of risk quantification process, outputs are generated. According to PMBOK, following inputs are considered and outputs are produced in risk quantification process of any project:

Table 1: Inputs and Outputs to Risk Quantification in Project Management (PMBOK)

Inputs	Outputs
Stakeholder Risk Tolerance: Every organization and different individuals may have different tolerance for risk value	Opportunities to Pursue, Threats to Respond to: The list of opportunities that should be pursued and threats that should be taken care of.
Sources of Risks: Categories of possible risk events that may negatively affect the outcome of a project. For example, designs errors, stakeholder actions, or poor estimates etc.	Opportunities to Ignore, Threats to Accept: List of opportunities that can be ignored and threats that can be accepted.
Potential Risk Events: Discrete occurrences that can occur during a project that may affect the outcome of the project. Such as natural disaster or departure of key member etc.
Cost Estimates: Assessment of likely cost required to complete the project activities.
Activity Duration Estimate: Quantitative assessment of likely number of work period required for activities of a project

Purpose and Concept

Figure 1: Example of risk matrix of a project [1]

The objective of risk quantification is to prepare contingencies in terms of costs, time, or human resources and prioritize them in terms of their severity and likelihood, so that appropriate action can be taken accordingly . In order to quantify risk, it needs to be identified first. Once risk is identified then it is analyzed in terms of probability of occurrence and impact that it could print on the outcome. The probability is assigned either based on intuition or the previous data of failure rates available for similar events in datasheets. Once probabilities of all events are calculated, a criterion for the likelihood of all the events is defined. For example, if a specific event may occur in exceptional circumstances, like for example less than 3% chance of occurrence, then its likelihood can be assigned as “Rare”. In a similar way, severity or consequence of the events on a project is also classified. For example, if an event may result in abandonment of project then it can be classified as “Catastrophic” or if it may result in a delay of 50% of schedule or 50% of additional cost then it may be classified as “Major”. The risk(R) is calculated by multiplying probability(P) with the impact(I) or severity.

$R = P\times I$

Once risks are quantified then these are evaluated against a defined risk criteria or risk matrix. Red zone in a risk matrix may represent unacceptable risks, yellow zone as an acceptable risk, and green zone as neglectable risks. For example, if an event has a likelihood of class “Likely” and it has a severity class “Catastrophic” then it may lie in the red zone of the risk matrix. This may mean that this risk is not acceptable and appropriate or immediate actions should be applied to lower this risk into the acceptable zone or prepare contingencies. Figure 1 shows an example of risk matrix of a project. The first column represents criteria for likelihood, whereas, the first row represents criteria for consequence. Further, nature of any possible risk is defined based on both likelihood and consequence from low, moderate, high, to an extreme.

Importance

Figure 2: Causes of project failure [2]

The term risk or risk assessment may sound like a modern scientific concept, but the idea of risk is as old as recorded human history. The gambling, the very essence of risk, was a popular pastime that inspired Pascal and Fermat’s revolutionary breakthrough into laws of probability . However, Risk as a scientific field is quite young. Around 30-40 years ago scientific journals, papers, and conferences started to cover this idea and principles on how to assess and manage risk . One of the main reasons of project failures is inadequate risk management. Figure 2 shows that 17% of projects fail due to inadequate risk management. Moreover, according to Standish Group (2013), 59% of IT projects overrun by original cost estimate and 74% are overrun by original time estimate. In software or IT projects, a number of factors contribute to the uncertain outcome of a project. Nogueira et al. (2014) concluded that when a scope is defined and software production teams are guided through the risk process then it becomes easier to take a rational decision. Present decisions may result in future losses or gains. If there is no risk assessment then banks will not be able to make decisions on which projects to finance and which not. Many construction projects fail to achieve their time, cost and quality goals due to several unforeseeable uncertain events like weather conditions, subcontractor failure, or different site conditions. Comprehensive risk assessment can help an organization to quantify risks and prepare contingencies beforehand so that projects can be completed in their original time, cost, and quality estimates.

This implies that the importance of risk assessment cannot be overlooked. First, risk quantification help in preparing contingencies for time and cost estimates. Second, It helps organizations in taking a rational decision in the presence of uncertainty. And third, it provides confidence of dealing unforeseeable events in future rather than acting irrationally.

Analysis of Risk Management Principles and Processes

Figure 3: Risk Management Processes of different Standards

Several standards exists that define principles of managing risks for effective risk management in an organization. Table 2 provides the comparison of risk management principles by PMBOK, ISO 31000, and PRINCE2. It can be seen that PMBOK and PRINCE2 follow almost the same principles. This might be due to the fact that both standards are designed for project management practices. Whereas, ISO 31000 lists a few similar principles and at the same instance also provides somewhat different principles as well. The generality of the scope of ISO 31000 might be one of the reasons. Although, each standard does not explicitly describe the uncommon principles as listed in table 2, but these are meant to be the part of the risk management process. For example, ISO 31000 clearly describes that human and cultural factors should be considered in risk management, while PRINCE2 does not explicitly list this principle but clearly categorizes these risks separately.

Figure 3 represents risk management processes of three standards. It can be seen that there is a small difference between processes of these standards, but when the definition of each step is critically analyzed, it can be realized that the basic concept behind all of these standards is not different. This implies different standards divide the risk management process into different steps but the core concepts remain the same. For example, PMBOK defines the third step as risk response development which means categorizing of assessed risks into acceptable or unacceptable risks and developing of responses accordingly. Whereas, almost the same definition exists for risk evaluation step in ISO 31000.

Table 2: Risk Management Principles

Application

Methods

Several tools and techniques are used in order to apply risk quantification in projects. PMBOK provides 5 methods that can be used in risk quantification process. These tools and techniques are described briefly below, along with application, advantages, and disadvantages of each tool.

(1) Expert Opinion

Figure 4: Example of expert opinion in a project (source: Yildiz A. Z. et al, 2014)

Merriam Websterdefines expert opinion as, “a belief or judgment about something given by an expert on the subject”. Expert opinion is one of the risk quantification techniques. In expert opinion, risks are quantified based on the opinions of experts or senior executives based on their experiences. One of the best ways to use expert opinion is to conduct risk assessments workshops where experts can discuss and consequently assign values to the risks identified. But, this may lead to group bias and can affect the outcome. This bias can be minimized by using Delphi method, but there still be a chance of high variation in opinion. [8] Although, expert opinion is not as concrete, as other methods may be, and may prone to personal subjectivity, but it is a very useful tool for risk quantification when data is scarce or no sufficient past experience is available or where risks are very company or project specific.[9] Figure 4 shows an example of risk quantification using expert opinion in a case study on construction project conducted by Yildiz et al. (2014). The ratings are estimated ratings, quantified by SEM (Structural Equation Modeling) [10] software based on the sub risks and attributes ratings assigned by experts using 1-5 Likert Scale.[11]

(2) Expected Monetary Value (EMV)

Figure 5: Example of EMV [3]

Expected monetary value is another way to quantify risk. According to PMBOK , expected monetary value is a product of two numbers, risk probability value and risk event value which is an estimate of loss or gain that will be incurred if the risk event occurs. These values can be positive and negative resulting in gain or loss respectively. For example, if there is 60% probability that a certain equipment will fail during a project that will result in USD10,000, then EMV will be USD -6,000. Figure 5 shows an example of EMV analysis. It can be perceived that a total of USD4,500 is required as a contingency, but in actual only USD1,100 are required as all of the events are not going to happen. This means, the risks which are not going to happen will add their value to EMV pool, where risks that are going to happen will utilize value from this pool. Hence, for this example, a project manager can add extra USD1,100 into project budget as contingency[12]. EMV helps project managers in two ways. First, it helps to manage to estimate the amount required to manage all identified risks. Second, it helps in selecting the choice to manage the risk by selecting the option with the minimum value.[13] EMV is generally used as an input to further analysis, for example, in decision trees. Benefits of using EMV are that it provides help in calculating contingency reserves, in procurement planning decision-making, in spreading the impact of a large number of risks, and in decision tree analysis. Whereas drawbacks of using this technique are that this technique is not used in small and small-medium sized projects, use of expert opinion may result in personal bias, and the chance of forgetting of inclusion of positive risks. [14]

(3) Statistical Sums

Figure 6: Example of Statistical Sums (3 point estimates) (source: PMBOK (2013)

Statistical sums is another way to quantify risks. In this technique cost estimates of individual work items are calculated and then are used to calculate range of total project costs using statistical probability distribution. The range of different project costs can help to quantify relative risks of alternative project budgets (PMBOK). In this method, instead of using one point estimate, 3 point estimates are used. Cost of each work item is estimated through 3 points of likelihood i.e. low, likely, and high. Then statistical distribution such as normal distribution or beta distribution is used to calculate mean and variance. To calculate mean and variance of total project estimate, means and variances are added together for all work items. Figure 6 shows an example of this method. It is an easy technique for calculating budget and time contingency of a project, but it cannot be used for unforeseeable risks that may happen during a project. Further, as estimates are provided on expert opinion bases so it may subject to personal bias.

(4) Monte Carlo Analysis or Simulation

Figure 7: 3 point estimates of e-learning project [4]

Figure 8: Result of probability distribution as a result of Monte Carlo simulation [5]

Monte Carlo is a computerized mathematical simulation technique that is used to quantify risks in project management. This technique is helpful in seeing the probable outcomes of decisions and assesses the impact of risk that is useful in decision-making [15]. Most likely and least likely estimates of risks are provided for each event and then these estimates are summed together to calculate a range of possible outcomes. Monte Carlo simulation then generates random values between the range and calculates the number of occurrences the value lies within each possible outcome. This probability is then distributed and the decision is made based on the most probable outcome. For example, if there are three tasks required in an e-learning project. Best case, most likely, and worst case estimates of all the tasks required are given in figure 7. It can be seen that the project is most likely to complete in between 11 and 23 days. Now for example, if Monte Carlo simulation is run 500 times generating random values between 11 and 23. The total number of times the simulation result was less than or equal to projected duration is calculated. Then, the probability of each projected duration is calculated and distributed as shown in figure 8. It can be seen, from figure 7, that the most likely projected completion time is 17 days. But, as per figure 8, Monte Carlo simulation shows that likelihood of project completion in 17 days is almost 33%. Whereas, the likelihood of project completion in 19 days is 88%. Hence, it can be estimated that the project will most likely complete in 19 to 20 days. [16]

Monte Carlo simulation is usually used in cost and schedule estimation. It can also be used in large projects or programs. The benefits of using Monte Carlo are easiness of tool, numerical estimation, and greate level of confidence [17]. Whereas drawbacks or challenges are the use of right distribution as wrong distribution may lead to wrong results, input estimates as right estimates are required to produce right results, and use of right mathematical formula in the software.[18]

(5) Decision Trees

Figure 9: Example of decision tree analysis [6]

Decision tree is a tool that uses tree-like graph or model of decisions and their corresponding consequences [19] that can be used to quantify risks and make a decision under uncertainty in a project. Expected Monetary Value (EMV) is usually used to quantify risks, where probability(P) of an event is multiplied by its impact(I) to calculate the EMV. For example, if there is a decision to make under uncertainty that whether to make a prototype or not in a project. This decision has only two options, prototype, and no prototype, shown in figure 9. Each of these choices has two consequences, success or failure. The probability of each consequence is also shown in figure 9. Impact in terms of costs for each option or chance and consequence or outcome is also shown in figure 9. Net path value for prototype with 70% success is equal to payoff minus prototype cost i.e. $500,000 - $100,000 = +$400,000. Similarly, net path values for rest of the paths are also shown in figure 9. EMV value for the path option of prototype is then calculated as [70%*($400,000) + 30%*(-$150,000)] = +$235,000. Similarly, the EMV value for no prototype is -$100,000. Hence, EMV value at decision node will be +$235,000, which means that the project manager should decide to select prototype option as the other option actually gives a loss. [20] Benefits of using decision tree analysis are ease of understanding and implementation, quantification of even little hard data, and a possibility to add several new scenarios. While disadvantages are biases of input data and increase in complexity for a large number of outcomes that are linked together. [21]

Selection of Technique

Selecting the right technique for right project can be tedious. Several factors can influence on the selection of the right technique for the right project. Some of these factors include:

1. Resources and capabilities required to execute a certain risk quantification method
2. Degree of uncertainty in the project
3. Complexity of the project
4. Availability of the past data

Table 3 shows a framework for selecting the right method based on the nature of the project. (This framework provides author’s subjective analysis and hence prone to disagreement.)

Table 3: Incfluencing factors to select right technique for project

Limitations and Challenges

One of the limitations in risk quantification is that probabilities are estimated either by past history or in some cases by expert opinion or intuition. Both of these cases cannot define the probability of an event with 100% certainty, which means no matter how much effort is put in risk quantification process, it can never be completely accurate. Another challenge is the quantification of the impact in terms of cost or time. It is very difficult to correctly estimate the exact cost of the impact or consequence even with utmost care. Although, risk quantification provides contingencies in terms of costs and time, still, several unforeseeable events can occur that may result in a project failure. Hence, risk can be quantified to a certain extent, but full confidence level cannot be assured.

Both modeling of the system and quantification of probabilities associated are tricky and prone to uncertainty especially when a complex system is under study. In probabilistic risk assessment, the subjective probability is used rigorously which means it is subject to human intuition and may vary from person to person . Further, availability of past data poses another limitation as many experts tend to use probabilistic values of similar past events due to the scarcity of the data. Although, methods or tools that are used in risk quantification process of a project, as mentioned in section 3, try to reduce the uncertainty level to some extent and help in building up confidence level, but the inputs to these methods are also prone to limitations of intuition and hence pose challenges in accurate risk quantification. Several researchers provide guidelines to deal with uncertainty in quantifying risks . But, All these facts, makes one question that when risk assessment or quantification cannot guarantee the success of a project then why do managers invest so much effort and money into risk quantification. If risk quantification cannot accurately predict the future, then why to do it in the first place. The answer lies in a famous phrase “better than nothing”. It is always better to perform risk assessment beforehand and be prepared for uncertain events than drastically act on them unprepared when they occur. Further, more research is required in the risk-benefit analysis in order to justify risk assessment process. Moreover, more and extra care is required in assigning probabilities and impacts to get a more accurate risk assessment.

Conclusion

Risk quantification is very important in project management and its importance cannot be overlooked. It helps in quantifying risks and aid in making rational decisions. It also helps in preparing contingencies for cost, time, and human resource estimates. Several national and international standards exist that explain the principles and processes of risk management. All of the standards are based on the same fundamental core concepts and organization can use any of the standards that best suits them. Risk can be quantified using several methods proposed by different standards. These methods can be applied to different projects based on their nature and influencing factors. Although, risk quantification help managers in seeing a quantitative output, but personal subjectivity to probability and impact creates challenges in risk quantification. Nonetheless, risk quantification should be an integral part of decision-making rather than irrational acting on the unforeseeable events.

Annotated Bibliography

1. Luko, Stephen N. “Risk Management Principles and Guidelines.” Quality Engineering, Vol. 25, No. 4, (2013), pp. 451-454. Web. This article analyzes principles and guidelines for risk management as outlined by international standards i.e. ISO 31000-2009 and ANSI/ASSE Z690.2-2011. This article provides deep review of effective risk management and its processes. It highlights the importance of risk management in quality management of an organization.
2. Jamshidi, Afshin et al. “Risk Assessment in ERP Projects Using an Integrated Method.” 3rd International Conference on Control, Engineering and Information Technology (ceit 2015), (2015), 7233184. Web. This article highlights the importance of risk assessment in Enterprise Resource Planning (ERP) projects and reasons on why these projects fail. This article proposes a framework based on Fuzzy Failure Mode Effect Analysis (FFMEA) and Grey Rational Analysis (GRA) tools that intends to help managers in identifying and mitigating risks in ERP projects. This framework also provides risk evaluation and help in listing critical risks. This framework can be easily expanded and modified. This article may prove useful for supply chain professionals interested in risk management in ERP.
3. Milena CHOLES ARVILLA, Sandra. “RISK ASSESSMENT IN PROJECT PLANNING USING FMEA AND CRITICAL PATH METHOD.” Scientific Papers Series : Management, Economic Engineering in Agriculture and Rural Development, (2014). The goal of this article is to analyse current risk management methodologies and integrate its elements to create a new agile risk management methodology. The focus of this article is software projects. This article discusses the elements of quality assurance tools that could meet agile development and discusses the possibility of using risk estimation in agile projects. It uses concept of failure mode effect analysis (FMEA) into life cycle of agile projects and produces a metamodel. This article might be useful for readers interested in application of risk assessment in agile projects.
4. Bogumil, R. J. “Limitations of Probabilistic Risk Assessment.” Ieee Technology and Society Magazine, Vol. 1, No. 3, (1982), pp. 24-28. Web. This article provides critical analysis of probabilistic risk assessment techniques. This article argues that probabilistic techniques attempt to quantify likelihood of events on mathematically generated physical model, but fundamental social issues remain unresolved. Hence, proposes a need of prospective risk/benefit analysis.