Levels of uncertainty

Latest revision as of 21:08, 9 May 2023

[edit] Abstract

Uncertainty is a common challenge in project, program, and portfolio management. It can arise from a variety of sources, including unforeseen risks and challenges in project scope, budget, or timeline. In the understanding by Knight that uncertainty diverts from risk, by being non-quantifiable, identifying the level of uncertainty can, yet, be used by managers to strategically prepare and adapt to the unforeseen events.

This article will explore various parameters for classifying levels of uncertainty, such as knowledge, severity, and source, and provide guidance on when to appropriately utilize each. Additionally, effective strategies for managing uncertainty at different levels in project, program, and portfolio management will be provided.

The article concludes by highlighting the importance of adopting a systematic approach to uncertainty management and will provide valuable insights for project-, program-, and portfolio managers, as well as anyone involved in the planning and delivery of projects, programs, and portfolios.

[edit] Introduction

When uncertainty is mismanaged project performance is at great risk. One of the most common mistakes, when it comes to uncertainty, is to underestimate or ignore what cannot be measured. This is most often done either because of a lack of awareness or because of an optimism bias in the project. In the way that unmeasurable challenges probably not are “that big a deal”. Of course, uncertainties can, also, be downplayed and used as a deliberate way of overselling projects (Geraldi et al., 2017).

Uncertainty is a fact of life that we all must contend with. From personal decisions to corporate strategies, uncertainty is an ever-present challenge that can affect the outcomes of our choices. When it comes to managing uncertainty, there are various levels of uncertainty that managers can use to identify the threats, that lie up ahead. Poor uncertainty management at any of these levels can lead to consequences that range from minor inconveniences to severe financial losses or even personal harm. In this wiki-article, we will explore the different levels of uncertainty, as well as strategies for effectively managing uncertainty at each level.

The right indentation of uncertainty levels has many different roots within different fields. Therefore, there is no “one size fits all” way to assess uncertainty but uncertainty management must be seen as a collection of methods that can be applied to different situations (Riesch, 2012). Levels of uncertainty can, thus, be described based on knowledge, severity, or the source of uncertainty. When each is applicable and how to mitigate the uncertainty, will be described further in this article.

[edit] Nature of uncertainty

Uncertainty is a vital concept in project management and engineering systems design. Knight (1921) provided distinct definitions for risk and uncertainty in the context of economics. Risk refers to situations where the probabilities of outcomes are known or can be quantified. In contrast, uncertainty describes scenarios where the probabilities of outcomes are unknown or cannot be quantified, making the future less predictable and more challenging to manage . Through the years, this definition has been debated by practitioners and academics, as uncertainty in its essence is more broadly applicable. Building on this, Chapman and Ward (2012) emphasize that uncertainty management must address non-quantifiable uncertainties and unforeseeable events, highlighting that uncertainty essentially signifies "a lack of certainty.". Oehmen and Kwakkel (2020) further argue for a more comprehensive definition that encompasses risk, uncertainty, and ignorance as three distinct levels (see figure 1). These categories are essential to differentiate, as they require distinct management strategies.

1. Risk: At this level, uncertainties are characterized by known or quantifiable probabilities, much like in Knight’s (1921) definition. Outcomes can be estimated, allowing project managers and engineers to develop contingency plans and mitigation strategies.

Risk can be managed through Project Risk Management (PRM) strategies, such as identifying potential risks, assessing their impact, and taking appropriate measures to minimize or eliminate their consequences (PMI & Project Management Institute, 2019).

2. Uncertainty: This level deals with situations where the probabilities of outcomes are unknown or cannot be quantified. Non-quantifiable uncertainties make it difficult to predict and manage the potential effects on a project or system. In this context, uncertainty management emphasizes adaptability and resilience, requiring project teams to be flexible and responsive to changing circumstances. Techniques such as scenario planning, robust decision-making, and real options analysis can be employed to navigate uncertainties (Oehmen & Kwakkel, 2020).

3. Ignorance: The third level, ignorance, refers to instances where the potential outcomes are entirely unknown, making it impossible to devise any management strategy. In these situations, even the range of possible outcomes remains unidentified. Addressing ignorance involves developing an awareness of potential blind spots and fostering a culture of learning and adaptability. This can be achieved by encouraging open communication, embracing diverse perspectives, and promoting continuous learning to recognize and respond to emerging challenges (Oehmen & Kwakkel, 2020).

[edit] The purpose of understanding uncertainty

Uncertainty analysis is an essential aspect of project management and engineering systems design, as it enables organizations to make well-informed decisions by assessing the potential impacts of uncertainties on project objectives. Chapman and Ward (2012) propose two lenses for uncertainty analysis: the 'performance lens' and the 'knowledge lens.' The performance lens focuses on evaluating the potential impact of uncertainties on project objectives, such as cost, schedule, and quality, while the knowledge lens emphasizes understanding the source and nature of uncertainties to improve decision-making.

Oehmen and Kwakkel (2020) similarly describe two types of management activities. One for understanding each level of uncertainty, and one for managing each level of uncertainty. To understand uncertainty, gathering knowledge through testing prototypes, doing Monte Carlo simulations and similar activities are recommended. For managing uncertainty, on the other hand, developing governance frameworks, decision-making heuristics and similar guidelines or code-of conducts are recommended.

Together, these approaches facilitate effective uncertainty management and informed decision-making in engineering systems design (Oehmen & Kwakkel, 2020). However, determining the appropriate approach depends on the specific context, as there is no one-size-fits-all solution for uncertainty management (Riesch, 2012). The feasibility of accommodating or reducing a particular uncertainty remains an open question. To provide practical guidance for project, program, and portfolio management practitioners, this article will further explore other ontologies and delve deeper into understanding uncertainty levels. A more detailed understanding of uncertainties will enable practitioners to better identify and apply the most suitable management strategies.

[edit] Uncertainty based on knowledge (2 levels) – Why are we uncertain?

One way of creating levels of uncertainty is to consider the knowledgebase for the uncertainty. This reveals two inherent levels of uncertainty, as some concerns can be diminished by doing more research, and some are caused by internal randomness of the phenomena (Wiki: Epistemic vs. aleatory uncertainty)

Epistemic uncertainty

Epistemic uncertainty refers to uncertainty arising from a lack of knowledge or information about a particular system, process, or event. It is a type of uncertainty that can be reduced through gathering more data, improving models or theories, and enhancing our understanding of the underlying mechanisms (Kiureghian & Ditlevsen, 2009).

Effective management of epistemic uncertainty involves identifying and addressing knowledge gaps, building robust models and decision-making frameworks, and continuously monitoring and updating assumptions as new information becomes available (van Asselt & Rotmans, 2002).

Aleatoric uncertainty

Aleatory uncertainty refers to inherent randomness or variability in a particular system, process, or event that cannot be reduced or eliminated through additional information or analysis. This type of uncertainty is often associated with natural phenomena, such as weather patterns, seismic activity, or the behavior of complex systems. A simple dice is also a perfect example of aleatory uncertainty, in which the internal randomness of the outcome cannot be altered, even though the odds are known (Kiureghian & Ditlevsen, 2009).

Effective management of aleatory uncertainty involves understanding and quantifying the range of potential outcomes, building robust systems that can withstand a range of potential scenarios, and implementing appropriate risk management strategies to minimize the impact of adverse events (van Asselt & Rotmans, 2002).

[edit] Uncertainty based on severity (4 levels) – How critical is the uncertainty?

Authors such as Meyer et al. (2002) propose a different framework for assessing levels of uncertainty. They propose four levels based on the severity of the situation (Meyer et al., 2002). The four levels and how to manage them are as follows:

Variation Activities such as worker sickness, that push the critical path of a project (Wiki: CPM) and similar occurrences can be seen as variation in a project (Meyer et al., 2002). Events on this level can further be compared to what Millet (2002) describes as uncertainty within a near-term future (Millett, 2002).

To manage variation, including buffers in a Critical Path strategy to catch up on delays, can be recommended. Foreseen uncertainty Meyer’s take on foreseen uncertainty is very similar to what have previously been described as risk management in this article. Foreseen uncertainties are identifiable influences that may or may not occur. To manage foreseen uncertainty, the tools in Project Risk Management can be applied as well as doing contingency plans for the project (PMI, 2017).

Unforeseen uncertainty This level of uncertainty can also be described as "unknown unknowns," as famously stated by former US Secretary of Defense, Donald Rumsfeld (Project Management Institute, 2019, p. 26). Unknown uncertainties refer to risks that are not accounted for in any plans or strategies because they are completely unexpected and are near-impossible to plan against.

To manage unforeseen uncertainty, an organisation must be both agile and continuously learning (Meyer et al., 2002). Too rigid a project plan is vulnerable to unforeseen uncertainty and, therefore, agility is key. Robustness of projects and companies can also be achieved through diversifying assets, relying on multiple partnerships, and having diversity in the project portfolio [Standard].

Chaos The final level in Meyer’s ontology is managing chaos. In chaos, the outcome of a project ends up being so far from the initial project scope that it has little to no application for it. This can mean project failure, or a completely unforeseen secondary benefit. An example being the invention and market introduction of Viagra, which was firstly intended as medication for high blood pressure (Meyer et al., 2002).

Managing chaos can seem like a paradox, but to do it, nonetheless, requires a high degree of autonomy for project managers and an iterative approach to the project work. R&D are often prone to work in chaotic environments and rapid prototyping is an effective way to do this. To allow working in chaos, companies must, however, also be ruthless in killing projects when the chances for success become too small.

[edit] Uncertainty based on source (5 levels) – What are we uncertain about?

Uncertainty can be classified based on its source, and there are five levels of uncertainty according to Riesch (2013). The source of uncertainty can also be formulated as the answer to the question: “What are we uncertain about?”.

Level 1: Uncertainty of the outcome The first level is the uncertainty of the outcome, which is often expressed as a measurable probability. This means that we know what we have done, we know the model, and we know the likelihood of a particular outcome occurring (Riesch, 2013). The uncertainty of the outcome is limited to an aleatoric uncertainty in Riesch’s model, as all is known, and the epistemic uncertainties thereby have been removed. An example here is again the rolling of a dice.

Managing the uncertainty of outcome in this sense, should refer to business decisions: Is the risk worth it based on cost, time, quality, etc.?

Level 2: Uncertainty of the parameters The second level is the uncertainty of the parameters, which can result from a lack of information. The idea is, to say “once we retrieve the necessary information, we can predict the outcome with a probability, P”. This level is thus, a sort of epistemic uncertainty that refers to the external factors to the project.

Level 3: Uncertainty about the model itself The third level is uncertainty about the model itself. An example of this can be the trust that the client has in the experts who are delivering the risk assessment. Different risk assessment models exist, but the client's level of trust in the process can determine how much they trust the model. This is uncertainty of the internal factors in the organisation and of the project work. This uncertainty is, thus, unquantifiable and relates to the human and emotional aspects of managing uncertainty. To manage this kind of uncertainty, the human relation and stakeholder management is important. However, uncertainties about “the model itself” can also refer to insecurities about the assessment due to a “gut feeling” within the team. This is to say, a hunch that you do not yet have all the facts, and thereby suspect unforeseen uncertainty in the project. Visualisations or a walk-through of the knowledgebase for the assessment, can in this case be recommended (Riesch, 2012).

Level 4: Uncertainty about acknowledged inadequacies and implicitly made assumptions The fourth level is uncertainty about acknowledged inadequacies and implicitly made assumptions. These assumptions are often based on past experiences or established conventions and can lead to blind spots in our risk assessments. For example, a company might assume that a certain market trend will continue, without considering the possibility of disruptive innovations or changes in consumer behavior (Riesch, 2012). One example of acknowledged inadequacies is the challenge of dealing with uncertainties associated with extreme events, also known as tail risk. These events are rare and have a low probability of occurrence but can have a significant impact on outcomes. Many risk models are not designed to handle tail risk effectively, and as a result, they may underestimate the potential impact of such events (Riesch, 2012).

This level of uncertainty is complex to handle and relates heavily to the human and emotional aspect of uncertainty management. Further, human biases can cause that these uncertainties are overlooked until execution has begone. So far, the levels of uncertainty proposed by Riesch (2013), has referred to an analysis stage, in which a decision is yet to be made. Now, they appear as unpleasant surprises during project execution.

Level 5: Unknown inadequacies The fifth level of uncertainty in Riesch’s (2013) framework also refers to the ‘unknown unknowns’ in Rumsfeld’s terminology. Like Meyer et al. (2002), Riesch (2013) describes this level of uncertainty as the most difficult to manage. Because we cannot predict what we do not know, we must focus on building resilient systems that can withstand a range of potential scenarios. This involves designing systems that are flexible and adaptable, with redundancies and fallback options built-in. It also means fostering a culture of innovation and continuous learning, where new ideas and approaches can be tested and evaluated (Riesch, 2013).

[edit] Discussion on the merits of each ontology

Figure 1. The overlapping of uncertainty ontologies (own production)

The proposed frameworks each contribute valuable insights and address various types of uncertainty. As the ontologies overlap, they can complement and enhance each other's understanding. For instance, the two inherent levels of epistemic vs. aleatory uncertainty are present in levels one, two, and three of Riesch's ontology, and similarly are Meyer and Riesch's frameworks not mutually exclusive. Therefore, uncertainty can refer to external parameters (Riesch Level 2) and be caused by variation within the project (Meyer Level 1). This integrative view of combining frameworks is also evident within the ontologies themselves. Riesch and fellow uncertainty researcher Wynne both argue that the levels can be applied “on top of each other”, rather than in a stepwise approach (Wynne, 1992).

However, determining the right combination of uncertainty levels for a specific challenge within project, program, and portfolio management can still be a complex task. This still requires a skilled practitioner. In figure 1, the different levels are attempted visualised, to show the overlapping effects. The usefulness of the model especially comes from the fact that epistemic uncertainty can be diminished by gathering additional knowledge. So, if, “level 4: uncertainty of acknowledged inadequacies” is present, this might be solved or transformed to a lesser level by obtaining more knowledge as well. If the uncertainty is aleatoric, as with “foreseen uncertainty”, then it is easy to decipher a way forward of using risk management strategies to accommodate the uncertainty. Some levels can lie between these two, for instance can variation both derive from a lack of knowledge and simple bad luck. Other uncertainties are slightly overlapping as well. This is the case for both “level three” and “level four” with “unforeseen uncertainty” as they both contain elements of unforeseen uncertainty within them. “Chaos” is placed far from the other’s as it describes cases in which a project has moved so far from its original scope, that it is unrecognisable.

[edit] Uncertainties for projects, programs, and portfolios

Portfolios

Managing levels of uncertainty in portfolios is essential for organizations to achieve efficient value delivery. PMI's Standard for Risk Management in Portfolios, Programs, and Projects (2019) and The Standard for Portfolio Management (2017) emphasize risk identification, analysis, response, and ongoing monitoring and control. Portfolio management involves two risk levels: strategic and tactical risks. Strategic risks, identified at the portfolio level, affect the organization's ability to meet strategic objectives. Tactical risks are identified through portfolio management processes or escalated from portfolio components, including changing business needs, resource availability, component interactions, and conflicting objectives (PMI, 2019). Portfolio risk management should be proactive, with trigger events launching responses before risks materialize. In dynamic environments, Petit (2012) suggests continuous assessment and uncertainty management, acknowledging data limitations and unclear information. Human resource management should also be integrated into portfolio management, as it requires continuous planning, monitoring, and controlling, matching resource demands with availability, and ensuring competence in both short and long terms (Petit, 2012). By following these best practices, organizations can better navigate uncertainty and make well-informed decisions, even in uncertain environments.

Programs

Managing levels of uncertainty in programs requires a comprehensive approach to risk management that addresses the complexity of programs and optimizes the realization of program benefits. Risks can be identified at three levels: • cascading from the portfolio or enterprise level • directly at the program level • and, escalated from the program components. Risks are further classified into operational and contextual perspectives. Operational risks arise from program activities such as integration, transition, change management, and operational activities, as well as risks escalated from components with impacts beyond their scope. Contextual risks result from the strategic and organizational environment, stakeholder influences, and variations in strategy, business environment, or program's business case. Effective risk management processes must be in place for all program components to manage the entire risk management life cycle, ensuring that risks are identified, assessed, and treated appropriately across all levels (PMI, 2019).

Projects

According to PMI's Standard for Risk Management (2019) project risks occur on two levels: As operational or contextual risks. Operational risks include scope, life cycle, work breakdown structure, estimates, dependencies, procurement plans, change requests, and historical data. Contextual risks involve stakeholder analysis, business case, program/portfolio governance factors, and enterprise environmental factors. By addressing both operational and contextual risks, project teams maintain a strong connection to strategic objectives, enabling proactive management and reporting of key opportunities and threats, ultimately navigating uncertainty, and delivering successful project outcomes.

[edit] Limitations

There are some limitations to this article. • This article showcases some ontologies for assessing levels of uncertainty in project, programs, and portfolios. However, there are many more out there. The ontologies included differ by what uncertainty is measured, and do therefore present different approaches. Other ontologies definitely has merit as well, but is more or less different indentations between risk, uncertainty, and ignorance, as presented early on by (Oehmen & Kwakkel, 2020). • This approach of assessing ontologies of uncertainty levels based on their source is something, I have not found in literature. The model presented is therefore, also, of own production, and should be viewed with some (healthy) scepticism.

[edit] Bibliography

1. Chapman, C., & Ward, S. (2012). Key motives for uncertainty management. In How to Manage Project Opportunity and Risk (pp. 73–100). https://doi.org/https://doi.org/10.1002/9781119208587.ch3
2. Geraldi, J., Thuesen, C., Oehmen, J., & Stingl, V. (2017). Doing Projects. A Nordic Flavour to Managing Projects: DS-handbook 185:2017. Dansk Standard.
3. Kerzner, H., & Saladis, F. P. (2009). Bringing the PMBOK guide to life : a companion for the practicing project manager. In Bringing the Pmbok® Guide To Life: a Companion for the Practicing Project Manager. Wiley.
4. Kiureghian, A. Der, & Ditlevsen, O. (2009). Aleatory or epistemic? Does it matter? Structural Safety, 31(2), 105–112. https://doi.org/https://doi.org/10.1016/j.strusafe.2008.06.020
5. Meyer, A. de, Loch, C. H., & Meyer, D. (2002). Managing project uncertainty: From variation to chaos Managing project uncertainty. MIT Sloan Management Review, 43(2), 60–67. https://ink.library.smu.edu.sg/lkcsb_research
6. Millett, S. M. (2002). Four levels of uncertainty. Strategy and Leadership, 30(2). https://doi.org/10.1108/sl.2002.26130bae.001
7. Oehmen, J., & Kwakkel, J. (2020). Risk, Uncertainty, and Ignorance in Engineering Systems Design. In A. Maier, J. Oehmen, & P. E. Vermaas (Eds.), Handbook of Engineering Systems Design (pp. 1–31). Springer International Publishing. https://doi.org/10.1007/978-3-030-46054-9_10-1
8. Petit, Y. (2012). Project portfolios in dynamic environments: Organizing for uncertainty. International Journal of Project Management, 30(5), 539–553. DOI:10.1016/j.ijproman.2011.11.007
9. Project Management Institute. (2017). The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. In Standard for Portfolio Management — Fourth Edition. Project Management Institute.
10. Project Management Institute. (2019). The Standard for Risk Management in Portfolios, Programs, and Projects. In Standard for Risk Management in Portfolios, Programs, and Projects. Project Management Institute.
11. Riesch, H. (2012). Levels of uncertainty. In Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk (pp. 87–110). Springer Netherlands. https://doi.org/10.1007/978-94-007-1433-5_4
12. Riesch, H. (2013). Levels of Uncertainty. In Springerbriefs in Philosophy (pp. 29–56). Springer Science and Business Media B.V. https://doi.org/10.1007/978-94-007-5455-3_2
13. van Asselt, M. B. A., & Rotmans, J. (2002). Uncertainty in Integrated Assessment Modelling. Climatic Change, 54(1), 75–105. https://doi.org/10.1023/A:1015783803445
14. Wynne, B. (1992). Uncertainty and environmental learning: Reconceiving science and policy in the preventive paradigm. Global Environmental Change, 2(2), 111–127. https://doi.org/https://doi.org/10.1016/0959-3780(92)90017-2