Levels of uncertainties

Revision as of 22:35, 9 April 2023

Abstract

This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article also discusses expressing uncertainty levels as a state-space, which can be viewed as a proactive way of describing the level of uncertainty. The transition between levels is not clearly defined, and the project manager should be careful about moving between state spaces. The article describes five different ways to express state spaces and probability statements in terms of levels of uncertainty, ranging from full probability density function to qualitative description of the likelihood of those states. The application of levels of uncertainties is essential in several domains, such as project management, decision making, and risk analysis. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.

BIG IDEA:

Uncertainties and Risks

Uncertainty is a multifaceted creature, and there are many definitions (none which there is a broad consensus of). In this article, we choose the definition of "Any departure from the unachievable ideal of complete determinism". Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker (2003), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time, and as such consist of (Bevan, 2022):

1. Nature (epistemic/aleatory). Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility.
2. Levels/type. Indeterminacy/ignorance to determinacy.
3. Division of one’s cognizance.
4. The location/source
5. Difficulties in communication/forming consensus
6. Human values/subjectivity, normativity

• The levels spectrum. Determinism -> Ignorance

The levels of uncertainty, or scales of uncertainty, is a way of probability the probability of uncertainties states within a given system (Bevan 2022). Usually the levels are between indeterminacy and determinacy with intermediate states between the two outer cases (Walker 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements (Bradley and Dreschel (2014), qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink, 2010). The original distinction between levels of uncertainties was proposed by Knight (1921) between risk and uncertainty, but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). In more recent years, Walker (2003) developed one of the more used distinctions of the different levels of uncertainties. This distinction in most used (Bevan 2022), and has gained the addition of qualitative uncertainty as proposed by Warmink (2010). The list below goes from determinacy to total ignorance (the two ends of the spectrum):

1. Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Stastical uncertainty is as close to determinism as uncertainty can get.
2. Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes.
3. Qualitative uncertainty – missing/on its way
4. Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help).
5. Total ignorance. We do not know what we do not know.

Expressing uncertainty levels as a state-spaces

Describing the level of uncertainty can be viewed in a more proactive matter as a state-space, from which the uncertainties may move (however, be aware of the limitations, see section “Limitations”). The levels which a manager may add to project information encode information about possibilities within state-spaces and their probability. Within these state-spaces, we may draw some likelihood description of found uncertainties within projects, and these may guide the project manager to evaluate the predictive capabilities of the information given to them or set about gathering more information. (Bevan, 2022) The transition between levels is not clearly defined, and thus the project manager should be careful about moving between state spaces. That said, depending on the magnitude of the uncertainty, there are some things the manager could attempt to mitigate some of the uncertainty. See section “Application”. Bevan (2022) summarizes 5 different ways to express these state-spaces and probability statements in terms of levels of uncertainties. It is here adapted to fit with the chosen description of levels of uncertainty:

Total ignorance is not part of this state/space description, as there would be no way of describing that state space with total ignorance of said state-space. When managing uncertainties, developing appropriate reactions is key. Hence, getting an understanding of the level of uncertainty is a start point, and while project workers may not have the vocabulary between levels, using the distinction as proposed by Bevan (2022) to describe state-spaces, it should become possible to classify uncertainties in projects, programs, and portfolios, which can be further used as described in the following sections.

APPLICATION

Application of Levels of uncertainties

The application of levels of uncertainties is on several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties.

Selection of appropriate analytical technique (Stirling 1998)

Missing/on its way.

Classification of expressed uncertainty-descriptions

When presented with uncertainties by project managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Tenney, Kværner & Gjerstad (2006) proposes a framework for managers to do so:

With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, in-probable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware.

What can be/is stated in practice (Enserink et al. 2013)

Missing/on its way.

Communicating uncertainties

Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney, Kværner & Gjerstad 2006). When communicating uncertainties and their levels, use the following questions to guide your strategy (Adapted from van der Bles (2019)):

• What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources.
• Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.).
• Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately.
• Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication.
• Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat
• Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations.

Embracing uncertainties when the arise

Missing/on its way

Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material)

Projects

At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to risks highly influenced by uncertainties can aid to focus further exploration to minimize the uncertainty. Knowing where uncertainties arises from early in programs/projects can aid the manager to e.g. change models (if the model implies uncertainty in the given context), bla bla bla. When considering if an uncertainty should be pursued to be reduced, it is very important to consider resource management aspects such as time, staff and money (Warmink 2010)

• Degree of information in projects that could be useful

Programs

At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds”. However, the interdependence between projects.

• Degree of information in projects that could be useful

Portfolios

Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon [PMI Portfolio]. Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio, and make better decisions by knowing where the uncertainty arises from.

• Degree of information in portfolios that could be useful

Limitations

• Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between inderterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are very unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022).
• We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact.

References

• The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)
• Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. & Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.
• PMI Portfolio standard
• PMI Program standard
• PMI Project standard
• Van der Bles AM, van der Linden S, Freeman ALJ, Mitchell J, Galvao AB, Zaval L, Spiegelhalter DJ. 2019 Communicating uncertainty about facts, numbers and science.
• Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., & Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011
• Tennøy, A., Kværner, J., & Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345
• Enserink, B., Kwakkel, J. H., & Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. *Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006
• Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., & Spiegelhalter, D. J. (2019). *Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870
• Stirling, A. (1998). Risk at a turning point? Journal of Risk Research, 1(2), 97–109. https://doi.org/10.1080/136698798377204

Articles that are currently also considered:

• “This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment
• Risk and Uncertainty Communication
• van Asselt, M. & Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)
• Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)
• Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk
• Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)
• Uncertainty concepts for integrated modeling - Review and application for identifying uncertainties and uncertainty propagation pathways (Kirschner, 2021)