Epistemic vs. Aleatory uncertainty

Revision as of 15:02, 22 February 2019

Panagiotis Vounatsos - s182563

Abstract

Uncertainty is embedded in many aspects of a project, program and portfolio management. It is present in decision making for project integration and complexity, scope management, schedule management, cost management and risk management as this is mentioned in PMI standards as well as in risk management given in AXELOS project management standards.

Uncertainty derives from not knowing for sure if a statement is true or false. More specifically, it is the absence of information and if put more scientifically, it is the difference between the amount of information required to perform a task and the amount of information already possessed^[1]. Uncertainty is considered crucial to be identified and mitigated as it can contribute to severe consequences to the aforementioned aspects of a project, program or portfolio. Depending on the level of the uncertainty and the consequence it may result in jeopardizing the outcome of an action or even of the whole project. It is worth mentioning that uncertainty is not only a part of the project management but also a part of the technical implementation of a project.

The capability to quantify the impact of uncertainty in the decision context is critical. Uncertainty can be divided in several categories but the most dominant ones in uncertainty theory are epistemic and aleatory uncertainty^[2]. Epistemic uncertainty derives from the lack of knowledge of a parameter, phenomenon or process, while aleatory uncertainty refers to uncertainty caused by probabilistic variations in a random event^[3]. Each of these two different types of uncertainty has its own unique set of characteristics that separates it from the other and can be quantified through different methods. Some of these methods include simulation, statistical analysis or measurements^[4]. There is still ongoing research for increasing the accuracy of a result and include more parameters in calculating an outcome.

What is Uncertainty

Different definitions have been given for uncertainty in project management, but their common denominator is “not knowing for sure”. There is information that are known to be true and other known to be false, but for a large portion of information there is not knowledge whether they are true or false, and therefore they are mentioned as uncertain^[1]. According to Lindley^[5] uncertainty can be considered as subjective between individuals and this is attributed to the fact that the set of information obtained from an individual can be different from another. Two facts that apply are: a) the degree of uncertainty between individuals may also differ, meaning that one person may think that an event is more likely to happen that another person, b) The number of uncertain information is vastly greater than the number of information each individual is sure that are true or false^[5]. The two aforementioned facts deeply affect decision making by taking into consideration that uncertainty creates the contingency for occurrence of risky events may lead to potential damage or loss.

Epistemic vs. Aleatory uncertainty

Uncertainty is categorized into two types: Epistemic uncertainty (also known as systematic uncertainty or reducible uncertainty) and aleatory uncertainty (also known as statistical uncertainty or irreducible uncertainty)^[6].

• Epistemic Uncertainty: derives its name from the Greek word “επιστήμη” (episteme) which can be roughly translated as knowledge. Therefore, epistemic uncertainty is presumed to derive from the lack of knowledge of information regarding the phenomena that dictate how a system should behave, ultimately affecting the outcome of an event^[2][6].
• Aleatory Uncertainty: derives its name from the Latin word “alea” which is translated as “the roll of the dice”. Therefore, aleatory uncertainty can be defined as the internal randomness of a phenomena^[2].

Key features of Epistemic and Aleatory Uncertainty

Key features characterizing pure epistemic and pure aleatory uncertainty are distinguished according to judgement and decision making.

Representation

Epistemic uncertainty targets single cases (or statements), while aleatory uncertainty focuses on a range of possible outcomes that can derive from the repetition of an experiment or situation. Robinson et al. (2006) as cited in Fox and Ülkümen (2011)^[7], have carried out an experiment asking children to predict the color of a toy building block (orange or green) that would be drawn from a bag containing only these two colors. The result presented that when the children were asked before the experimenter has drawn a block, then they chose both colors as a possible outcome. If the children were asked to predict after the experimented has drawn a block, then they usually made one choice based on their best guess on the already determined colour. This experiment suggests that when the likelihood of a single event or a group of events are calculated, then this may prime epistemic and aleatory representation, respectively.

Focus of Prediction

When purely epistemic uncertainty is assessed, it generally leads to the evaluation of events that will be true or false. In contrast, when purely aleatory uncertainty is assessed, it leads to the evaluation of trend of each event on continuous unit interval. According to that, small changes in evidence strength have a big effect on pure epistemic events leading them towards extreme values (yes or no, true or false), compared to judgement of events that include aleatory uncertainty^[7]. An example is that if there is high confidence that a project idea is slightly costlier than another, then the probability that the first project is more expensive than a second one is judged as 1. However, if there is confidence that a project is a marginally more innovative than another, then it can be supposed that the probability it will create more value (money) than the other is less than 1, probably 0.6 or 0.7.

Probability interpretation

The interpretation of pure aleatory uncertainty is carried out as an extentional measure of relative frequency, while the interpretation of pure epistemic uncertainty is conducted as an intentional measure of confidence. In this manner, using relative frequency may trigger more aleatory thinking than drawing out probability numbers. Several studies indicate that the error contained in judging that the occurrence of a combined probable and improbable event is more likely to happen than an improbable event alone, occurs less often when judging relative frequencies than single event probabilities^[7].

Attribution of uncertainty

Unpredictable outcomes that are treated as stochastic (e.g the result from the roll of a dice) relate to aleatory uncertainty. Events or outcomes that occur due to missing information or expertise (e.g. giving the correct answer to an exam), or inefficiency of an aleatory uncertainty model (e.g. the assumptions made for forecasting energy demand are valid) is associated with epistemic uncertainty^[7].Ellsebergs’ paradox provides a very good illustration on decision making under epistemic and aleatory uncertainty. Consider two urns filled with green and yellow balls, and a bet in which someone must choose an urn and guess the color of the ball that will be picked out of it. The first urn contains 10 green balls and 10 yellow balls. Choosing from this urn has a known probability and therefore pure aleatory uncertainty. The second urn contains 20 balls in total but without the known amount of green and yellow balls. Choosing from this urn presents mixed epistemic and aleatory uncertainty. The aleatory uncertainty derives from the randomness of the draw while the epistemic uncertainty derives from the unknown composition. Let’s presume that the decision maker must choose from the first urn with known probability, after the experimenter draws a ball but before the color is revealed. The event has turned from pure aleatory uncertainty, to epistemic uncertainty of lack of knowledge.

Information search

From the attribution feature, epistemic uncertainty is attributed to missing information or expertise. Therefore, it can be reduced by searching for knowledge that will allow to predict its outcome with greater accuracy. On the contrary, the determined relative frequency of possible outcomes for aleatory uncertainty cannot be further reduced^[7].Let’s assume an example where a program manager has to choose between a set of projects, each project sometimes contributes to the programs benefits and outputs, and sometimes does not. An epistemic mindset suggests that one would alter the choices exploring the combination of projects (type, size, domain, complexity etc.) that govern the sequence for success contributing to the benefits. An aleatory mindset would find which of the project size and complexity is more often successful and run only projects of this size and complexity every time.

Linguistic Markers

Hutchins as cited in Fox and Ülkümen (2011)^[7] has identified that natural languages reflect the intuitive distinguish of cognitive concepts from individuals. The fact that epistemic and aleatory uncertainty had been incorporated in natural language was anticipated and empirically validated (Teigen and Fox, Üklümen and Malle as cited in Fox and Ülkümen (2011)^[7]). These authors presented that there are phrases that express epistemic uncertainty (e.g. “I am 70% sure that…”) and aleatory uncertainty (e.g. “I think there is a 75% change that…”).

The following table (Table 1) summarizes the key features of pure aleatory and epistemic uncertainty.

Whereas epistemic uncertainty can be reduced by acquiring knowledge and information on the system, aleatory uncertainty cannot be reduced in this way, and for this reason is often called as irreducible uncertainty.

Causes of epistemic and aleatory uncertainty

Different causes of uncertainty can be recognized as given by Armacosta and Pet-Edwards, and Zimmermann cited in Zio and Pedroni ^[6]. All three types of management (Project, Program and Portfolio), may seem to have different expressions of epistemic uncertainty but all of decision making can be made by following the same methodologies. Depending on the level of knowledge of the decision maker for the system, the uncertainties can be aleatory, epistemic or a combination of these two in all of the aforementioned cases.

1. Lack of information (or knowledge). The main cause of uncertainty is the lack of information or knowledge regarding the systems or events under investigation. Lack of information can either be categorized as a lack of a precise probabilistic value for an event (quantitative nature), or as lack of knowledge as how to analyze mathematically the known probabilistic values for an event (qualitative nature). Lack of knowledge also affects the detail of the mathematical method used to analyze the probabilistic values of an event. This situation is called approximation, and it occurs when there is not enough information or reason to describe the event in a high level of detail, and therefore lower detail level is used. An example for approximation is when a project manager searches and takes into consideration several parameters to calculate the project’s cost. But to what extent is it needed to calculate the value of some low-cost equipment down to two decimals, if the total cost exceeds several million cost units?
2. Abundance of information (or knowledge). In principle, humans are incapable of simultaneously assimilating and elaborating many pieces of information (or data), and that leads to uncertainty from abundance of information. Due to the explained human nature, when there is overwhelming information, attention is only given to pieces of information considered as the most important, while others are neglected. For example, this uncertainty occurs when there are different models for the analyst to choose among, in order to analyze an event.
3. Conflicting nature of pieces of information/data. This uncertainty occurs when some pieces of information give contradicting knowledge, and it cannot be reduced by increasing the amount of information. This conflict can derive from the facts a) that information is affected by unidentified from the analyst errors, or b) that information is irrelevant to the event analyzed, or c) the model used to analyze the system is incorrect.
4. Measurement errors. Uncertainty is created by errors in the measurement of a physical quantity and occur either due to an error of the measurement taker, or due to insufficient accuracy of the used instrument.
5. Linguistic ambiguity. All languages and communication forms can be structured in a way that can be differently interpreted depending on the analysis context. This cause of uncertainty is included in the “lack of information” category, because it can be reduced by clarifying the context.
6. Subjectivity of analyst opinions. This uncertainty emanates from the subjective interpretation of information by the analyst, depending on their personal experience, competence and cultural background. The uncertainty deriving from this cause, can be reduced by taking into consideration the opinion of several different experts.

Uncertainty in Management

In this subsection, details are going to be given regarding where is uncertainty met in program, project and portfolio management.

Project Management

Project management deals with uncertainty in several different levels. At one level, uncertainty within an organization or its environment leads to the increase of the complexity of the project. Moreover, uncertainty is part of the projects complexity and is expressed as ambiguity which is uncertainty of emerging issues and lack of understanding ^[8]. Furthermore, high uncertainty in projects may lead to not understanding the scope at the beginning of the project or leading the scope to evolve during the project^[8].Moreover, high uncertainty in the current competitive marketplace leads to the necessity to effectively adopt and tailor development practices for better fit in the changing environment, and this includes introducing more effective project scheduling methods^[8].Additionally, project cost management is affected by high degrees of uncertainty because high uncertainty leads to frequent changes and that fact does not permit detailed cost calculations. It rather calls for lightweight estimation methods providing an easily adjustable high level forecast^[8]. In order to reduce any impact of the risk associated with the uncertainty, Project Risk Management addresses risk (an effect of uncertainty) in individual objectives and in the overall of the project^[8].

Program Management

Uncertainty is an inevitable challenge of program management, and in the beginning of the program where the outcomes are not yet clear it is considered to be very high. Two factors that contribute greatly to the high unpredictability and uncertainty regarding the outputs, benefits and outcomes of the program’s work are the internal organizational environment and the changes in the external environment. Within the organization’s environment, programs have higher uncertainty compared to individual projects. While programs can tackle some uncertainty regarding their goals, budget and timeline by changing the direction and implementation of projects, this practice is the source of more uncertainty regarding the programs’ final direction and outcomes. The management style used for carrying out a program needs to be chosen so that it can identify and tackle the uncertainty created by the continuously progressing and altering scope and content of the program. Another factor that adds uncertainty to program management is the fact that some of the individual components of the program may not actually create some added value to the program’s outcomes and benefits, even though their completion was successful meeting all criteria and providing the planned outputs, products or services^[9].

Portfolio Management

In portfolio management, uncertainty is a factor that is incorporated in risk taken in order to maximize the portfolio’s value. Balancing the risks of different actions is challenging due to the complex nature of portfolios and the inherent characteristic of uncertainty associated with risk. A source of uncertainty is imperfect or incomplete information, and the higher the uncertainty the more important risk attitude perception becomes. Uncertainty can also derive from not minimizing threats but rather embracing them in anticipation of high rewards. An example is investing in a new promising, yet unproven technology in order to be “the first in the market”, expecting highly profitable sales. In the previous case, uncertainty lies in the decision to trust a technology while at the same time accepting the possibility of the technology failing. Furthermore, uncertainty can originate from within the organizations internal environment based on different decision making actions such as whether a management practice chosen is the appropriate one or whether depending on highly specialized external assistants should be implemented. Compared to projects and programs, uncertainty is higher at the portfolio level, because of the impact of uncontrolled variables on the portfolio. For that reason, when uncertainty is growing, solutions are based on the perception to fill in imperfect or incomplete information ^[10].

All three types of management (Project, Program and Portfolio), may seem to have different expressions of epistemic uncertainty but all of decision making can be made by following the same methodologies. Depending on the level of knowledge of the decision maker for the system, the uncertainties can be aleatory, epistemic or a combination of these two in all of the aforementioned cases.

Epistemic uncertainty, value uncertainty and decision making

Decision making under epistemic uncertainty is categorized in four typical types^[11], depending on the level of epistemic uncertainty and the uncertainty of the gained value.

Figure 1: Types of decision under epistemic uncertainty [Inspired by Sahlin,2012 ^[11]]

• Type 1 decision: I am sure what I want, and I know the odds. Type 1 decision considers that the decision maker has a lot of information regarding the precision of probability estimates, as well as clear personal preferences and values. Example: Consider a situation in which a company runs a PC refurbishing line and in a PC there have been added two RAM memory components and two more are left with the capacity to be installed. A decision needs to be taken regarding what to do with them. They can either be installed in the existing PC, tested in another system to check if they are faulty and could create short-circuit, or throw them away. If the knowledge of the supplied quality is good, then the probability that the two remaining RAM memories are also good is known. The epistemic uncertainty is low to non-existent and a PC with more RAM is far better than one with half of RAM or a burnt PC.
• Type 2 decision: I am sure what I want, but I do not know the odds. Type 2 decision takes into consideration that the decision maker has clear values and preferences, but the available information is poor in terms of quality and quantity and is insufficient to represent uncertainty as a probability. Example: Consider a situation where an organization wants to invest in a new and innovative technology which can create a lot of profit by being the first in the market. However, the technology is unproven and there is uncertainty in both the company and the organizational environment. In this case the epistemic state is unknown as well as the magnitude and nature of the risks.
• Type 3 decision: I am not sure what I want, but I know the odds. Type 3 decision regards the decision maker in a place where his preferences and values are not well defined. Furthermore, the information quantity and quality are good enough in order for precise probabilities to be assessed. Example: Consider a situation in which a subject is having a good idea and is considering to invest all his money to a startup company in order to pursue that goal, that will in turn bring in some good profit. After some research, the subject identifies that startup companies have a high failure rate, and if this happens they end up bankrupt, in dept and with a lot of bureaucracy problems. The probability for such events are known to a reasonable precision through the existing corporation databases. This is a situation where our preference can be blurred and unclear from the lack of experience.
• Type 4 decision: I am not sure what I want, and I do not know the odds. In Type 4 decision neither the preferences/values nor the available information can be considered fixed or reliable. Example: Consider a situation where a company wants to start a project for creating a social media platform. Let’s assume that after several expensive dissemination and marketing campaigns, the platform gets a really low market share, far lower than the minimum expectations. The estimation of the probabilities that the dissemination and marketing campaigns are going to be successful after some attempts, is becoming increasingly harder to make. That means that our epistemic state is not clear and it deteriorates. This fact affects the preference (keep the social media platform running or not) and makes it increasingly unstable.

Despite the categorization of epistemic uncertainty into four types (Figure 1), the complete picture is more complex.

Quantification of epistemic uncertainty

Aleatory uncertainty quantification methods

Aleatory uncertainty is referred as inherent probabilistic uncertainty. Therefore, for quantifying aleatory uncertainty systematic probability and mature statistical tools are used. The most common representations of aleatory uncertainty are^[12]:

1. Probability density function (given in figure 2) which is used to quantity the probability density at any point within the interval of a random variable
2. Cumulative distribution function (given in figure 3), which gives the probability that a variable will equal or less that a given value.

Epistemic uncertainty quantification methods

In the literature, this uncertainty has been modeled in a number of different ways, the most popular of which are

• Bayesian probability is a method that appoints a frequency or probability of an event based on an educated guess or a personal belief.
• Evidence theory, also known and as Dempster-Shafer theory or theory of belief functions. It is a generalized version of Bayesian theory of subjective probability. The method bases the belief that an event is true on the probabilities of other hypothesis or events, which are related to the event under investigation^[13].
• Interval analysis or probability boxes, assumes that the input variable and is described by a range with lower and upper bound. This variable is subjected to interval mathematics that can result on providing all the possible values of the probability for the identified boundaries ^[13].
• Probability theory allows for better results than evidence theory because it represents uncertainty that includes the specification of more structure. The method can result in providing only the probability as a measure of chance ^[14].
• Possibility theory is a representation of uncertainty that allows a better structure compared to interval analysis. It provides a measure of the likelihood or a subjective confidence for an event, for every state of the given event ^[14]
• Fuzzy set theory which can be extended to probability theory where membership functions are interpreted in possibility distributions ^[15].
• Generalized information theory, which dictates that if an uncertainty is adequately quantified, then the information (data or knowledge) that is gained from an action that leads to the reduction of uncertainty can be measured ^[16].

By modelling and quantifying epistemic uncertainty, it is gradually converted to aleatory uncertainty. And gradually refers to the remaining epistemic uncertainty deriving from the lack of knowledge to create a very detailed model.

Limitations

Quantification of uncertainty does not come without limitations. There are some cases where uncertainty cannot be fully analyzed. The following figure is an example of the representation of the unknown parameters of an event (epistemic uncertainty), that after applying quantification methods are reduced to aleatory uncertainty. However, as mentioned in the sources of uncertainty there may be some parameters that we still lack the knowledge of how to transform them into aleatory. Therefore, our initial problem will still keep some epistemic uncertainty, and that fact can be considered as a limitation to fully identifying the underlying uncertainty. An example of the representation of this limitation is given in Figure 4.

Figure 4: Example of uncertainty quantification limitations [Figure created by the author]

Inability to quantify uncertainty due to:

• Lack of knowledge (we don’t know that something could affect our project)
• Lack of data (we don’t know how something could affect our project)
• Lack of enough data (we don’t know with certain degree of confidence how something could affect our project)

Limitations on quantification:

• Density function is calculated based on past data or expert opinion
  • past data may not describe accurately the future (unexpected event)
  • expert opinion: biased due to education, knowledge, experience of decision-maker
• Limitations on approaching method -> the chosen method may not be the optimal to describe a certain event/situation
• Inevitability of assumptions/simplifications -> to what extend it makes sense to the modeler
• Measurement errors
• Limited time
• Cost – resources limitation
• Uncertainty propagation

Annotated Bibliography

1. C. R. Fox and G. Ülkümen, "Chapter 1: Distinguishing Two Dimensions of Uncertainty," in Perspectives on Thinking, Judging, and Decision Making, Oslo, Universitetsforlaget, 2011, pp. 22-28. This chapter explains in depth the theory for distinguishing between epistemic and aleatory uncertainty, giving research finding that support the distinctions.
2. D. V. Lindley, "Uncertainty," in Understanding Uncertainty, New Jersey, John Wiley & Sons, Inc., 2006. The whole book provides a good general introduction regarding the role of uncertainty in everyday life. It provides simple examples for everyday situations explaining the logic of uncertainty behind them. The book does not get involved with complex mathematical concepts, however it provides the reader with good understanding of aspects of uncertainty.
3. Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk, New York, Springer, 2012. Uncertainty is closely related to risk management. This handbook provides an overview for several aspects or risk theory, addressing several ones relative to the decisions that a projects/program/portfolio manager may be called to make (e.g. decision theory, risk perception, ethics and social implications). It aims to promote communication and information among all those who are interested in theoretical issues concerning risk and uncertainty.
4. T. Aven and E. Zio, "Some considerations on the treatment of uncertainties in risk assessment for practical decision making," Reliability Engineering & System Safety, vol. 96, no. 1, pp. 64-74, 2011. This research paper presents how to treat and quantify uncertainty in risk assessment, when it comes to using risk assessment for supporting decision making.
5. K. Rui, Z. Qingyuan, Z. Zhiguo, E. Zio and L. Xiaoyang, "Measuring reliability under epistemic uncertainty: Review on non-probabilistic reliability metrics," Chinese Journal of Aeronautics, vol. 29, no. 3, pp. 571-579, 2016. In this journal paper, a review of non-probabilistic reliability metrics is conducted to assist the selection of appropriate reliability metrics to model the influence of epistemic uncertainty.

References

1. ↑ ^{1.0 1.1} G. Grote, Management of Uncertainty - Theory and application in the design of systems and organizations, London: Springer, 2009.
2. ↑ ^{2.0 2.1 2.2} A. D. Kiureghiana and O. Ditlevsen, "Aleatory or epistemic? Does it matter?," Structural Safety, vol. 31, no. 2, p. 105–112, March 2009.
3. ↑ S. Basu, "Chapter 2: Evaluation of Hazard and Risk Analysis," in Plant Hazard Analysis and Safety Instrumentation Systems, London, Elsevier, 2017, p. 152.
4. ↑ T. Aven and E. Zio, "Some considerations on the treatment of uncertainties in risk assessment for practical decision making," Reliability Engineering & System Safety, vol. 96, no. 1, pp. 64-74, 2011.
5. ↑ ^{5.0 5.1} D. V. Lindley, "Uncertainty," in Understanding Uncertainty, New Jersey, John Wiley & Sons, Inc., 2006, pp. 1-2.
6. ↑ ^{6.0 6.1 6.2} E. Zio and N. Pedroni, "Causes of uncertainty," in Uncertainty characterization in risk analysis for decision-making practice, number 2012-07 of the Cahiers de la Sécurité Industrielle, Toulouse, France, Foundation for an Industrial Safety Culture, 2012, pp. 8-9.
7. ↑ ^{7.0 7.1 7.2 7.3 7.4 7.5 7.6} C. R. Fox and G. Ülkümen, "Chapter 1: Distinguishing Two Dimensions of Uncertainty," in Perspectives on Thinking, Judging, and Decision Making, Oslo, Universitetsforlaget, 2011, pp. 22-28.
8. ↑ ^{8.0 8.1 8.2 8.3 8.4} Project Management Institute, Project Management: A guide to the project management body of knowledge (PMBOK guide) – Sixth Edition, Project Management Institute, 2017, pp. 68, 133, 177, 234, 397.
9. ↑ Project Management Institute, "Program and Project distinctions," in The standard for Program Management - Fourth Edition, Project Management Institute, 2017, pp. 28-29.
10. ↑ Project Management Institute, "Portfolio Risk Management," in Portfolio Management: The standard for portfolio management, 4th Edition, Project Management Institute, 2018, pp. 86-92.
11. ↑ ^{11.0 11.1} N.-E. Sahlin, "Unreliable Probabilities, Paradoxes, and Epistemic Risks," in Handbook of Risk Theory, New York, Springer, 2012, pp. 492-493.
12. ↑ C. J. Roy and W. L. Oberkampf, "A Complete Framework for Verification, Validation, and Uncertainty Quantification in Scientific Computing (Invited)," in 48th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, Orlando, Florida, 2010.
13. ↑ ^{13.0 13.1} K. Rui, Z. Qingyuan, Z. Zhiguo, E. Zio and L. Xiaoyang, "Measuring reliability under epistemic uncertainty: Review on non-probabilistic reliability metrics," Chinese Journal of Aeronautics, vol. 29, no. 3, pp. 571-579, 2016.
14. ↑ ^{14.0 14.1} J. C. Helton and J. D. Johnson, "Quantification of margins and uncertainties: Alternative representations of epistemic uncertainty," Reliability Engineering and System Safety, vol. 96, no. 9, pp. 1034-1052, 2011.
15. ↑ L. A. Zadeh, "Fuzzy sets as a basis for a theory of possibility," Fuzzy Sets and Systems , vol. 1, pp. 3-28, 1978.
16. ↑ G. J. Klir, "Generalized information theory," Fuzzy Sets and Systems, vol. 40, pp. 127-142, 1991.