Revision as of 10:02, 2 May 2023

Abstract

Robust Decision Making (RDM) is a computational framework integrating Decision Analysis, Assumption-Based Planning, Scenario Analysis, and Exploratory Modelling. This article critically reviews RDM, its principles, and applications in project management. The article suggests that RDM enables project managers to effectively address uncertainty, offering a powerful analytical framework.

Conceptualising Robust Decision Making at times of Uncertainty

Origins and Functions

Robust Decision Making (RDM) emerged in the 1980s, when analysts of the RAND Corporation, a California-based think tank affiliated with the U.S. Government, developed a framework to evaluate the effectiveness of nuclear weapon systems ^{[1] [2]}. Designed to mitigate the uncertainty and ambiguity experienced by U.S. Government officials involved in the planning and implementation of nuclear deterrence strategies, RDM included simulation techniques, sensitivity analysis, and real options analysis. In the 1990s and 2000s, RDM received increasing interest from private companies interested in exploring new project management techniques applicable to a wide range of industries, including construction, software development, and environmental management. Today, RDM is an established approach in project management, recognized for its ability to help project managers making well-informed and timely decisions under pressure and at times of uncertainty. According to former United States Secretary of Defence Donald Rumsfeld, there are different types of knowledge: known knowns, known unknowns, and unknown unknowns. Known knowns refer to things that we know for sure. Known unknowns refer to things that we know we do not know. However, the most challenging category is the unknown unknowns, which refers to things that we do not know we do not know ^{[3] [4]}. The decision-making process in situations affected by a great level of uncertainty is defined as decision making under deep uncertainty (DMDU) ^[2].

Robustness is a crucial aspect of effective DMDU ^{[5] [6]}. Conventional decision-analytic techniques for risk and decision analysis are designed to identify optimal strategies based on a characterization of uncertainty that follows the axioms of probability theory ^[7]. However, in scenarios where there is uncertainty about the system model or the distributions of its inputs, traditional decision-analytic approaches often utilize sensitivity analyses to assess the dependence of the optimum strategy on the specification of model and distributions ^[8]. While this approach may be suitable when the optimum strategy is relatively insensitive to these key assumptions, it can pose both conceptual and practical challenges when this is not the case. RDM is part of a new breed of computational, multi-scenario simulation approaches that integrates ideas from scenario-based planning into a quantitative framework ^{[9] [10] [6] [11]}. It inverts traditional sensitivity analysis by seeking optimization strategies which good performance is insensitive to the most significant uncertainties. Beginning with one or more system models that link optimization strategies to outcomes and a collection of several plausible probability distributions over the uncertain input parameters to these models, RDM describes uncertainty with various, plausible perspectives of the future ^[12]. RDM suggests robust strategies, identifies vulnerabilities, and suggests new or modified strategies.

Foundations of Robust Decision Making

RDM combines four crucial concepts - Decision Analysis, Assumption-Based Planning, scenarios, and Exploratory Modelling - to provide evidence-based analysis that informs choices. This approach empowers decision-makers to navigate uncertainty, assess vulnerabilities, and identify robust strategies that can withstand potential futures.

Decision Analysis (DA)

Decision Analysis is a discipline that involves the theory, methodology, and practice of designing and using decision aids to help people make better decisions by being explicit about their goals, using the best available evidence to understand potential consequences, considering trade-offs among alternatives, and following agreed-upon rules and norms to enhance the legitimacy of the decision-making process ^[2]. RDM exploits this approach by focusing specifically on finding trade-offs and describing vulnerabilities to create robust decisions based on stress testing of probable future routes. Both DA and RDM seek to improve the decision-making process by being clear about goals, utilizing the finest information available, carefully weighing trade-offs, and adhering to established standards and conventions to assure legitimacy for all parties involved. However, while DA seeks optimality of decisions ^[13], RDM seeks robustness assuming uncertainty as unescapable and probabilities as imprecise ^[14].

Assumption-Based Planning (ABP)

By expanding awareness of how and why things could fail, RDM encapsulates the ideas of stress testing and red teaming to lessen the harmful impacts of overconfidence in current plans and processes. The former idea puts a system through rigorous testing to establish its breaking points, whereas the latter forms an independent group of people (often external parties) to find ways to thwart an organization's plans ^[15]. This approach was first implemented in the so-called Assumption-Based Planning (ABP) framework. Starting with a written version of an organization's plans, ABP finds the explicit and implicit assumptions made during the formulation of that plan that, if inaccurate, would result in failure. These assumptions are identified by decision-makers, who then create backup plans and "hedging" strategies to be used in case of necessity ^[16].

Scenario Analysis (SA)

To deal with deep uncertainty, RDM builds upon the idea of SA ^[17]. Scenarios are defined as collections of potential future occurrences that illustrate various worldviews without explicitly assigning a relative likelihood score ^[18]. They are frequently envisioned in deliberative processes involving stakeholders and probabilities are often not considered. This is because, without considering probabilities, stakeholders are more prone to broaden the range of scenarios taken into consideration and consider their decisions from a larger variety of angles and perspectives, leading to including unexpected outcomes as well. In incorporating analytical techniques borrowed from SA, RDM declines the knowledge about the future into a selected range of potential situations—a technique that assists decision-makers in envisioning future risks and better navigating situation characterised by deep uncertainty.

Exploratory Modeling (EM)

In his study on predictive models and computer simulations, Bankes argued that Exploratory Modelling (EM) is one of the most appropriate tools that allows the integration of DA, ABP, and SA in RDM ^[19]. Without prioritizing one set of assumptions over another, EM factors a wide range of assumptions into a limited number of results. In other words, EM is strongly beneficial when a single model cannot be validated because of a lack of evidence, insufficient or conflicting ideas, or unknown futures. Therefore, by lowering the demands for analytic tractability on the models employed in the study, EM offers a quantitative framework for stress testing and scenario analysis and allows the exploration of futures and strategies. As EM favours no baseline scenario as an anchor point, it allows for genuinely global, large-N studies in support of the more qualitative methods ingrained in SA approaches.

Application

Theoretical framework

RDM is a learning process based on a process that the scholarship defines as “deliberation with analysis” ^[20]. The framework requires that the decision-making parties discuss their goals and alternatives, which are based on assessments provided by analysts who imagine scenarios and policy options. This is especially recommended in situations in which a variety of decision-makers operate in rapidly changing circumstances, and whose objectives may change because of their collaboration with others. ^[21] As illustrated in Figure 1 RDM methodology follows 5 major steps, described in the paragraphs below.

Figure 1: XLRM framework

Step1: Decision Framing.

The RDM process starts with a decision framing workshop in which stakeholders brainstorm and define the key factors in the analysis. These include decision-makers’ goals and criteria, the potential courses of action they may choose to accomplish those goals, the uncertainties that might impact the link between actions and results, and the connections between actions, uncertainties, and goals. Once gathered, this information is put into a framework known as “XLRM” ^{[17] [22]}, where:

• “X” stands for exogenous variables (factors not under the control of the decision makers)
• “L” stands for policy levers (policies that affect the system to achieve goals)
• “R” stands for relationships (relevant variables needed to correctly evaluate and benchmark policies)
• “M” stands for measures of performance (metrics, not necessarily quantitative, given from stakeholders to evaluate policies)

The output is a set of potential robust strategies.

Step 2: Evaluate strategies.

According to the ABP approach, RDM exploits simulation models to assess the proposed strategies of Step 1 in each of many plausible paths into the future. This process of generating strategies may use a variety of techniques, spanning from optimization methods to public debate ^[23]. It is commonly observed, however, that strategy evaluation usually combines them all ^[24].

Step 3: Vulnerability analysis.

Data analytics and visualization techniques are then used to search for and describe vulnerabilities of the strategies under consideration. Specifically, statistical methods are used to find the critical variables that best distinguish futures in which these strategies succeed or fail. ^[25]. The output of this step is a multitude of scenarios which are then clustered based on the identified vulnerabilities.

Step 4: Trade-off analysis.

In this step, both analysts and decision makers use the scenarios to present and assess the trade-offs between strategies. To indicate the implications of selecting one strategy over another, it is often helpful illustrating these trade-offs. To do so, the effectiveness of one or more strategies can be plotted against the probability of the scenario the strategies belong to. This step is useful to give insights on how the future would look like if a strategy was chosen.

Step 5: New futures and strategies.

The results from Step 4 are necessary to identifying and appraising alternative solutions, allowing to focus on the most robust ones. Sometimes, the identification and appraisal process relies on experts’ opinions ^{[24] [17]}. In other cases, optimization approaches are preferred Cite error: Closing </ref> missing for <ref> tag

^[6]

^[7]

^[8]

^[9]

^[10]

^[11]

^[12]

^[1]

^[2]

^[16]

^[3]

^[4]

^[13]

^[14]

^[15]

^[17]

^[18]

^[19]

^[21]

^[22]

^[23]

^[24]

^[25]

^[26]

^[27]