RDM

Robust decision-making (RDM), is a relatively new and innovative methodology in the science of decision support. The RDM framework is a key tool in decision-making processes under deep uncertainty and provides decision-makers with a method to make a plan for the future without having to predict it. Decisions are taken everywhere and on all levels, whether it comes to project, program or portfolio management, there will always be decisions to make. As RDM is focused on supporting Decision Making Under Deep Uncertainty (DMDU), concerning decisions with long-term consequences, the decisions are most often taken on the top-level . As RDM supports groups of decision-makers in finding a strategy that makes it possible to succeed despite the uncertainties associated with the future, there is not necessarily a definitive end date. Whether RDM, therefore, focuses more on program or portfolio management than project management depends on the situation, but since RDM is used to develop an adoptive long-term strategy, it must be expected that RDM is more relevant within program or portfolio management compared to project management. RDM has e.g. been used to analyze and support decision processes concerning climate change, infrastructure building, water management and coastal planning which all are very complex and long-term issues involving a high degree of uncertainty. When decision-makers have to make long-term decisions they're often called upon to anticipate future needs, resources and circumstances. The problem with the traditional methods based on predictions is that decisions made on predictions are less reliable the farther the prediction reach forward in time, as time entails the prediction to become more vulnerable to uncertainty . This uncertainty could e.g. be unforeseen economic crashes, terrorism attacks, political instability, climate change, chains of actions or reactions leading to more possible outcomes than a single prediction can handle. RDM uses computer simulations and advanced modelling techniques to stress-test strategies not only against one future but against thousands or millions of possible futures, in order to acquire knowledge from a wide range of diverse futures . The main purpose of RDM is not to make a better prediction but through its concepts and processes to contribute with knowledge for the decision-makers to be able to design more robust strategies, that perform no matter what the future holds.. In this article, the concept of RDM, its purpose, and RDMs foundation will be analysed. Further in-depth guidance on how RDM is implemented and used, as well as an analysis of when RDM is applicable, will be presented. Finally, a critical reflection on RDM will seek to find and enlighten the benefits and limitations of RDM.

The idea of RDM

History of RDM

Decision-making under uncertainty is not a new problem, but the way to approach and handle these often complex issues has changed over time. RDM has become one of the more common approaches to handle DMDU. The first person to describe the RDM methodology was Robert J. Lempert from the RAND cooperation in 1997 in "When we don't know the costs or the benefits: Adaptive strategies for abating climate change ". But R.J. Lempert was not the first one who tried to develop a new method to handle the increasing number of decisions under deep uncertainty. The company Envolving Logic had in the 1990s, parallel with the RAND corporation and R.J. Lempert's development of RDM, developed the first software tool for DMDU. Their software was called the Computer-Assisted Reasoning system (CARsTM) and they described it as a tool to support decision-makers in the area of the Robust Adaptive Planning (RAP) methodology, while the RAND corporation and R.J. Lempert instead adopted the Robust Decision Making (RDM) methodology .

According to R.J Lempert, the traditional “predict-then-act” framework has proven to be successful for decades, which both a large number of theories and mathematical techniques is proof on. RDM was original developed to make policymakers able to make better decisions about issues that could have consequences in the long-term. RDM was mostly focused and used for decisions about climate changes in the beginning of it's lifetime. It was not a coincidence that RDM mainly were used for decisions regarding climate changes, because it was particular climate changes commonly known property of being embraced with deep uncertainty, that made RDM suitable for this area specifically . With the arrival of the RDM, it was no longer necessary to determine the probabilities of the future consequences of climate changes in order to make a strategy, which many also believe is something near impossible. The need for RDM is thus targeted more towards complex decision processes under deep uncertainty, where relatively simple prediction analysis is not enough and often will be misleading. R.J. Lempert thought it would be better to seek a robust solution rather than trying to predict the future and then choose the solution that fits the predicted future best. He disagreed with the traditional framework for assessing alternatives in decision processes, in which he thinks predictions were imposed too great a significance. He argued that the reason for decision-makers to use predictions were to make the future less scary and that the "the quest for prediction probably fills some deep human need. Even though the accuracy of most predictions has proven to be poor"". A combination of new technologies and the need for a method to handle more complex issues resulted in the development of RDM, that instead of finding the alternative that seems to be the best based on specific predictions and probabilities, uses modern technology to analyse and evaluate alternatives against thousands or even millions of possible scenarios. The RDM method gives the decision-maker a tool to identify trade-offs when testing strategies against different scenarios. In that way, decision-makers can now determine which strategies that perform best under a large range of possible scenarios. If used correctly, RDM should make the decision-makers able to develop the most robust strategies that can handle the future, regardless of how the future ends up being.

RDM consist of the following key elements:

• Testing of multiple plausible futures. The combination of scenarios for the future should be significantly diverse. The reason for this is that the main idea of RDM is that you are interested in analysing how different strategies perform in as many different futures as possible. These multiple scenarios can also correspond to different persons in the decisionmaking groups view of the world. Instead of spending time debating which prediction of the future too chose, as one would do in the traditional “predict-then-act” framework, the decisionmaking group can use their time more effective while testing everybody's idea of the future at the same time.

• The goal is not to find the absolute best strategies but to find the most robust ones. The main focus in RDM is to find strategies that perform well over most if not all possible scenarios.

• Apply adaptive strategies. One way to secure that strategies are robust is to choose some who are adaptive. Adaptive means that when the future comes, the strategies can adapt to the new circumstances and still perform well under the new conditions.

• The computer is used to ease the human job, not to give the final recommendation of which specific strategy that should be used.

The foundation of RDM

The primary theoretical focus lays around uncertainty theories more specifically in the area of Decision Making Under Deep Uncertainty (DMDU), the reason for this is that the RDM methodology is used for developing strategies under deep uncertainty. As stated in The standard for portfolio management, uncertainties are in general just the result of lack or limited knowledge about a certain state in time and is almost impossible to avoid when taking decisions in our modern world . Many books have been written about decisionmaking under uncertainty, but only a very limited amount includes DMDU. Even though the literature about DMDU is limited, however increasing, several tools and different analytical approaches exist. They share the underlying paradigm that they all seeks to reduce strategies vulnerability against the uncertainties associated with the future, which in some literature is defined as Assumption Based Planning (ABP).

Uncertainty arises from imperfect or incomplete information.

RDM is developed on the basis of some already existing methods, paradigms and approaches, the most important ones will be reviewed in the following section.

Decision Analysis

A part of RDM's foundation is Decision Analysis (DA). The reasoning behind DA is that the human, by including decision tools, make better decisions. It applies whether we are a group or an individual. DA provides the decision-makers with tools to enlighten what affect certain decisions will have on different situations under uncertainty.. DA can take form as e.g. Decison Trees or Expected Value (EV) calculations, but for more complex decisions under deep uncertainty, it is necessary to include more advanced methods and computer modelling. It could e.g. be RDM which is categorised as a quantitative DA method. DA can be divided into two different approaches, the first approach is called "agree-on-assumption" or "predict-then-act" and the second one, which RDM follows, is an "agree-on-decisions" approach. The idea of stress testing is one concept RDM has derived directly from "agree-on-decision" approach. The first approach seeks to rank different strategies based on agreed probabilities of how likely it is that the world is in a certain state in the future, known from the traditional DA framework. The second approach test different strategies against multiple scenarios of the future, which is used to find vulnerabilities of the strategies and mitigating actions to handle these vulnerabilities . According to C. Helgeson (2018), the "agree-on-decision" approach as RDM follows, can be summarized as an approach to support the decisionmaking group in the process of framing decisions under deep uncertainty .

Assumption-Based Planning

RDM uses the properties of stress testing to challenge a variety of scenarios and strategies, to reduce the possibility of failure by improving the decision-makers understanding of when, why and what that can go wrong. More specifically, RDM takes advantage of a certain practice of this, called Assumption-Based Planning (ABP) . ABP takes a starting point in a written plan and then seeks to identify the underlying assumptions on which the existing plan is based on. Only crucial assumptions, as if not goes as expected can lead to failure, is relevant in ABP. Further, ABP structures the crucial assumptions and uses them in the framework seen in RDM to develop adaptive strategies. Adaptive strategies or adaptive planning consist of several key elements, which all share that they in order to uncover uncertainties, uses computational power . The ABP approach secures that decision-makers using RDM tests a large number of uncertainties and develops both actions to be taken in the short-term and long-term, but with the possibility for long-term actions to change according to how situations develop. This includes developing signposts/signals that if observed in the future, will result in different actions to secure that the adaptive strategies won't fail .

Scenarios

Scenario analysis is another concept RDM uses. It is a way to represent different descriptions of plausible future states. Scenario analysis contributes to dividing different future states into multiple scenarios which makes it easier for the decision-makers to analyse and discuss. In contrast to ranking future states based on probabilities, scenario analysis analyses different views of how a situation will develop. Focusing on possibilities instead of probabilities gives the decision-makers the possibility to analyse more and diverse events, compared to if they had to focus on only the events that most likely would happen. At the same time, the decision-makers are faced with more extreme and unexpected future events which help to test the relevant strategies against unlikely and more extreme scenarios than one might normally do . There exist different types of algorithms to do scenario analysis and the ones RDM uses is a quantitative approach of Scenario Discovery (SD) algorithms. In RDM, the decisionmakers use the final scenarios from the scenario analysis to capture the most important findings from the ABP stress test, which at the end of the RDM forms the basis for the decision-makers to make adaptive strategies.

Exploratory Modeling

RDM would not exist without Exploratory Modeling (EM). EM is particularly beneficial when it is not possible to use a model, because of e.g. missing data or if the future is associated with great uncertainty. It unites DA, ABP and SD and combines these concepts, by running the same model with sometimes only small changes in the input, even though many would normally argue that it is not good practice to run a model many times. In RDM this repeating is one of the key factors to achieve the targeted knowledge and to develop a large database of results. Another benefit of EM is that it does not have any starting point, and thus there are no restrictions for the possibilities of exploring different plausible futures and strategies . EM can be used to analyse a variety of topics but for RDM, the goal of using EM is focused on finding strategies with a high degree of robustness. Besides being preferred because of the ability to support decisions under deep uncertainty, EM's computational simplicity is a big advantage, especially compared to alternative approaches that include Dynamic Programming or different optimization methods.

Application of RDM

RDM has been used to support a wide range of decisions, but RDM is not suitable in any decision process. RDM is most suitable if all or some of the following characteristics apply to the problem that needs to be handled. The primary condition is that there is a high degree of uncertainty (deep uncertainty) associated with the problem, other conditions that to some extent is related to deep uncertainty is if the problem is very complex, or if the decision-makers have a wide range of possible options to choose from . It also includes if it is a problem that needs computational power in order to be solved, e.g. if it is necessary to test a large number of strategies against multiple scenarios. In general, RDM is applicable to most problems regarding DMDU as long as the decision-makers are not bound to a small number of mitigating possibilities . For less complex issues, it will normally be preferred to use a "predict-then-act" methodology, even though RDM in most cases also would be able to support the decision process in these cases. However, it would be an over-complication of the decision process, which is why RDM is not used in all decision processes.

Steps in RDM

Steps in an RDM analysis

Depending on the literature, the framework for an RDM may vary slightly, but overall the content is similar. The figure shown here is the latest version of R.J. Lempert's frameworks (founder of RDM) and is based on previous versions of the framework . A more in-depth go-through of the steps can be found in R.J. Lemperts book Robust Decision Making in Decision Making under Deep Uncertainty.

It is important to keep in mind that RDM is an iterative process, which the figure also shows, and that the new knowledge acquired in each step of the RDM can lead to a necessity for re-framing the problem and its challenges.

Step 1 Decision Framing

As shown in the figure, the stakeholders start out with a decision framing exercise, in which they have to define the key factors relevant for further analysis:

• Objectives and criteria
• Strategies/Alternatives that successfully can handle the objectives
• Uncertainties and their consequences
• Relationship between strategies, objectives and uncertainties

Step 2 Evaluate Strategy Across Futures

Next step is to use an "agree-on-decision" approach". To do so, RDM uses simulations of the strategies found in Step 1, against a variety of plausible scenarios of the future. These simulations generate a large number of results, which is merged into a large database and the results are evaluated across the different futures .

Step 3 Vulnerability Analysis

Step 3 tries to identify, explore and characterize vulnerabilities. The decision-making team can use visualisations and data analytics based on the database made in step 2. In order to identify the key factors that contribute to the success or failure of the strategies, it is common practice to use Scenario Discovery (SD) algorithms. The use of SD algorithms results in clusters of futures that will enlighten which factors the strategies are most vulnerable to.

Step 4 Tradeoff Analysis

A tradeoff analysis is performed on the scenarios from step 3. In step 4 different strategies are evaluated in order to analyse the tradeoffs among different strategies. This is often done by making multi-objective tradeoff plots in which tradeoff curves can be used to e.g. compare cost against reliability for different scenarios. The tradeoff curves can be used by the decision-makers to determine what balance they evaluate to be most suitable among the objectives defined in step 1.

Step 5 New Futures and Strategies

Finally, the decision-making team can then use the findings from step 4 to evaluate if they have found any robust strategies, with better tradeoffs, compared to existing strategies. The final strategies from the RDM will often be what you call, Adaptive Decision Strategies, which are strategies that adapt depending on how the future develops. The adaptive strategies consist of a short-term strategy with actions to be taken in the near future and so-called signposts which are predesignated signals that when observed, certain actions will be taken. These adaptive strategies can either be established from a jury of experts , or the most suitable combination of near-term actions, signposts and actions to be taken if the signals are observed, can be determined with the help of optimization algorithms.

Benefits and Limitations of the RDM Framework

Benefits

This section may be added

Limitations

RDM is not an easy method to follow and it certainly has it's requirements in order to be performed correctly, but as the philosophy behind RDM is " that it is better to be roughly right than precisely wrong", the models used to perform the RDM is often relatively simple and fast models. Even if a model solely is simple, problems can occur when it is necessary to run the model thousands or even millions of times, with a large number of different strategies and scenarios, which RDM involves. It sets requirements for the amount of data which in some cases, e.g. in developing countries, maybe doesn't exist causing it to be almost impossible to perform RDM . Besides data and computing requirements, then expert knowledge on the field of concern is also needed. If the decision-making group does not possess this knowledge, they have to hire external experts which can be costly. Another cost could be external help to perform the modelling, there even though the models are simple, acquires some computer modelling skills to perform.

Another limitation, as the annotated literature on RDM shows, the RDM methodology is mostly used to support DMDU that have consequences in the long-term. Among other things, this is due to the fact that the uncertainty associated with a given decision is largely related to time. Short-term strategies will only very rarely be associated with a high degree of uncertainty (deep uncertainty), which is why RDM is primarily limited to long-term decisions.

Annotated bibliography

Lempert R.J, Marchau V., Walker W., Bloemen P., Popper S. (2019) Robust Decision Making in Decision Making under Deep Uncertainty .

This book is some of the latest literature about DMDU. It contains various approaches, applications, and implementations to handle DMDU, including RDM. It provides an in-depth guide on how RDM is performed and a concrete example of the implementation of RDM, both are very useful to look into if one wants to perform RDM. Besides being one of the newest books it is also one of the most detailed so those who want to make use of RDM this book is a must-read.

Lempert R.J., Popper S.W., Bankes S.C. (2003), Shaping the next one hundred years: New methods for quantitative, long-term policy analysis .

Shaping the next one hundred years was groundbreaking in relation to the hypothesis it was built around back in 2003. The authors stated that with the help of computers, it was possible to change the way long-term futures is predicted. They suggested scenario generator- and exploratory modelling software. Among other things, they mention in the book that the long-term future is defined by deep uncertainty. They also talk about robustness and adaptive strategies, all something that can be related to RDM.

xxx

xxx

xxx

xxx