Robust Decision Making (RDM)

From apppm
(Difference between revisions)
Jump to: navigation, search
(→‎Step by Step methodology)
(→‎Limitations)
 
(85 intermediate revisions by one user not shown)
Line 1: Line 1:
 
'''Abstract''':  
 
'''Abstract''':  
−
In some situations, decisions on projects or program management must been made under deep uncertainty <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. In this case, the classic risk management tools are not well adapted. Robust Decision Making (RDM) is one of the solutions which can be used to correctly consider this high level of uncertainty. In contrary of the classic risk management tools, RDM helps decision makers to choose not the optimal solution, but the solution (of set of solutions) which is the less bad regarding the set of possible future<ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>). After having presented the general idea and main principles of RDM, this article aims to give a guideline so that the readers can implement it to their project. This includes a step-by-step methodology and a very short presentation of the type of tools needed. The relevance and limitations of RDM are discussed at the end of this article. In this article if nothing is specified, the notions used are defined according to the PMI standards <ref name="standard">Project Management: A guide to the Project Management Body of Knowledge (PMBOK guide), 7th Edition (2021) </ref>
+
In some situations, project, program or portfolio management decisions must be made in the face of deep uncertainty. <ref name="name">Vincent A. W. J. Marchau Warren E. WalkerPieter J. T. M. Bloemen Steven W. Popper (2019)Decision Making under Deep Uncertainty https://doi.org/10.1007/978-3-030-05252-2 </ref>. In this case, it is not possible to define probabilities for possible futures.  Therefore, classical risk management tools are not adapted. <ref>Project Management Institute, Inc. (PMI). (2019). Standard for Risk Management in Portfolios, Programs, and Projects. Project Management Institute, Inc. (PMI). Retrieved from https://app.knovel.com/hotlink/toc/id:kpSRMPPP01/standard-risk-management/standard-risk-management </ref>. Robust decision making (RDM) is one solution that can be used to properly account for this high level of uncertainty. Unlike traditional risk management tools, RDM helps decision makers choose not the optimal solution, but the solution (or set of solutions) that is the least bad compared to the set of possible futures. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. After presenting the general idea and main principles of RDM, this article aims to give a guideline so that readers can apply it to their project. This includes a step-by-step methodology and a very brief presentation of the type of tools needed. The relevance and limitations of RDM are discussed at the end of this article. In this article, if nothing is specified, the concepts used are defined according to the PMI standards.<ref name="standard">Project Management: A guide to the Project Management Body of Knowledge (PMBOK guide), 7th Edition (2021) </ref>.
  
  
Line 7: Line 7:
  
 
=== General Idea ===
 
=== General Idea ===
−
As defined by Dr Alistair Hunt , Robust decision making is “ a methodology which aims to identify adaptation options or strategies which can perform well over a wider range of possible futures. “. <ref name="ecotoolbox">https://econadapt-toolbox.eu/robust-decision-making </ref>. The particularity of this method compared to the classic risk management tools, is that the goal is not to find the optimal solution to a problem but to create a robust solution that is satisfactory for a high number of potential futures <ref >https://doi.org/10.1016/j.envsoft.2016.09.017</ref>
+
As defined by Dr Alistair Hunt , Robust decision making is “ a methodology which aims to identify adaptation options or strategies which can perform well over a wider range of possible futures. “. <ref name="ecotoolbox"> Dr Alistair Hunt ROBUST DECISION MAKING https://econadapt-toolbox.eu/robust-decision-making </ref>. The particularity of this method compared to classical risk management tools is that the goal is not to find the optimal solution to a problem but to create a robust solution that is satisfactory for a large number of potential futures.<ref name="Robust decision making" >Jan H. Kwakkel, Marjolijn Haasnoot, Warren E. Walker,Comparing Robust Decision-Making and Dynamic Adaptive Policy Pathways for model-based decision support under deep uncertainty,Environmental Modelling & Software, Volume 86, 2016, Pages 168-183, ISSN 1364-8152, https://doi.org/10.1016/j.envsoft.2016.09.017</ref>.
  
−
In other terms, RDM doesn’t consist of trying to find the most plausible future and adapt your strategy to it. Instead, the gist of this method is to consider many potential future situations (see next parts for more details) and to assess strategies regarding all the different scenarios. Then the ones which are most robust are valorized compare to those which are optimal in one specific situation <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. This method conducts to choose not best but the less bad strategy. The general foundations and a step by step approach are described later in this article.
+
In other words, RDM is not about trying to find the most plausible future and adapting one's strategy to it. Instead, the essence of this method is to consider many potential future situations (see the following sections for more details) and evaluate strategies under all the different scenarios. Then, the most robust strategies are valued over those that are optimal in a specific situation. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. This method leads to choosing not the best but the least bad strategy. The general foundations and a step-by-step approach are described later in this article.
  
 +
=== Fields of application===
  
−
=== When to use it ?===
+
A situation of uncertainty and/or ambiguity implies having limited and imprecise knowledge of future events.<ref name="standard">Project Management: A guide to the Project Management Body of Knowledge (PMBOK guide), 7th Edition (2021) </ref>. When these uncertainties are deep, it becomes so difficult to accurately anticipate the future that traditional prediction or risk management tools cannot be used effectively. Indeed, a high level of uncertainty implies that it is not possible to put reliable statistics or probability distributions on possible future scenarios. And classical approaches need this according to project management standards<ref name="standard">hProject Management: A guide to the Project Management Body of Knowledge (PMBOK guide), 7th Edition (2021) </ref>.An alternative is to consider as many possible futures as possible in the evaluation process in order to reduce the risk that their solutions will not fit the actual future that will occur.
  
−
A situation of uncertainties and/or ambiguity implies to have a limited and unclear knowledge about future events <ref name="standard">Project Management: A guide to the Project Management Body of Knowledge (PMBOK guide), 7th Edition (2021) </ref>. When those uncertainties are deep, anticipate precisely the future become so hard that the classic tools of predictions or risk management cannot been use efficiently. Indeed, a high level of uncertainties implies that it is not possible to put reliable statistics or probabilities distributions on possible futures scenarios. And classic approaches need it according to <ref name="standard">hProject Management: A guide to the Project Management Body of Knowledge (PMBOK guide), 7th Edition (2021) </ref>.An alternative is then to consider as many possible futures as possible in the evaluation process in order to reduce the risk of their solutions to be totally not adapted to the actual future which will happen.
 
  
 +
As already mentioned, one of the particularities of RDM is that it takes into account a very large number of scenarios in the decision-making process. It is therefore specifically adapted to the context of deep uncertainties. More precisely, these deep uncertainties can appear mainly in three situations: <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>:
  
−
As we mentioned it before, one particularity of RDM is to consider a very large number of scenarios in the decision process. Therefore, it is specifically adapted to the context of deep uncertainties. A review of the state-of-the art uses of RDM can show us that almost all the current projects which use it are living in a context of such high level of uncertainties.
 
  
 +
-  First, if the contextual uncertainties are profound. That is, if the characteristics of the context in which the project, program or portfolio will evolve cannot be precisely defined.  This is the case, for example, when the prediction of the future requires climate modeling. Indeed, global warming will have impacts that are very difficult to describe and anticipate precisely. <ref>Bhave, Ajay Gajanan, Conway, Declan, Dessai, Suraje and Stainforth, David
 +
A. (2016) Barriers and opportunities for robust decision making approaches to support climate change adaptation in the developing world. Climate Risk Management, 14 . pp. 1-10. ISSN 2212-0963 DOI: 10.1016/j.crm.2016.09.004</ref> .
  
−
More precisely, these deep uncertainties can appears mainly in three situations <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref> .  First if the contextual uncertainties are deep. That is to say if the characteristics of the context in which will evolve the project, program or portefolio cannot be precisely defined.  It is for example the case when the prediction of the future include climate modelization. Indeed, global warming will have impact that it is really difficult to precisely describe and anticipate. <ref>http://eprints.lse.ac.uk/68318/1/Conway_Barriers%20and%20opportunities%20.pdf </ref> . Then uncertainties increase when “the set of policies has more rather than fewer degrees of freedom” <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. It means that the more policies can change and influence your project, the more difficult it is to anticipate future situations. Finally, another source of uncertainties is the disagreement which can exists between different experts on the fields concerned by your projects. Indeed, if there is as strong disagreement, it is then impossible to set only one reliable hypothesis. Then many possible futures must be considered.  
+
- Second, uncertainties increase when “the set of policies has more rather than fewer degrees of freedom” <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. This means that the more policies can change and influence your project, the more difficult it is to anticipate future situations.  
  
 +
- Finally, another source of uncertainty is the disagreement that may exist between different experts on the fields concerned by your projects. Indeed, if there is such a strong disagreement, it is impossible to establish a single reliable hypothesis. And so you have to consider many possible futures.
  
−
The field of application we can find are numerous. Here is a non-exhaustive list <ref>https://millennium-project.org/wp-content/uploads/2020/02/22-Robust-Decisionmaking.pdf </ref> :defense, higher education, insurance, science and technology planning, counter-terrorism…Within the state of the art, RDM is very often used when a project, a program or a portfolio has to include climate change in the decision <ref name="ecotoolbox">http://eprints.lse.ac.uk/68318/1/Conway_Barriers%20and%20opportunities%20.pdf </ref> <ref name="ecotoolbox">https://econadapt-toolbox.eu/robust-decision-making </ref>
+
 
 +
The fields of application that can be found are numerous. Here is a non-exhaustive list: <ref>Robert Lempert, Steven Popper, Steve Bankes. ROBUST DECISIONMAKING (RDM) https://millennium-project.org/wp-content/uploads/2020/02/22-Robust-Decisionmaking.pdf </ref> :defense, higher education, insurance, science and technology planning, counter-terrorism... In the current state of knowledge, RDM is very often used when a project, program or portfolio needs to include climate change in the decision. <ref name="ecotoolbox">http://eprints.lse.ac.uk/68318/1/Conway_Barriers%20and%20opportunities%20.pdf </ref> .
  
 
=== RDM fondations ===
 
=== RDM fondations ===
  
−
As mentioned in literature <ref name="open knowledge">https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref> RDM don’t use computer models and data as a prediction tool to anticipate the best estimate future. Instead, RDM use the principle of exploratory modelling <ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref> . It means that a lot of possible future with many different paths are generated in order to stress proposed strategies. The goal is not to rank the plausible scenarios but to explore as many of them as possible. The confrontations of the pre-decided strategies to a large range of scenarios are then used to create a database of KPI and outcomes. In this data base two main types of data can be found.  
+
As mentioned in the literature <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref> , RDM does not use computer models and data as a predictive tool to anticipate the best estimate of the future. Instead, RDM is based on the principle of exploratory modeling <ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref> . This means that a large number of possible futures with many different paths are generated to test the proposed strategies. The objective is not to rank the plausible scenarios but to explore as many as possible. Comparisons of the pre-decided strategies with a wide range of scenarios are then used to create a database of key performance indicators and outcomes. In this database, there are two main types of data.  
  
  
−
The first is all the relevant data to assess if one strategy is adapted to one specific future regarding the achievement of pre-defined goals. The data use to say if a strategy is adapted or not to a situation really depends on the situation. It exists a large literature which describes how to choose KPI.Give example of data which can be relevant. RDM can use these results and methodologies to include them in the process.
+
The first is the set of data relevant to assessing whether a strategy is suitable for a specific future regarding the achievement of predefined goals. The data used to say whether a strategy is suitable or not for a situation really depends on that situation. There is an extensive literature that describes how to choose KPIs and gives examples of data that may be relevant. RDM can use these results and methodologies to include them in the process.
−
The second type of data is a description of the path used to generate the range of possible futures. That is to say, all the characteristic of one path which are relevant to differentiate it from another one. Those date are mostly generated by coupling algorithms able to generate scenarios and software which can explore those scenarios (comparing Robust Decision-Making and Dynamic Adaptive Policy Pathways for model-based decision support under deep uncertainty )- see steps-by-steps section for more info.
+
  
 +
The second type of data is a description of the path used to generate the range of possible futures. That is, all the characteristics of one path that are relevant to differentiate it from another. This data is mainly generated by coupling algorithms capable of generating scenarios and software capable of exploring these scenarios. <ref name="Robust decision making" >https://doi.org/10.1016/j.envsoft.2016.09.017</ref> - For more information, see the Steps by Steps section.
  
−
With this database<ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>data scientists can then use analytics to identify what are the common characteristics of the futures which make a strategy miss or reach its goals. They can also put the light on the strengths and weaknesses of the strategy regarding a set of different scenarios. These two first steps can be repeated for a large variety of input strategies and assumptions. Contrary to many classic risk assessments tools, the output of analytics in RDM process is not a predefined solutions but a set of key features which identify under which assumptions, which policies and which scenarios a strategy reach its goals.
 
  
 +
With this database, data scientists can then use analytics to identify common characteristics of futures that cause a strategy to fail or achieve its goals. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. They can also highlight the strengths and weaknesses of the strategy against a range of different scenarios. These first two steps can be repeated for a wide variety of input strategies and assumptions. Unlike many traditional risk assessment tools, the outcome of the analysis in the RDM process is not a predefined solution, but a set of key characteristics that identify under which assumptions, policies, and scenarios a strategy achieves its objectives.
  
−
When all those features are known, they can be transmitted to the decisions makers who will realize a tradeoff analyses in order to come up with a robust strategy. Strategies can then be modified of combine in order to find one which is robust. That is to say for <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>“ones that meet multiple objectives over many scenarios “. The all process described above can be reiterated multiples time to stress again the new strategies and to come up with an even more robust decision.
 
  
 +
When all of these characteristics are known, they can be passed on to decision makers who will perform a trade-off analysis to find a robust strategy. The strategies can then be modified or combined to find a robust one. That is to say “ones that meet multiple objectives over many scenarios “<ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. The process described above can be repeated several times to reveal new strategies and result in an even stronger decision.
 +
 +
Because RDM is useful for defining strategy, it should be used early in the project/program or portfolio life cycle.
  
 
=== Collaboration Human/machine ===
 
=== Collaboration Human/machine ===
  
−
One important point of RDM is that it relies on a collaboration human-machine <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. What make the strength of this method is to combine the power of human decision making and the computational power of computers. Computers are used to generate and explore many different scenarios which are themselves used to stress strategies imagined by human. When it comes to deep uncertainties with a huge number of assumptions to explore, computational power is absolutely needed to cope with the massive among of data to proceed. On the other hand, the decision part and the creation of potential tradeoff then only relies on decision makers who can use the output of the previous computer-oriented steps as a support. This mix between computer science and human intelligence implies to use a set of different tools. As we can see it through different applications of RDM (insert different links with applications), the RDM method only define which general type of tool you need for each step. The exact tools to use are context dependent and must be defined for every new project. A very quick overview and guidance regarding the appropriate type of tools to use are detailed under the step-by-step section.
+
An important point of RDM is that it is based on human-machine collaboration. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. The strength of this method is that it combines the power of human decision making with the computational power of computers. Computers are used to generate and explore many different scenarios which are in turn used to stress the strategies devised by humans. When it comes to deep uncertainties with a large number of hypotheses to explore, computing power is absolutely necessary to cope with the mass of data to be processed. On the other hand, the decision-making part and the creation of potential trade-offs then rely solely on the decision-makers who can use the results of the previous computational steps as support. This mix between computer science and human intelligence implies the use of a different set of tools. As the various applications of the RDM method show, it only defines the general type of tool you need for each step. The exact tools to use depend on the context and must be defined for each new project. A very quick overview and advice on the type of tools to use are detailed in the "step by step" section.
  
−
[[File:Interaction_human_machine.jpg]]
+
[[File:RDM.drawio.png]]
  
 
== Step by Step methodology ==
 
== Step by Step methodology ==
  
−
The aim of this section is to give a detailed guideline for the reader to be able to apply RDM to a new project, program, or portfolio management situation. Each sub-section presents a step of the process and briefly discuss the useful tools for this section. The presentation order is chronological. The different steps are based on the methodology illustrated by Robert Lempert in the following graph <ref name="open knowledge"> https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>
+
The purpose of this section is to provide a detailed guide for the reader to apply RDM to a new project, program or portfolio management situation. Each subsection presents a step in the process and briefly discusses the tools useful for that section. The order of presentation is chronological. The different steps are based on the methodology defined by Robert Lempert. <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>
  
  
−
[[File:Step by step method for RDM.drawio.png|400px|thumb|right|Illustration of RDM process by Swann Roussillon and inspired by  Robert Lempert <ref name="open knowledge">https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>.]]
+
[[File:Step by step method for RDM.drawio.png|400px|thumb|right|Illustration of RDM process by Swann Roussillon and inspired by  Robert Lempert <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>]]
  
  
 
=== Define Strategies  ===
 
=== Define Strategies  ===
  
−
As for every Projet, Program or Portefolio management situation, decision makers have to define a set of strategies. These strategies are probably going to evolve during the RDM process ( “Shaping the Next One Hundred Years book”) but a foundation is needed to start the process. For a new project with high level of uncertainties, it may be useful to define a set of strategy that has to be tested instead of just one. Indeed, later the process will maybe lead to a combination of different strategies in order to get a robust one.  
+
As with any project, program, or portfolio management situation, decision makers must define a set of strategies. These strategies will likely evolve over the course of the RDM process, but a baseline is needed to start the process. <ref name="Shaping the Next One Hundred Years">Lempert, Robert J., Steven W. Popper, and Steven C. Bankes, Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis, Santa Monica, Calif.: RAND Corporation, MR-1626-RPC, 2003. As of March 05, 2022: https://www.rand.org/pubs/monograph_reports/MR1626.html </ref>. For a new project with a high level of uncertainty, it may be useful to define a set of strategies to test rather than a single one. Indeed, later on, the process may lead to a combination of different strategies to obtain a robust strategy.
−
 
+
  
 
===  Define the uncertainties as much as possible ===  
 
===  Define the uncertainties as much as possible ===  
  
−
One of the crucial aspects to use correctly RDM, is to manage to be as comprehensive as possible in the description of the uncertainties. This is a huge foundation to unable simulation modeling to generate a lot of plausible solutions. The sources of uncertainties that must be explored are principally <ref name="name"> https://doi.org/10.1007/978-3-030-05252-2 </ref>:  
+
One of the crucial aspects of using RDM correctly is to be as complete as possible in describing the uncertainties. This is a huge basis for simulation modeling to be unable to generate a large number of plausible solutions. The sources of uncertainties that need to be explored are primarily: <ref name="name"> https://doi.org/10.1007/978-3-030-05252-2 </ref>:  
  
−
- The contextual uncertainties. Basically, it is related to all the physical things included in your scope and for which you are not able to predict the evolution. One of the major examples of this part is all the strategies which must include climate or environmental simulations. Because it is not yet possible to have precise predictions, the level of uncertainties is very high.  
+
- Contextual uncertainties. These are basically all the physical things included in your scope that you are not able to predict. One of the main examples of this part is the set of strategies that must include climate or environmental simulations. Since it is not yet possible to have precise predictions, the level of uncertainty is very high.
  
−
- The policies uncertainties. This part includes all the things related to a decision making which can influence the result of your strategies and which are out of your control. For example, futures taxes, future environmental laws, future local restrictions etc… can be part of this category.
+
- The policy uncertainties. This section includes all elements related to decision making that may influence the outcome of your strategies and that are beyond your control. For example, future taxes, future environmental laws, future local restrictions, etc. may fall into this category.
−
 
+
−
- The assumptions uncertainties. It may happen that experts disagree on some assumptions related to your strategies. If the disagreement is high, those assumptions generate uncertainties that you have to consider.
+
  
 +
- Uncertainties related to assumptions. Experts may disagree on certain assumptions related to your strategies. If the disagreement is significant, these assumptions create uncertainties that you must consider.
  
 
=== Set objectives and metrics for the Strategies ===
 
=== Set objectives and metrics for the Strategies ===
  
−
Later in the process, the goal will be to stress the different strategies with a large set of possible future and to evaluate if they are adapted or not. But a prerequire step is to define how this evaluations can be down <ref name="open knowledge">https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>. At the end of this step, it is crucial to be clear on what have to be measured, how it has to down and what values defined an acceptable result.  
+
Later in the process, the goal will be to subject the different strategies to a wide range of possible futures and evaluate whether or not they are appropriate. But a preliminary step is to define how these evaluations can be done. <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>. At the end of this step, it is crucial to be clear about what is to be measured, how it is to be measured, and what values define an acceptable outcome. Objectives and measures are used to assess the degree of success of the project, program or portfolio. A typical example might be the mayor of a city who has plans to build a new dam to protect the urban area from flooding in the context of global warming. He/she might decide on the following set of objectives and actions:
−
# to_do_later (include here some link to relevant tool to choose metrics and objectives)
+
  
 +
- The future dam should reduce property damage in the event of a flood. The total property damage in dollars could be defined as the metric. And a maximum acceptable level can be decided ($100,000 for example).
  
−
=== Define a plan for conducting the simulation modeling ===
+
- The future dam should protect the citizens. A goal of zero people injured could be defined.
  
−
The simulation modeling is a key factor of RDM. Two things have to be down during the simulation (“Shaping the Next One Hundred Years book”). First, the model has to be able to generate a large range of potential futures based on the uncertainties define previously. Second the model needs to give access to the different steps it is creating in all the potential futures. This is essential for enable humans to understand what are the impact of the hypothesis made at each steps on the strategies. The first point requires a scenario generator (include links to present some scenarios generators) and the second an exploratory modeling software. It is then required to choose software adapted to the field and the scope of your project.  
+
- The future dam must be affordable for the city. The total price of construction and maintenance in dollars could be defined as the measure. And a maximum of $10 million might be an acceptable outcome.
  
−
Also, your simulations will generate a lot of data. In order to make it usable for the next steps, it is crucial to define how this data will be collected and stored.
+
=== Define a plan for conducting the simulation modeling ===
  
 +
The simulation model is a key factor in RDM. There are two things that must be considered during the simulation.<ref name="Shaping the Next One Hundred Years">Lempert, Robert J., Steven W. Popper, and Steven C. Bankes, Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis, Santa Monica, Calif.: RAND Corporation, MR-1626-RPC, 2003. As of March 05, 2022: https://www.rand.org/pubs/monograph_reports/MR1626.html </ref>. First, the model must be able to generate a wide range of potential futures based on the uncertainties defined above. Second, the model must provide access to the different steps it creates in all potential futures. This is essential to allow humans to understand the impact of the assumptions made at each stage on the strategies. The first point requires a scenario generator and the second requires exploratory modeling software. It is then necessary to choose a software adapted to the domain and the scope of your project.
  
−
A last point which must be considered during the planning phase is the computational power requirement by the scenario generation. As mentioned in (find in one of the previous article the part on computational power) it can requires a lot of computational performances. And even if computers are making more and more progress, a too large number of scenarios may overtake the capacities you have or create a really long waiting time. Therefore, it is necessary to anticipate on those aspects and to reduce the complexity of the scenario generations if needed.
+
In addition, your simulations will generate a lot of data. In order to make it usable for the next steps, it is crucial to define how this data will be collected and stored. The knowledge and tools used in the field of data analysis can be applied here.
  
−
More information can be found on the references dedicated to exploratory modeling. <ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref>
+
A final point that needs to be considered during the planning phase is the computational power required by the scenario generation. As mentioned in the collaboration Human/Machine part, this can require a lot of computing performance. And even if computers are making more and more progress, too many scenarios can exceed the capacities you have or create a very long waiting time. It is therefore necessary to anticipate these aspects and to reduce the complexity of scenario generations if necessary.
  
 +
More information can be found in the references dedicated to exploratory modeling. <ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref>
  
 
=== Define a strategy of exploration and analyze of the data produced by the simulation modeling ===
 
=== Define a strategy of exploration and analyze of the data produced by the simulation modeling ===
  
−
When all the scenarios would have run, you will ended with a big database. And it is the exploration and the analyze of this database which will help to create value for RDM. So, it is important to have predefined strategies to exploit these data in an effective way. The method of RDM helps you to define what you are looking for. Indead <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref> the goal is to identify and visualized the key factors of the scenarios which make your strategies work or not. In order to do this, classic tools of data analysis and machine learning can be used. (give an example of tools reference for data analyses or machine learning tools). You can also decide how you want to present your results. An important point to keep in mind is that you are not directly looking for one option to select. But you want to compare your different options. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>
+
When all the scenarios have been run, you will end up with a large database. And it is the exploration and analysis of this database that will create value for RDM. So it's important to have predefined strategies to leverage that data effectively. RDM's methodology helps you define what you are looking for. In fact, the goal is to identify and visualize the key factors in the scenarios that make your strategies work or not.<ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref> To do this, you can use classic data analysis and machine learning tools. You can also decide how you want to present your results. An important point to keep in mind is that you are not directly looking for an option to select. But you want to compare your different options. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>
−
 
+
  
 
=== Run the simulation ===
 
=== Run the simulation ===
  
−
This step only consists of applying the strategy defined in step D. The expected output is the data base mentioned previously and a way to visualize the path through the different futures.
+
This step consists only of applying the strategy defined in the action "Define a plan for conducting the simulation modeling". The expected result is the database mentioned above and a way to visualize the path through the different futures.
−
 
+
  
 
=== Analyze and characterize the database produced ===  
 
=== Analyze and characterize the database produced ===  
  
−
This step consists of applying the methodology defined in the point e. The expected outcome is an identification and a visualization of the key factors of the scenarios for which your strategies are work or not. The goal here is not to make a conclusion but to create a support which will help decision makers to do the tradeoff Analysis described below.
+
This step consists of applying the methodology defined in the section "Define a strategy of exploration and analyze of the data produced by the simulation modeling". The expected result is an identification and visualization of the key factors in the scenarios for which your strategies do or do not work. The goal here is not to make a conclusion but to create a support that will help decision makers make the trade-off analysis described below.
−
 
+
  
 
=== Realize a tradeoff Analysis ===
 
=== Realize a tradeoff Analysis ===
  
−
This step is where the value off the RDM in enlighten. During this phase, decision makers must use the results from the previous step to compare the different strategies. Specifically, the strengths and weaknesses of each method can be comparer considering the different scenarios. The goal is not to directly select a strategy. Instead, decision makers may want to see if it possible to combine two or more strategies in order to obtain a one which is more robust <ref name="name">https://doi.org/10.1007/978-3-030-05252-2</ref>. That is to say, a strategy which is better than the others when considering all the possible futures.
+
It is in this step that the value of the RDM is highlighted. In this phase, decision makers should use the results of the previous step to compare the different strategies. Specifically, the strengths and weaknesses of each method can be compared by considering the different scenarios. The goal is not to directly choose a strategy. Instead, decision makers may want to see if two or more strategies can be combined to produce a more robust strategy.<ref name="name">https://doi.org/10.1007/978-3-030-05252-2</ref>. That is, a strategy that is better than the others when considering all possible futures.
  
  
 
=== Iterate the process if necessary  ===
 
=== Iterate the process if necessary  ===
  
−
After the tradeoff analysis, decision makers can come up with new strategies they would like to try or can review some assumptions used to creates the different scenarios. Then they can redo the all process and hope that it will help them to come up with a more robust solution. These new iterations are less time consuming because all the methodologies and the tools are the same than during the first one.
+
After analyzing the trade-offs, decision-makers can propose new strategies they would like to try or revisit some of the assumptions used to create the different scenarios. They can then start the whole process again, hoping that this will help them find a more robust solution. These new iterations take less time because all the methodologies and tools are the same as in the first iteration.
  
 
==Limitations ==
 
==Limitations ==
  
−
As it a has been presented before, one of the particularities of RDM is to be well adapted to situations with a high level of uncertainties. But the relevance to use this method and not the classic probabilistic risk analysis is really limited when we are out of this context <ref name="open knowledge">https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>. Indeed two mains points are often mentioned in the literature:
+
As mentioned above, one of the peculiarities of RDM is that it is well suited for situations with a high degree of uncertainty. But the relevance of this method and not of the classical probabilistic risk analysis is really limited when we go beyond this context. <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>. Indeed, two main points are often mentioned in the literature:
  
−
- RDM requires to have a high computational power because generating and exploring many possible futures have an high computational cost. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. So if the uncertainties are not deep and can be handle by the probabilistic approach, RDM will use unnecessary computational resources.
+
- RDM requires high computing power because generating and exploring many possible futures has a high computational cost. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. Thus, if the uncertainties are not deep and can be handled by the probabilistic approach, RDM will use unnecessary computing resources.
  
−
- The output of RDM is to provide a robust strategy. This strategy is then not necessary an optimal solution for one specific future but a tradeoff which minimize the negative aspects on many possible futures. In a context of lower uncertainties, i.e in a context where statistical tools can rank futures by probability, decision makers would prefer to find an optimal solution for the most probable futures. Then the idea of finding the less bad strategy become way less interesting than the idea of finding the best strategy. And then RDM is no more the method to prefer.
+
- The result of RDM is to provide a robust strategy. This strategy is then not necessarily an optimal solution for a specific future, but a compromise that minimizes the negative aspects of many possible futures. In a context of low uncertainty, i.e. in a context where statistical tools can classify futures by probability, decision-makers will prefer to find an optimal solution for the most probable futures. The idea of finding the least bad strategy then becomes much less interesting than the idea of finding the best strategy. And then RDM is no longer the preferred method.
  
  
−
An other limitations to RDM is the complexity of the tools needed for creating and exploring the different scenarios <ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref> . First, those tools can be difficult to use and requires expert that should be part of the project. Second, it may appear that adapted scenarios generators don’t exist for very specific type of projects. In this case they must have to be implemented before to proceed to RDM. And this requires important human and financial resources.
+
Another limitation of RDM is the complexity of the tools needed to create and explore the different scenarios.<ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref> . First, these tools can be difficult to use and require an expert who should be part of the process. Second, it may seem that suitable scenario generators do not exist for very specific types of projects. In this case, they must be implemented before proceeding with RDM. And this requires significant human and financial resources.
  
 
== Annotated Bibliography==
 
== Annotated Bibliography==
  
−
Will be done later
+
'' Only the titles of the references are listed in this section. Links to the articles can be found in the references section.''.
 +
 
 +
*Decision Making under Deep Uncertainty
 +
This book offers a comprehensive examination of the approaches and tools for designing plans under deep uncertainty and their application. It identifies barriers and enablers for the use of the various approaches and tools in practice. More specifically, a section is dedicated to the analyse of Robust Decision Making.
 +
 
 +
*Project Management Institute, Inc. (PMI). (2019). Standard for Risk Management in Portfolios, Programs, and Projects. Project Management Institute, Inc. (PMI)
 +
This book is an update and expansion upon PMI's popular reference, The Practice Standard for Project Risk Management. This standard describes the concepts and definitions associated with risk management and highlights the essential
 +
components of risk management for integration into the various governance layers of portfolios, programs, and projects.
 +
 
 +
*Comparing Robust Decision-Making and Dynamic Adaptive Policy Pathways for model-based decision support under deep uncertainty
 +
This paper compares Robust Decision-Making (RDM) and Dynamic Adaptive Policy Pathways (DAPP)-which is another approach of designing plans under deep uncertainty. To do so, it first describes the two tools before underlining their strengths and weaknesses.
 +
 
 +
*Barriers and opportunities for robust decision making approaches to support climate change adaptation in the developing world
 +
This review paper examines the potential to expand the geographical and sectoral foci of RDM. The Strengths and Weaknesses, data requirement and limitations of RDM are discussed. The article examines potential entry points for RDM approaches through Environmental Impact Assessments and Strategic Environmental Assessments.
 +
 
 +
*Ensuring Robust Flood Risk Management in Ho Chi Minh City
 +
Ho Chi Minh City faces significant and growing flood risk. This study demonstrates how robust decision making can help Ho Chi Minh City develop integrated flood risk management strategies in the face of such deep uncertainty.
 +
 
 
== References ==
 
== References ==
 
<references />
 
<references />

Latest revision as of 22:27, 15 March 2022

Abstract: In some situations, project, program or portfolio management decisions must be made in the face of deep uncertainty. [1]. In this case, it is not possible to define probabilities for possible futures. Therefore, classical risk management tools are not adapted. [2]. Robust decision making (RDM) is one solution that can be used to properly account for this high level of uncertainty. Unlike traditional risk management tools, RDM helps decision makers choose not the optimal solution, but the solution (or set of solutions) that is the least bad compared to the set of possible futures. [1]. After presenting the general idea and main principles of RDM, this article aims to give a guideline so that readers can apply it to their project. This includes a step-by-step methodology and a very brief presentation of the type of tools needed. The relevance and limitations of RDM are discussed at the end of this article. In this article, if nothing is specified, the concepts used are defined according to the PMI standards.[3].


Contents

[edit] Presentation of the RDM

[edit] General Idea

As defined by Dr Alistair Hunt , Robust decision making is “ a methodology which aims to identify adaptation options or strategies which can perform well over a wider range of possible futures. “. [4]. The particularity of this method compared to classical risk management tools is that the goal is not to find the optimal solution to a problem but to create a robust solution that is satisfactory for a large number of potential futures.[5].

In other words, RDM is not about trying to find the most plausible future and adapting one's strategy to it. Instead, the essence of this method is to consider many potential future situations (see the following sections for more details) and evaluate strategies under all the different scenarios. Then, the most robust strategies are valued over those that are optimal in a specific situation. [1]. This method leads to choosing not the best but the least bad strategy. The general foundations and a step-by-step approach are described later in this article.

[edit] Fields of application

A situation of uncertainty and/or ambiguity implies having limited and imprecise knowledge of future events.[3]. When these uncertainties are deep, it becomes so difficult to accurately anticipate the future that traditional prediction or risk management tools cannot be used effectively. Indeed, a high level of uncertainty implies that it is not possible to put reliable statistics or probability distributions on possible future scenarios. And classical approaches need this according to project management standards[3].An alternative is to consider as many possible futures as possible in the evaluation process in order to reduce the risk that their solutions will not fit the actual future that will occur.


As already mentioned, one of the particularities of RDM is that it takes into account a very large number of scenarios in the decision-making process. It is therefore specifically adapted to the context of deep uncertainties. More precisely, these deep uncertainties can appear mainly in three situations: [1]:


- First, if the contextual uncertainties are profound. That is, if the characteristics of the context in which the project, program or portfolio will evolve cannot be precisely defined. This is the case, for example, when the prediction of the future requires climate modeling. Indeed, global warming will have impacts that are very difficult to describe and anticipate precisely. [6] .

- Second, uncertainties increase when “the set of policies has more rather than fewer degrees of freedom” [1]. This means that the more policies can change and influence your project, the more difficult it is to anticipate future situations.

- Finally, another source of uncertainty is the disagreement that may exist between different experts on the fields concerned by your projects. Indeed, if there is such a strong disagreement, it is impossible to establish a single reliable hypothesis. And so you have to consider many possible futures.


The fields of application that can be found are numerous. Here is a non-exhaustive list: [7] :defense, higher education, insurance, science and technology planning, counter-terrorism... In the current state of knowledge, RDM is very often used when a project, program or portfolio needs to include climate change in the decision. [4] .

[edit] RDM fondations

As mentioned in the literature [8] , RDM does not use computer models and data as a predictive tool to anticipate the best estimate of the future. Instead, RDM is based on the principle of exploratory modeling [9] . This means that a large number of possible futures with many different paths are generated to test the proposed strategies. The objective is not to rank the plausible scenarios but to explore as many as possible. Comparisons of the pre-decided strategies with a wide range of scenarios are then used to create a database of key performance indicators and outcomes. In this database, there are two main types of data.


The first is the set of data relevant to assessing whether a strategy is suitable for a specific future regarding the achievement of predefined goals. The data used to say whether a strategy is suitable or not for a situation really depends on that situation. There is an extensive literature that describes how to choose KPIs and gives examples of data that may be relevant. RDM can use these results and methodologies to include them in the process.

The second type of data is a description of the path used to generate the range of possible futures. That is, all the characteristics of one path that are relevant to differentiate it from another. This data is mainly generated by coupling algorithms capable of generating scenarios and software capable of exploring these scenarios. [5] - For more information, see the Steps by Steps section.


With this database, data scientists can then use analytics to identify common characteristics of futures that cause a strategy to fail or achieve its goals. [1]. They can also highlight the strengths and weaknesses of the strategy against a range of different scenarios. These first two steps can be repeated for a wide variety of input strategies and assumptions. Unlike many traditional risk assessment tools, the outcome of the analysis in the RDM process is not a predefined solution, but a set of key characteristics that identify under which assumptions, policies, and scenarios a strategy achieves its objectives.


When all of these characteristics are known, they can be passed on to decision makers who will perform a trade-off analysis to find a robust strategy. The strategies can then be modified or combined to find a robust one. That is to say “ones that meet multiple objectives over many scenarios “[1]. The process described above can be repeated several times to reveal new strategies and result in an even stronger decision.

Because RDM is useful for defining strategy, it should be used early in the project/program or portfolio life cycle.

[edit] Collaboration Human/machine

An important point of RDM is that it is based on human-machine collaboration. [1]. The strength of this method is that it combines the power of human decision making with the computational power of computers. Computers are used to generate and explore many different scenarios which are in turn used to stress the strategies devised by humans. When it comes to deep uncertainties with a large number of hypotheses to explore, computing power is absolutely necessary to cope with the mass of data to be processed. On the other hand, the decision-making part and the creation of potential trade-offs then rely solely on the decision-makers who can use the results of the previous computational steps as support. This mix between computer science and human intelligence implies the use of a different set of tools. As the various applications of the RDM method show, it only defines the general type of tool you need for each step. The exact tools to use depend on the context and must be defined for each new project. A very quick overview and advice on the type of tools to use are detailed in the "step by step" section.

RDM.drawio.png

[edit] Step by Step methodology

The purpose of this section is to provide a detailed guide for the reader to apply RDM to a new project, program or portfolio management situation. Each subsection presents a step in the process and briefly discusses the tools useful for that section. The order of presentation is chronological. The different steps are based on the methodology defined by Robert Lempert. [8]


Illustration of RDM process by Swann Roussillon and inspired by Robert Lempert [8]


[edit] Define Strategies

As with any project, program, or portfolio management situation, decision makers must define a set of strategies. These strategies will likely evolve over the course of the RDM process, but a baseline is needed to start the process. [10]. For a new project with a high level of uncertainty, it may be useful to define a set of strategies to test rather than a single one. Indeed, later on, the process may lead to a combination of different strategies to obtain a robust strategy.

[edit] Define the uncertainties as much as possible

One of the crucial aspects of using RDM correctly is to be as complete as possible in describing the uncertainties. This is a huge basis for simulation modeling to be unable to generate a large number of plausible solutions. The sources of uncertainties that need to be explored are primarily: [1]:

- Contextual uncertainties. These are basically all the physical things included in your scope that you are not able to predict. One of the main examples of this part is the set of strategies that must include climate or environmental simulations. Since it is not yet possible to have precise predictions, the level of uncertainty is very high.

- The policy uncertainties. This section includes all elements related to decision making that may influence the outcome of your strategies and that are beyond your control. For example, future taxes, future environmental laws, future local restrictions, etc. may fall into this category.

- Uncertainties related to assumptions. Experts may disagree on certain assumptions related to your strategies. If the disagreement is significant, these assumptions create uncertainties that you must consider.

[edit] Set objectives and metrics for the Strategies

Later in the process, the goal will be to subject the different strategies to a wide range of possible futures and evaluate whether or not they are appropriate. But a preliminary step is to define how these evaluations can be done. [8]. At the end of this step, it is crucial to be clear about what is to be measured, how it is to be measured, and what values define an acceptable outcome. Objectives and measures are used to assess the degree of success of the project, program or portfolio. A typical example might be the mayor of a city who has plans to build a new dam to protect the urban area from flooding in the context of global warming. He/she might decide on the following set of objectives and actions:

- The future dam should reduce property damage in the event of a flood. The total property damage in dollars could be defined as the metric. And a maximum acceptable level can be decided ($100,000 for example).

- The future dam should protect the citizens. A goal of zero people injured could be defined.

- The future dam must be affordable for the city. The total price of construction and maintenance in dollars could be defined as the measure. And a maximum of $10 million might be an acceptable outcome.

[edit] Define a plan for conducting the simulation modeling

The simulation model is a key factor in RDM. There are two things that must be considered during the simulation.[10]. First, the model must be able to generate a wide range of potential futures based on the uncertainties defined above. Second, the model must provide access to the different steps it creates in all potential futures. This is essential to allow humans to understand the impact of the assumptions made at each stage on the strategies. The first point requires a scenario generator and the second requires exploratory modeling software. It is then necessary to choose a software adapted to the domain and the scope of your project.

In addition, your simulations will generate a lot of data. In order to make it usable for the next steps, it is crucial to define how this data will be collected and stored. The knowledge and tools used in the field of data analysis can be applied here.

A final point that needs to be considered during the planning phase is the computational power required by the scenario generation. As mentioned in the collaboration Human/Machine part, this can require a lot of computing performance. And even if computers are making more and more progress, too many scenarios can exceed the capacities you have or create a very long waiting time. It is therefore necessary to anticipate these aspects and to reduce the complexity of scenario generations if necessary.

More information can be found in the references dedicated to exploratory modeling. [9]

[edit] Define a strategy of exploration and analyze of the data produced by the simulation modeling

When all the scenarios have been run, you will end up with a large database. And it is the exploration and analysis of this database that will create value for RDM. So it's important to have predefined strategies to leverage that data effectively. RDM's methodology helps you define what you are looking for. In fact, the goal is to identify and visualize the key factors in the scenarios that make your strategies work or not.[1] To do this, you can use classic data analysis and machine learning tools. You can also decide how you want to present your results. An important point to keep in mind is that you are not directly looking for an option to select. But you want to compare your different options. [1]

[edit] Run the simulation

This step consists only of applying the strategy defined in the action "Define a plan for conducting the simulation modeling". The expected result is the database mentioned above and a way to visualize the path through the different futures.

[edit] Analyze and characterize the database produced

This step consists of applying the methodology defined in the section "Define a strategy of exploration and analyze of the data produced by the simulation modeling". The expected result is an identification and visualization of the key factors in the scenarios for which your strategies do or do not work. The goal here is not to make a conclusion but to create a support that will help decision makers make the trade-off analysis described below.

[edit] Realize a tradeoff Analysis

It is in this step that the value of the RDM is highlighted. In this phase, decision makers should use the results of the previous step to compare the different strategies. Specifically, the strengths and weaknesses of each method can be compared by considering the different scenarios. The goal is not to directly choose a strategy. Instead, decision makers may want to see if two or more strategies can be combined to produce a more robust strategy.[1]. That is, a strategy that is better than the others when considering all possible futures.


[edit] Iterate the process if necessary

After analyzing the trade-offs, decision-makers can propose new strategies they would like to try or revisit some of the assumptions used to create the different scenarios. They can then start the whole process again, hoping that this will help them find a more robust solution. These new iterations take less time because all the methodologies and tools are the same as in the first iteration.

[edit] Limitations

As mentioned above, one of the peculiarities of RDM is that it is well suited for situations with a high degree of uncertainty. But the relevance of this method and not of the classical probabilistic risk analysis is really limited when we go beyond this context. [8]. Indeed, two main points are often mentioned in the literature:

- RDM requires high computing power because generating and exploring many possible futures has a high computational cost. [1]. Thus, if the uncertainties are not deep and can be handled by the probabilistic approach, RDM will use unnecessary computing resources.

- The result of RDM is to provide a robust strategy. This strategy is then not necessarily an optimal solution for a specific future, but a compromise that minimizes the negative aspects of many possible futures. In a context of low uncertainty, i.e. in a context where statistical tools can classify futures by probability, decision-makers will prefer to find an optimal solution for the most probable futures. The idea of finding the least bad strategy then becomes much less interesting than the idea of finding the best strategy. And then RDM is no longer the preferred method.


Another limitation of RDM is the complexity of the tools needed to create and explore the different scenarios.[9] . First, these tools can be difficult to use and require an expert who should be part of the process. Second, it may seem that suitable scenario generators do not exist for very specific types of projects. In this case, they must be implemented before proceeding with RDM. And this requires significant human and financial resources.

[edit] Annotated Bibliography

Only the titles of the references are listed in this section. Links to the articles can be found in the references section..

  • Decision Making under Deep Uncertainty

This book offers a comprehensive examination of the approaches and tools for designing plans under deep uncertainty and their application. It identifies barriers and enablers for the use of the various approaches and tools in practice. More specifically, a section is dedicated to the analyse of Robust Decision Making.

  • Project Management Institute, Inc. (PMI). (2019). Standard for Risk Management in Portfolios, Programs, and Projects. Project Management Institute, Inc. (PMI)

This book is an update and expansion upon PMI's popular reference, The Practice Standard for Project Risk Management. This standard describes the concepts and definitions associated with risk management and highlights the essential components of risk management for integration into the various governance layers of portfolios, programs, and projects.

  • Comparing Robust Decision-Making and Dynamic Adaptive Policy Pathways for model-based decision support under deep uncertainty

This paper compares Robust Decision-Making (RDM) and Dynamic Adaptive Policy Pathways (DAPP)-which is another approach of designing plans under deep uncertainty. To do so, it first describes the two tools before underlining their strengths and weaknesses.

  • Barriers and opportunities for robust decision making approaches to support climate change adaptation in the developing world

This review paper examines the potential to expand the geographical and sectoral foci of RDM. The Strengths and Weaknesses, data requirement and limitations of RDM are discussed. The article examines potential entry points for RDM approaches through Environmental Impact Assessments and Strategic Environmental Assessments.

  • Ensuring Robust Flood Risk Management in Ho Chi Minh City

Ho Chi Minh City faces significant and growing flood risk. This study demonstrates how robust decision making can help Ho Chi Minh City develop integrated flood risk management strategies in the face of such deep uncertainty.

[edit] References

  1. ↑ 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 1.12 Vincent A. W. J. Marchau Warren E. WalkerPieter J. T. M. Bloemen Steven W. Popper (2019)Decision Making under Deep Uncertainty https://doi.org/10.1007/978-3-030-05252-2
  2. ↑ Project Management Institute, Inc. (PMI). (2019). Standard for Risk Management in Portfolios, Programs, and Projects. Project Management Institute, Inc. (PMI). Retrieved from https://app.knovel.com/hotlink/toc/id:kpSRMPPP01/standard-risk-management/standard-risk-management
  3. ↑ 3.0 3.1 3.2 Project Management: A guide to the Project Management Body of Knowledge (PMBOK guide), 7th Edition (2021)
  4. ↑ 4.0 4.1 Dr Alistair Hunt ROBUST DECISION MAKING https://econadapt-toolbox.eu/robust-decision-making
  5. ↑ 5.0 5.1 Jan H. Kwakkel, Marjolijn Haasnoot, Warren E. Walker,Comparing Robust Decision-Making and Dynamic Adaptive Policy Pathways for model-based decision support under deep uncertainty,Environmental Modelling & Software, Volume 86, 2016, Pages 168-183, ISSN 1364-8152, https://doi.org/10.1016/j.envsoft.2016.09.017
  6. ↑ Bhave, Ajay Gajanan, Conway, Declan, Dessai, Suraje and Stainforth, David A. (2016) Barriers and opportunities for robust decision making approaches to support climate change adaptation in the developing world. Climate Risk Management, 14 . pp. 1-10. ISSN 2212-0963 DOI: 10.1016/j.crm.2016.09.004
  7. ↑ Robert Lempert, Steven Popper, Steve Bankes. ROBUST DECISIONMAKING (RDM) https://millennium-project.org/wp-content/uploads/2020/02/22-Robust-Decisionmaking.pdf
  8. ↑ 8.0 8.1 8.2 8.3 8.4 Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y
  9. ↑ 9.0 9.1 9.2 Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty.
  10. ↑ 10.0 10.1 Lempert, Robert J., Steven W. Popper, and Steven C. Bankes, Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis, Santa Monica, Calif.: RAND Corporation, MR-1626-RPC, 2003. As of March 05, 2022: https://www.rand.org/pubs/monograph_reports/MR1626.html
Retrieved from "http://wiki.doing-projects.org/index.php?title=Robust_Decision_Making_(RDM)&oldid=114958"
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox