Data-Driven Decision-Making under Uncertainty

Developed by Anton Reiling

There are several approaches to reach decisions under uncertainty under the assumption of reliable, fixed data. The degree of complexity and scope among the approaches differs considerably, which is why they should be selected according to the use case.

Starting with the simple approaches the Minimax and Maximax criterion consider the worst or best possible outcome of decision alternatives. The Hurwicz Criterion offers a mix between these two approaches depending on the predefined Optimism Parameter. The Minimax Regret Criterion considers the opportunity costs caused by choosing an alternative to reach a decision and the Laplace Criterion compares the expected values of the alternatives. However, due to simplifications and assumptions these methods are limited to rather simple problems with low complexity. This is because they lack the possiblity to assess risk, qualitative inputs and heterogenous probabilities for different outcomes.

For more complex problems, approaches like the Monte Carlo Simulation and Sensitivity Analysis can be used, while also requiring more effort to be conducted. However, they can also be used to support and analyze the classical approaches. In this article, the Laplace Criterion will be examined using a Monte Carlo Simulation and the critical Optimism Parameter of the Hurwicz Criterion is determined using a Sensitivity Analysis. There is also the concept of Robust Decision Making, which combines different tools to help to make better and more profound decisions.

All these approaches can be used within Project, Program and Portfolio Management, for example to compare two offers for Outsourcing Contracts within Project Management, which is illustrated later in this article. Depending on the decision-making approach different results will be yielded, which clarifies their respective focus and susceptibility to outliers.

The Decision-Making Process

Figure 1: Overview of the Decision-Making Process (based on Shaltry 2009)

In order to reach a sensible decision for a specific problem, certain steps have to be followed:

• Step 1: Define the problem: Specify the problem framework with the respective requirements regarding costs, time horizon, etc. It is important to reflect, which criterion should exactly be examined. This could be the for example the cost, profit, payback or Return on Investment. If non-economic issues should be compared, methods to quantify those apects have to be used. (p.116-117)
• Step 2: Establish the objectives: Describe the objectives for the given problem to create a foundation to compare later alternatives on depending on their degree of fulfillment and performance.
• Step 3: Identify alternatives: Work out possible alternatives that can satisfy the objectives and narrow them down to a shortlist that can be examined under reasonable effort later.
• Step 4: Identify uncertainties: Elaborate possible scenarios that can inlfuence the utility or cost of alternatives and connect them to the alternatives with their respective values.
• Step 5: Assess alternatives: Conduct research or apply decision-making tools to find out, which alternative satisfies the given objectives best.
• Step 6: Gather information and update the assessment: Gather data to verify if the preferred alternative from step 5 is really the best alternative of all the available ones. In case crucial data was overlooked, repeat step 5.
• Step 7: Decide and implement: Make a final decision regarding which alternative to choose and assign tasks and responsibilities to it
• Step 8: Monitor: Continously track the progress of the implementation to be able to plan adjustments if necessary (Good Decision-Making: The Key to Project Success, p.14-16)

Within the scope of this article, steps 5, 6 and 7 will be examined in further detail.

Differences between Decision-Making under Uncertainty and Risk

While decison-making under risk implies a certain probability for each possible outcome, decision-making under uncertainty assumes that these probabilites are unknown and thus each scenario is considered equally important. This subsequently leads to a higher inaccuracy of uncertainty problems compared to risk problems. An example for that would be a problem with 3 possible outcomes: Best case, most probable case and worst case. While for decision-making under risk the both extreme scenarios would be considered with a presumable low probability and thus weight, for decision-making under uncertainty all of the outcomes would be regarded in the same way, which means that in this case outliers have a considerably higher impact to the decision.

(p. 607)

Decision-Making Approaches

Minimax Criterion

The Minimax or Maximin approach shows a pessimistic conception: Thus, only the worst possible outcome of each alternatives is taken into accout and compared with the others. This means that all of the outcomes for the remaining scenarios remain unconsidered.

(ref: Development of fuzzy multi-criteria approach, p. 507)

Maximax Criterion

The Maximax or Minimin Criterion works opposite to the Minimax approach: Here, the best possible outcome is considered, which means that all other outcomes are ignored.

(ref: Development of fuzzy multi-criteria approach, p. 507)

Hurwicz Criterion

The Hurwicz Approach acts as a combination of the Minimax and the Maximax Criterion. Depending on the Optimism Parameter $\lambda \in [0;1]$ this approach can represent a rather optimistic or pessimistic attitude towards the outcomes of the examined alternatives. As extremes, $\lambda = 1$ would correspond to the Maximax Criterion and $\lambda = 0$ to the Minimax Criterion.

(ref: Development of fuzzy multi-criteria approach, p. 507)

Minimax Regret Criterion

The Minimax Regret Criterion tries to minimize the opportunity costs by choosing one alternative.

(ref: Development of fuzzy multi-criteria approach, p. 507)

Laplace Criterion

The Equal Likelihood Criterion or Laplace Criterion compares the expected value of each alternative with each other. Thereby, every outcome is considered which means that outliers that would act as extreme values for the Minimax or Maximax Criterion will have a considerably lower importance with the Laplace Criterion.

(ref: Development of fuzzy multi-criteria approach, p. 507)

Application to Outsourcing Contracts within Project Management

These approaches can also be applied to Project, Program and Portfolio Management. However, since they assume that the given data is certain per scenario and can be expressed as a number, the range of sensible application descreases. One example for utilizing these approaches are simple decisions regarding choosing suppliers for Outsourcing.

Let's take the following example to see how the different methods influence the decision which supplier to select: For a software project, parts of the front-end development are outsourced to a third-party supplier, which needs to be chosen according to their costs. Since mostly supplier and customer agree on a quotation with a cost estimate, the resulting costs can be higher or lower than the estimate, depending on incidents during the projects like delays, reworking or other unplanned costs. Of course, some additional costs are already included in the cost estimate to grant the supplier a buffer. This example will consider the following 5 scenarios:

• Scenario 1: None of the predicted additional costs actually occured (best case)
• Scenario 2: The actual costs are lower than the estimated costs, but not as low as in scenario 1
• Scenario 3: The actual costs match the estimated costs
• Scenario 4: The actual costs are higher than the estimated costs, but not as high as for scenario 5
• Scenario 5: On top of the estimated costs a high number of unexpected costs occurs which increases the overall costs drastically (worst case)

The following three suppliers can be chosen: Supplier A guarantees a fixed price of 100 monetary units, no matter which scenario will occur (i.e. how many unexpected costs will occur). Supplier B and C however, propose an estimated price of 100 monetary units. The actual price can vary depending on the scenario, which the following table illustrates:

Suppliers	S1	S2	S3	S4	S5
Supplier A	100	100	100	100	100
Supplier B	80	95	100	105	150
Supplier B	90	95	100	110	115

In order to determine which supplier to chose, the previously described approaches will be applied:

Beginning with the Minimax Criterion, the costs for scenario 5 are compared. It becomes clear that since Supplier A offers their service for a cost matching the estimated cost, it will also offer the lowest cost in the worst case scenario. Thus, Supplier A would be picked for this approach.

For the Maximax Criterion, the costs for scenario 1 are taken into account. Supplier B as the tenderer with the highest price deviations shows the lowest costs for the best case scenario, which it would be selected in this case. The fact that the costs for supplier A don't change with the scenarios shows the opposite effect to the Minimax Criterion: Since there are also no deviations towards lower prices, this supplier will not be the most attractive in case the best outcome occurs.

Figure 2: Sensitivity Analysis to determine the critical Optimism Parameter for the Hurwicz Criterion for supplier selection

The Hurwicz Criterion represents a tendency towards the Minimax or the Maximax Criterion, depending on the Optimism Parameter λ. To find out, how the risk attitude of the project manager affects the supplier decision, a sensitivity analysis will be conducted: λ is in this case the parameter that is modified to observe how the result changes accordingly. The value for each supplier can be represented as a linear function:

$H_A(\lambda) = 100$
$H_B(\lambda) = 80 \lambda + 150 \times (1 - \lambda) = -70 \lambda + 150$
$H_C(\lambda) = 90 \lambda + 115 \times (1 - \lambda) = -25 \lambda + 115$

To find the critical values for the Optimism Parameter, the intersections of these 3 functions have to be determined. Equating the functions leads to the following values for λ:

$\lambda _{A,B} = 0.714$
$\lambda _{A,C} = 0.600$
$\lambda _{B,C} = 0.778$

Figure 2 shows a graphical represenation of these functions. It can be seen that there are three sections in the diagram depending on the λ value. From the Minimax approach (λ=0) until the intersection of the function for Supplier A and C supplier A will be chosen, since it shows the lowest value in this region. After that, supplier C becomes more attractive and will thus be picked as a resulting decision. However, after the intersection between the functions for supplier B and C it is superseded by Supplier B, with Supplier B staying the most attractive option until the Maxmimax approach (λ=1). While it was already clear after the Minimax and the Maximax approach, how the project manager would decide in the extreme cases (λ=0 and λ=1), a new insight could be gained by examining the Hurwicz criterion using a sensitivity analysis: Picking Supplier C can also be an option if the Optimism parameter lies between 0.600 and 0.778, which indicates a required above-average risk attitude by the project manager.

The next approach will determine the optimal alternative according to the Minimax Regret principle: Since there are no parameters to alter, the result can directly be calculated. Supplier A will be chosen in this case, since it has the lowest opportunity costs (which means the highst opportunity income), followed by Supplier C and then B.

Figure 3: Histogram for a Monte Carlo Simulation for supplier selection

Lastly, the Laplace Criterion will be calculated. Calculating the expected values for the suppliers yields the lowest cost for Supplier A, followed by Supplier B and finally C. However, this average value does not give an overview about the distribution of the possible outcomes, which can be an additional information helping the project manager to make a decision. For example, two alternatives could have the same value for the Laplace criterion, while one shows a broad distribution, while the values for the outcomes of the other alternative lie closer together. In order to display the distribution for the supplier selection, a Monte Carlo Simulation will be conducted:

Since the scenario that will occur is uncertain, it can be modelled by generating random numbers. In this case, a random number from 1 to 5 was produced and then attributed to the respective costs for each supplier for the scenario matching the random number. This Simulation was run 1,000 times to ensure a comparably stable result. The results are illustrated in a histogram in figure 3.

Applying the described approaches to this problem yields the following results:

• Laplace Criterion: $\min(100,106)=100$ → Supplier A

It can be seen that due to the assymetric distribution towards higher average costs for Supplier B the Minimax Regret and the Laplace Criterion suggest to choose Supplier A. For the same reason the Hurwicz Criterion suggests to choose Supplier A for a higher Range in $\lambda$ than to choose Supplier B. Since we have costs independent from the scenario for Supplier A it stands to reason that the Minimax Criterion suggests Supplier A and the Maximax Criterion Supplier B.

It can be critisized that only the costs were considered in this example and not features like reliability or service quality. Also, all the outcomes were assumed to have an equal probablity of happening which is also rather unrealistic. Especially considering scenario 5 equally probable put a high weight on the 150 monetary units of Supplier B for this scenario. However, for minor supplier selection decisions these approaches provide results with low effort.

Comparing the simple approaches, the Minimax criterion should be used if it is necessary to set a lower bound for a decision. This can be the case for projects with heavy investments or tight deadlines (which must be met, albeit with additional costs to speed up the processes). The Maximax criterion on the other hand, can be used if the impact of the decision is minor and having a higher risk can be afforded.

Concluding, the selection of the approach for a problem depends on the risk the project manager wants to take.

(p. 607-608)

Limitations

.

Annotated bibliography

.