Multi-criteria decision making (MCDM) for Project selection

Developed by Christian Lolk Thomsen

Multi-criteria decision making (MCDM) also called Multi-criteria decision analysis (MCDA), can be used as a decision support tool when making decisions with multiple criterions to evaluate simultaneously. In Portfolio management, ensuring that the resources are allocated optimally, is a complex task. No company have one single objective to maximize, but several conflicting objectives to manage simultaneously. MCDM can be used as a decision support tool and provide assistants in the process of making such decisions. MCDM is a good method for selecting which projects a company should initiate. This decision is based on a list of potential projects. All potential projects have assigned values or scores to all criterions. The objective function in the model could be to maximize the sum of scores in the selected projects subject to a number of constraints, such as available capacity (man hours, budget) and project diversity (at least one project in every category). The decision variables will be which projects to initiate and which not to. This list provides the decision makers with a starting point, and argumentation for making the decision.

Big idea

Companies today have a wide range of activities such as manufacturing, innovation, new product development, services and more. It is essential that companies take the right actions. Should the company penetrate a new market, develop new products and services for existing markets, start efficient projects in production and so on? Such activities have potential benefits and request different types and amount of resources. In essence the question is: Which projects or activities to initiate that maximizes the benefits with respect to the scar resources available? This question seems simple, but in reality it is very complex.

To manage a company efficiently we need to maximize our objectives. Increase profit, reduce costs, increase innovation, reduce lead times, decrease inventory levels etc. It is then up to management to identify the relevant objectives and which of them are most important. Furthermore, there are a limited number of resources available. These resources need to be allocated to different activities. Which projects will achieve the highest possible objective without using more resources than available? With a small number of objectives and few potential projects, this can normally be done by management just discussing the different choices. But if the potential projects are in hundreds, and with several objectives many different resource types this can simply not be done without a decision support tool.

Portfolio management

The PMI standard . propose scoring models as a good tool for evaluating portfolio components (Project proposals). The method is a MCDA. Each component is evaluated by a number of criterions. All components will be evaluated in the same way, with the same criterions and scales.

A similar scoring model is mentioned in the Management of Portfolio page 60 .

This indicates that the use of multiple criterions to evaluate a component, or projects, in a portfolio is a recommended method in the portfolio management literature. Multi-criteria are also listed as a key to success in prioritization in Management of Portfolios

Multi-criteria decision making (MCDM) theory

It is difficult to know exactly when MCDM was invented. The roots of today’s MCDM goes back to the early 1930s .

Single decision (Matrix)

When the objective is to make a single decision from a number of options, displaying the options and criterions in a matrix can be useful. Compute the weighted scores for each option and choose the options with the highest score. This method is very good for a single decision or when the number of options are limited. In the following YouTube video, the MCDA process is explained with choosing college as an example. https://www.youtube.com/watch?v=-JUaQcO4Dfo

Goal programming

Goal programming is a mathematical method for solving optimization problems with multiple objective functions. It can both be linear and non-linear. For a more detailed description of linear goal programming see the book Introduction to operations research page 332 The Method builds upon linear programming and converts the goals into a linear programming problem. This is well explained in the OR book at page 323-339.

Steps in MCDM

Decision processes can follow many different steps with different activities, based on what kind of problem the decision maker faces. In MCDM the following steps needs to be present. The list is not comprehensive and the order can change from company to company.

1. Identify criterions
2. Weighting of the criterions
3. Identify constraints
4. Collect data
5. Formulate mathematical model
6. Solve model
7. Evaluate results
8. Decision making

Identify criterions

It is important that the objectives are known. This step has a link to the strategy, and should be aligned with the overall goals. For example, if a research and development department has a goal of bringing new innovative products to the market place quickly, development time and degree of innovation could be very relevant criterions to include.

Weighting of criterions

When all criterions are identified, the weighting of them needs to be undertaken. The weighting is very essential for the results, and need to be done in a systematic and transparent way. One method is to just assign weights to the criterions. A major benefit from this technique is that it is simple and fast. The risks are though, that this method will not obtain the “correct” weights. Another possible method is Analytical Hierarchy process (AHP). AHP uses parallel comparison to compute the weights. This gives more reliable and robust weights. A disadvantage with AHP is that it can be time consuming.

Identify constraints

If there are no constraints, the solution will in many cases be to initiate all projects, because no scarce resources are limiting the number of projects that can be initiated. Constraints can be divided into two main categories: Resource- and Political constraints.

Resource constraints insure that the number of resources allocated does not exceeds the number of resources available.

Political constraints are rules that needs to be complied. A portfolio manager for dairy products in a supermarket chain could have a political constraint saying that at least 30% of all dairy products needs to be organic.

Collect data

To facilitate MCDM data is needed. If each potential projects should be evaluated by a number of criterions and constraints this information needs to be present. Values for all criterions and resource consumption are normally the information needed.

Formulate a mathematical model

Based on the weights and criterions the objective function will be formulated. The objective function is what we want to maximize or minimize eg. profit or cost. In a MCDM model we will have more than one criteria in the objective functions, and in some mathematical models we will have several objective functions. All identified constraints are formulated. Constraints can both be equalities, inequalities and functions.

Solve model

Based on the type of mathematical model a suitable technique for solving the problem is chosen. An example could be, if the objective function and all constraints are linear the problem can be solved by using the simplex method. The result will be the values of the decision variables. These variables could be as simple as 1 and 0 that show if a project should be initiated or not (1=yes, 0=no).

Evaluate results

In this step the decision makers will evaluate the computed results. A model cannot include all aspects of the real world and the results needs be understood hereafter. The result can also be used as argumentation.

Project selection process

The process of selecting which projects to initiate, evolves several steps. Archer and Ghasemzadeh (1999) present a framework for portfolio selection in new product development (NPD). This framework is specially developed for NPD portfolio, but project portfolio selection could follow similar steps. In figure 1 is an illustration of the framework.

Framework for Project Portfolio Selection

The framework starts with Pre-Screening. Based on the strategy a number of guidelines helps the decision makers in deciding what projects should go through to the Individual Project Analysis and witch should be discarded. Activities in this process can be feasibility analysis. After Pre-Screening, a number of projects will go to the Individual Projects Analysis. In the Individual Project Analysis phase, a more detailed analysis is made. Activities in this phase can be: Scoring, Risk assessment, return on investment etc. The screening phase removes projects that does not deliver pre-defined criterions such as minimum return on investment. The screening phase insures that unfeasible projects are not included in the Optimal Portfolio selection process.

Optimal Portfolio Selection. This process is where MCDM can be applied. 0-1 Integer Linear programming can be used to compute an “optimal” portfolio. A model for this is developed and tested in a simple case in section below. In this stage the goal is to select the projects that generates the highest befits with resource constraints. Based on the computed optimal portfolio, decision is made in the Portfolio Adjustment phase. The two phases are interrelated and will normally loop some times before the decisions can be made. After Portfolio Adjustment the execution starts.

Application

To show how MCDM can be applied in project selection, we will go through the steps and formulation a binary Integer programming model. The application is shown by computing the optimal portfolio for a made up ned product development portfolio.

case

The objective is to make a model to help a Portfolio manager in deciding which development projects to initiate to maximize the objectives. The company have five criterions identified based on the overall strategy in the company. The objectives are listed and explained in the following table.

The criterions are then weighted. NPV is the most important and is weighted 0.5. Innovation is the second most important and is weighted 0.3. Risk the least important of the three criterions and is rated 0.2.

Constraints

In this example is there two types of constraints. The most important constraint is resource constraint. All personal is divided into four different categories: Project manager, Non-technical Project employee, Technical Project employee and Expert. There are limitations on the number of resources that can be used for each of these four personal groups. Five project managers are available. 50 non-technical Project employees are available. Only five Technical Project employees available and 10 experts. There is also one political constraint that stats that Project 1 must be initiated.

Collect data

Based on the three criterions and the four resource data on how the different projects are performing in the criterions and the number of resources, each project consume of each resource type. These data can be seen in table below.

There are limitations on the number of resources available. The total amount of available resources can be seen in the table below.

Only 5 Project managers and 5 Technical Project employees are available. 50 Non-technical Project employees are available and 10 experts.

Formulate mathematical model

To include criterions with different scales the number needs to be normalized. This is done by dividing each score by the sum of all scores for that criteria, see equation below.

All normalized criterion scores can be seen in the figure below. C1 is NPV, C2 is Risk and C3 is Innovation.

The problem is formulated as a Binary Integer Programming (BIP) problem. This means that the decision variables all are binary (0 or 1). Initiate a project gives the value 1 and if a project should not be initiated the value will be zero.

Objective function:

Resource constraint: Resources must not exceed the available resources:

Political constraint:

Specific projects must be initiated. This is formulated by making a constraint that a specific x must be equal to 1. In this example must Project 1 be initiated. This is formulated as following: $x_1=1$

Each project can only be initiated ones and it is not possible to do a fraction of the project. This is formulated so the decision variables must be binary.

Solve model

There are many different programs that can solve a BIP problem. In this example will OpenSolver be used. OpenSolver is a spreadsheet based solver. This means that the problem can be formulated in excel and solved by the add-on program OpenSolver.

The spreadsheet used to solve the model can be seen in the figure below.

When the problem is solved a solution is displayed. 11 out of the 20 potential projects should be initiated. Not all resources are utilized fully. There is one Project Manager and 6 Non-technical employees not allocated. The objective function gives 0,678. If all projects were initiated the value would be 1. The total sum of NPV is 764.000$ and the average Risk and Innovation is 5,4 and 6,6 respectively. The solution should now be evaluated by management before making the final decision. The excel spreadsheet can be downloaded from: https://www.dropbox.com/s/kz9vir71knlyfr7/MCDM%20model.xlsx?dl=0

Comparing BIP with sorting method

A different approach to select which projects to initiate, could be by sorting the score from high to low, and then initiate projects until one or more resource groups are used up. The results of this method can be seen in the figure below.

Only 8 projects can be initiated before the expert resource are used up. The value of the objective function is 0,58 witch is lower than 0,68 from the BIP solved model.

More complex sorting systems can be used, but the results can never exceed the optimized method.

Project selection including time planning

The example with project selection illustrated how simple binary variables can be used to model the decision process, and help the decision makers in the selection process. One major limitation in this model is that it only considers a single time period. The model can be extended to include multiple time periods, but this will not be illustrated in this article. A model including multiple time periods can be seen in . The presented model is advanced, so knowledge in nonlinear programming is recommended before applying the model.

Supplier portfolio management

Demirtas and Ustrun, 2005 have developed a Multi-objective mixed integer linear programming (MOMILP) model for selecting suppliers with multiple criterions. The article is an example of how mathematical modelling can be used as a support tool for decision makers when other methods can be too time consuming. In the article AHP is also evaluated, but the number of pairwise comparisons increases exponential with the number of criterions.

Limitations

The proposed method has a number of limitations. Some limitations are results of assumptions underlining in linear programming: - Criterions known - Non stochastic - Resource known In LP it is assumed, that all variables are known. In this case it means, that the scores and requested resources for each project are assumed to be that precise value. In reality all these variables are associated with uncertainty. If a value of a constraint changes, it can result in the computed decisions not being optimal anymore and/or one or more constraint violated. The impact of changing variables can be investigated by conduction a sensitivity analysis.

Limitations of the model:

The model is static and only consists of a single time period. This limitation can be released by extending the model. The decision variables are not just if a project should be initiated, but also in what time period. Extending the model have the benefits of a more precis decision where time and planning of when to initiate witch projects are included, but it also adds complexity because the number of decision variables and constraints will increase. Instead of including the time horizon in the selection model, this planning step can also be modelled by itself.

Annotated bibliography

• Hillier, F. S. (2012). Introduction to operations research. Tata McGraw-Hill Education.
  • The book provides the theory and mathematics behind optimization in the Operations research field. The book both consists of linear and nonlinear programming. The book provides a very in-depth knowledge into the underlying algorithms that are used to solve different optimization problems.

• https://www.youtube.com/watch?v=o6SXYWPaHY0
  • An introduction to OpenSolver. It is recommended to see the video if you are new to spreadsheet based optimization. The video explains how to install and use the program by solving a short simple LP model.

• Chien, C. F. (2002). A portfolio-evaluation framework for selecting R&D projects. R&D Management, 32, 359-368.
  • The focus on the article is project selection for R&D projects. In a literature review relevant articles are listed and divided into themes. In the last part of the article the author shows a model developed by Hall et al. (1992) for choosing what projects to fund at the National Cancer Institute on the American Stop Smoking Intervention Study. The example is short and easy to understand and illustrates very well how political constrains can be used. In this case a range of geographical and prevalence quartile constrains.