The Decision Matrix

Innovation within technology is moving faster than ever. Companies are forced to keep innovate new products or improve exsisting products. Due to strict time frames and many different competitors on the technological market, decision-making within product-development/innovation have to be credible and solid. However, in the decision making process in product-development/innovation, low knowledge and many uncertainties are dominating. By using the so-called ”Decision Matrix” (DM), the former can be increased and the latter can be decreased and will be elaborated further. Hence, in product-development/innovation the DM is extremely relevant and may create more knowledge and turn these uncertainties into measurable quantities.

This wiki-article wishes to give a deeper insight and background description of the DM e.g. The Concept Screening Matrix and examine both the quantitative and qualitative approches (Stuart Pugh's approach). In addition the application procces will also be examined by focusing on innovation-projects, –programs and –portfolios including a ”Six Steps”-process description. Moreover, it will contain which parameters/criterion are key and appropriate in order to obtain an realistic and solid output when applying the DM, e.g. define the end-goal and stakeholders (customers), reflect on output and list the weight of parameters/criterion.

Challenges and difficulties when using the DM will also be examined focusing on how to chose the DM parameters/criterion and their weights and how to relate objectively to the outcome and how to use the DM as an iterative tool. Finally, suggestions are identified in order to accomodate and improve the challenges and difficulties and to identify for what the tool is applicable, for when the DM is applicable, how the DM is applicable and for who it is applicable and who can benefit from the DM.

The structure of the article is as follows:

- Concept desription of the tool, the DM (both quantitative and qualitative approaches

- Application in innovation-projects, –programs and –portfolios and choice of approriate parameters/criterion when using the DM

- Challenge and limitations in the DM

- Suggestions for improving the challenges and limitations of the DM

Introduction

The following wiki-article focus on the tool the Decision Matrix (DM). The DM is based a set of different candidates and criteria in order to make the right decision. The Decision Matrix (DM) is a tool which can be used in many relations. Both everyday-problems like “What should we have for dinner?” and the list is end-less. However, the DM can also be conducted in more complex situations as a management tool. E.g. in innovation and development projects, where problems like “Which product should we improve?” and in bid-situations where many criteria are influencing the final candidate of the bid.

Hence, the DM is a very relevant tool for both Programs, Projects and Portfolios since decision making, no matter the complexity, is an inevitable factor in all perspectives of achieving an end-goal.

The article will elaborate on the concept of the DM, application of the DM, challenges in relation to the DM, suggestions to improvements of the DM and finally a personal review of the tool is conducted.

Big Idea

The basic concept

The Decision Matrix (DM) is a decision-making tool invented by Stuart Pugh in 1990 [Source: Product Design and Development] and hence the method is also known as the Pugh-Method. Initially the Pugh-Method was based upon qualitative assessment using only “+” and “-” but has later on evolved to also use a quantitative assessment and hence a grading system. By using the DM a quick overview of different alternative solutions (candidates) is formed. The DM contains a list of candidates. Each of the candidates are judged and evaluated based on a row of different criteria. The candidates with the highest score is chosen as the final candidate and the decision is made. However, if two candidates score the same, a new iteration is performed. Moreover, if the external environment changes and influences the criteria, a new DM needs to be formed etc. Hence, the DM should be perceived as an iterative tool. Table 1 shows the basic DM without any ranking:

Table 1. The basic Decision-Matrix, with four candidates and four criteria (this may vary relating to the complexity), a total sum column showing the best candidate and an additional comment column.

To elaborate how the DM works a very simplified example has been established in table 2. The example concerns a decision everyone has to make during their life and hence can relate to:

'“What do I want to eat for breakfast?”

A list of optional candidates and criteria are listed in the DM. They are rated by using a quantitative grading system: 1-5, where 5 is the best.

Table 2. Example ”What do I want to eat for breakfast today?”

A lot of different questions may arise in relation to this example e.g. ”what if the price is more important to me than the taste”, “How do I come up with the different candidates?” etc. In the following section the process in choosing the final candidate is elaborated via Six Steps. Moreover, the rating is based on both objective criteria like the price but also on a subjective criterion like the taste. The challenge in using subjective criteria is elaborated in the section Challenges.

The Six Step process

Whenever the DM is conducted, a specific process should be followed. The process consists of six steps. The different steps vary in complexity and hence in duration.

1. Prepare the matrix by setting up all optional candidates and identify which criteria the decision should be based upon. From example, different types of breakfast and factors to rate i.e. price, taste. This step is properly the most complex step. The candidates and criteria are the basic when applying the DM. The DM follows the concept that the output is a reflection of the input. So when the input is based on solid research the output will be reliable. To identify the most appropriate criteria an elaboration is given further down under the Application section.
2. Rating of the candidates can be done either by using a quantitative or qualitative approach as in the given example from table 2 ”What do I want to eat for breakfast today?”. When many different candidates are occurring a qualitative approach is suitable. However, this is most suitable during the initial phase when you have to narrow down the candidates rather quickly. Moreover, a reference candidate is chosen and the other candidates are then rated compared to the reference candidate. This is also referred to as “The Concept Screening Matrix”. In this approach only +, 0 or – are assigned. This gives a quick overview of potential candidates to pursue and hence is suitable in an early stage [Source: Product Design and Development]. From table 3, the usual breakfast is chosen as reference and the other candidates are rated compared to the usual, if they are more + than the reference then eat that. The quantitative approach is based upon a scale like the one from table 2 and like the example, the candidate with the highest ranking is chosen. In some cases, a candidate who has a higher score for all criteria might occur and will naturally be chosen and decision process will end. This type of candidate is referred to as a “Dominant Candidate”. [Source: Lecture, 42457 Supply Chain Management, 13.09.2016]
3. Ranking each criterion can be assigned with a weight if they are of greater importance for the end-decision. From example, health is twice as important as the price and then health’s rating is multiplied with two.
4. Combine and improve candidates if this is possible and feasible.
5. Picking possible candidates (one or more) to pursue.
6. Reflect on the results and the process. The project team has to agree upon the final candidate, to achieve an optimal output.

Table 3. The Concept Screening Matrix where one candidate is chosen as reference and proceed with oatmeal and Skyr with fruit

Application

Initially when applying the DM for an innovation projects many uncertainties and low knowledge is occurring. It is thus necessary to identify who are the customer and key stakeholders and what are their needs. The reason for this is that output from the DM relies on the right set of criteria and those are based on these identified needs.

Different approaches to identify the needs are existing, however a general method also follow a six step process in order to select the right criteria:

• Define the scope of the candidates. This can be done by describing the stakeholders (both internal and external), their needs and goals, identify and define the given market and hence identify the benefit of a product and without getting a specific product concept. Moreover, it is important to carefully setup assumptions in order to comply the project scope.
• Gather raw data from customers. This is important in order to get a basic understanding of the customers. The raw data gathering can be handled by using either:
  • Interviews: Members from the innovation project discuss needs with the customer in their environment.
  • Focus groups: A member from the innovation project facilitates a discussion with 8-12 customers.
  • Observe an existing product in use: Members from the innovation project go out and an existing product in use because that can reveal what the customers’ needs really are.
• Interpret the raw data. After having gathered the raw it is important to clarify what the product has to do and hence use the identified needs as raw data. There are different approaches to express the needs:
  • What does the product have to do instead how the product might do it? By using this approach technological solutions do not stand in the way and limit the product development.
  • The level of needs should be on the same detail level as the raw data to secure information.
  • Use positive phrasing when expressing the needs, however this can be quite challenging. E.g. “I spend so much money on breakfast” is very negative whereas “I like cheap food” is more positive but somehow more difficult to phrase.
  • Use the need as attribute of the product.
  • Avoid must and should since those words may contribute to limiting the innovation project.
• Organize the need into hierarchy by writing down needs on a i.e. a post-it, eliminate redundant statements, group the post-its in the needs, review and edit the statements
• Establish the importance of the needs and compare to other needs.
• Reflect on the results.

Once all the customer’s needs are gathered and defined and the criteria have been listed the innovation project team can proceed in the decision making process.

The need for applying the DM occurs when several options can be conducted for one end-goal. The DM is expedient when there is no clear option and an objective approach is needed. However, a dominant candidate may occur when rating has been done and this candidate is superior in all criteria . The appropriate phases to conduct the DM is when a project team narrowing options and this is often the case in the early phases of an innovation project or when selecting and accommodate customers’ needs. Even though, this article focuses on innovation projects, the DM is applicable for other purposes.

Applicable areas

In the construction industry a tendering process is carried out before a construction site is launched. During turnkey contracts different interested contractors give an offer on the construction. Here the DM is very appropriate. There are different criteria that influence the final choice of the contractor. Often architecture, time-frame, operation and maintenance and of course cost are the primary criteria which a contractor is evaluated upon. Moreover, these criteria are also weighted since operation and maintenance may be considered of higher influence on the final decision compared to architecture.

The building owner or its consultants are responsible ranking and evaluate each candidate based on each criterion. The contractor with the highest score then win the tender. As mentioned earlier it may be possible to combine to different candidates when they have been evaluated. This could result in more contractors then being responsible for subject areas. A new decision-making process could then be launched in order to decide if this solution is more feasible. Hence, new criteria for this DM could be based upon planning (which order should the work be done), more internal challenges with handling the logistic of workers etc. This shows that using the DM is an iterative process and that the DM is a vivid tool. [Conversation with a Construction Manager from BaneDanmark, 12.06.2016]

Picture 1. A section how to evaluate in a tender process, taken BaneDanmark’ folder concerning “Information about tender, evaluation, negotiation and award”

As it can be read from picture the contract is awarded to the “most economically advantageous tender”, however the tendered prices and the technical solutions are both subcriteria which are weighted equally. The reasons could be that the prices and technical solutions often go hand in hand. [Lecture]

During procurement of products it is often supplied from an external source. To decide the right suppliers, the DM could be used. Moreover, if you are the supplier you could also use the DM in order to decide where the best location is. The criteria used could then be based upon criteria like customers, price, transportation and costs.

Figure 2. Example of a simple DM when a production has to choose a new site location. Due to no weight added to the criteria, it is not possible to determine which location is most suitable, solely assess on behalf of the table above.

[Lecture, 42457 Supply Chain Management, 13.09.2016]

Challenges

Even though the DM is a useful tool and can give a quick overview when many options are present, there are still challenges related to the tool.

As mentioned under Application, the DM is an iterative tool. This is especially the case when candidates are being combined or somehow improved. The iterative process also demands a new set of criteria for each iteration. The reason for this is that the criteria may change due to impacts from external environments. Moreover, the candidates should be check thoroughly in order to see if it is minor details that caused a low score and if it could easily be fixed.

The overall purpose with the DM is to be as objective as possible so decisions are not based upon subjective interests. However, to keep the DM objective, is one of its main challenges. The subjective interests could be of either a financial interest or a personal interest, like a reward, or if project team member have put a lot of effort into one of the candidates. By using the criteria selecting approach described under Application the risk for subjectivity should be decreased. However, there still might be a risk that the team will manipulate the stakeholders in order to conduct their own interest in the criteria selection even though the manipulation occurs unconsciously. Hence, a solution could be to use an external resource to identify the needs and interests of the relevant stakeholders and execute the six steps process from figure 1.

The main influences to having an objective and thorough DM is by having objective and clear criteria and have right weight conducted to the criteria. Hence if the input is invalid the output will reflect that – hence the principle; “Crap in, Crap out”. Three common traps during the identification of criteria are:

• Incorrect criteria: this often happens whenever a team member or the entire team uses their personal interests instead of using the identified criteria from the internal and external stakeholders
• Incomplete criteria: If all the relevant stakeholders have not been identified a lack of criteria will occur. Important perspectives and needs might be overlooked and hence the criteria will only focus on partial areas. A risk is also that the team/team member overlook important criteria deliberately to get their own interests carried out.
• Inadequate criteria: This is caused due to lack in criteria definition and may result in confusion and multiple interpretation scenarios. [Product Design and Development]
• Identical criteria: This is caused by two different criteria, which indirect indicates the same, but are weigthed differently. An example could be cost and quality. It is often desirable to have low cost product and high-quality products and both are the criteria are weighted high. However, these two often go hand in hand as good-quality products are often more expensive and, hence the criteria are not representative.

[Lecture, 42457 Supply Chain Management, 13.09.2016]

Choosing the right criteria is, as elaborated above, really challenging. If the wrong set of criteria are chosen, the DM will indeed has its limitation and the outcome will most likely result in poor decisions and will fail.

Improvements and recommendations

One of the key elements for a successful DM are the criteria, however, as previous mentioned these are highly depended internal and external stakeholders and their input to the criteria. It is also a fact that with complex problems the specified weights are highly recommended on order achieve the maximum of the DM. As for many cases, the recommended grading system should be performed by using a quantitative scale. This opens up for bigger differentiations during the decision-making-process.

However, as Pugh’s initial concept was to use a qualitative approach, only using “+” or “-“, this was not performed due to lack of knowledge to a quantitative grading. Pugh was convinced that the DM was to be used as a guideline and not as the resource actually taken decision. As he has stated: “The matrix does not make the decisions: it is simply a procedure for controlled convergence onto the best possible concept and is not composed for absolutes in the mathematical sense; the decisions remain with the user.” [No-nonsense Management] One of the main challenges in conducting an objective DM and hence take the most suitable decision lies within the criteria setting. This can be caused due to manipulation of stakeholder. The U.S. Small Business Administration states that often the wrong set of criteria is often when teams/companies get external advisors. To avoid manipulation in criteria it is advised by The U.S. Small Business Administration to always rely on own expertise and experience. As stated on their website: ”"Oftentimes, people have a tendency to place too much emphasis on what experts say. Remember, experts are only human and have their own set of biases and prejudices just like the rest of us." [1] Hence, it is key to listen to what the guts are telling you and way could be to vizualise the outcome of each alternative canditate and afterwards make a critical evaluation. Another improvement could also be to sure you have enough time to sit down and set of criteria and choose the right candidates. This might be costly however in the long run this will pay off knowing decisions was made on a solid basis. [2] The use of the DM can be used in different areas e.g. tendering process, procurement. However, the focus for this article has been on innovation projects. By using the DM during early stages innovation projects can build up portfolio and hence prioritize the ideas to different projects in order to succeed within a certain program. By using the DM allows a team use the ideas when internal and external circumstances are ready for new development. As highlighted in the abstract innovation projects are highly dominated by many uncertainties. This condition is of course not desirable and hence the goal is to have as few uncertainties as possible. An approach is to provide more knowledge in this early process stage which in the abstract also described as a threat in the beginning of a project since knowledge is often minimal on this stage. Providing more knowledge to a project can be done by performing through research, analysis of experiments and former experiences, and thereafter use the results to hopefully minimize the uncertainties by handle them as tangible risks. Hence the number of candidate can be narrowed down and the DM’s iterative approach may continue as knowledge will increase and new risks will be identified.

Annotated bibliography

Ulrich, Karl T.; Eppinger, Steven D. (2013) Product Design and Development

• Description of perspectives on marketing, design and manufacturing where theory and practical cases are explained in order to elaborate and support the theory

Lecture (13.09.2016) 42457 Supply Chain Management, DTU Engineering Management

• Lecture in the course 42457 Supply Chain Management regarding strategic network design, Location-allocation problem and hence tools and methods to manage the problem the DM

Conversation with a Construction Manager from BaneDanmark (12.06.2016)

• Unformal conversation with a construction Manager from BaneDanmark disussing the tender process for public construction projects with focus on the grading system and the process evolving that.

Sloma, Richard S. (2000), No-Nonsense Management

• A guide for managers including 70 principles which managers should follow if they wish to be successful and have a maximum performance.

The U.S. Small Business Administration () https://www.sba.gov/managing-business/leading-your-business/making-decisions

• Article formulating tips for Decision-Making, common Decision-Making mistakes and the importance of making time during a desicion process.

Brooks, Chad (01.04.2014) http://www.businessnewsdaily.com/6162-decision-making.html

• Article formulating Desicion-making tips and tools and explaning common mistakes.