Risk assessment using Failure mode and effects analysis

Author: Niels Peter Lindegaard - s194473

Abstract

This article explores the risk assessment method Failure mode and effect analysis (FMEA). Before diving into the specifics of the tool the article will explain the purpose of risk assessment, its placement in the risk management process as well as the purpose of the tool usage. This is followed by a thorough introduction to the FMEA tool and the associated Risk Priority Number (RPN) which leads to an explanation of why it is a qualitative method with quantitative traits. The article covers how to calculate the RPN, followed by a description of the 3 RPN parameters: severity, occurrence and detection. The parameters are rated on a scale from 1-10 (with 10 being the highest), however the scale of these numbers has been criticized in other articles (such as Garcia et al., 2005) therefore a standard scale for evaluation of the parameters is presented , along with an introduction to the idea of Fuzzy-FMEA. A thorough guide explaining the application of the FMEA follows as well a discussion of the importance of diversity in relation to the tool application and outcome of the RPN ranking. This is all to help the project manager get the most effective tool usage and thus why the articles ends with a discussion of limitations of the tool, based on when and how it is applied.

Indtroduction

FMEA is a tool used in project management to systematically identify and eliminate known or potential failures in complex systems. Thereby providing data and information for risk management decisions. The tool can be traced back till the 1940’s where it originated in the US military and were used for complex development projects. Originally it was developed to assess failures in equipment and systems but has also been applied in different forms to address projects risks. The tool identifies possible failure modes, causes and resulting effects in relation to either a product, process, or service. It then evaluates the causes and proposes countermeasures to overcome these effects. The tool is a qualitative method but has quantitative traits and provides the user with a numerical output value. The way of doing this is using the RPN, which in the aspect of risk assessment is a vital part of the FMEA. .

Risk Management and FMEA

Before we dive into a further explanation of the FMEA tool, we begin with the purpose of the tool. This starts with an overview of what risk management is, therefore, we must understand what risk means. The risk definition used in this article comes from the PRINCE2 guide on managing projects. It states risk as: An uncertain event or set of events that, should it occur, will have an effect on the achievement of objectives. A risk is measured by a combination of the probability of a perceived threat or opportunity occurring, and the magnitude of its impact on objectives. . PRINCE2 thereby defines risk as both threats and opportunities, however usually the focus is on the threats because of the negative impact they can have on projects. Part of the project managers task is therefore to be able to manage risk. This can be done using the risk management process, which involves assessing the threats and vulnerabilities in each project. The purpose of the FMEA method is to do exactly that and it fits into the risk management process as its purposes is to identify risk and analyze risk as well as propose plans for risk response.

The risk management process

Depending on what organization one consults the major steps to be taken in the process of analyzing and managing risks is somewhat inconsistent. ISO has one approach, Society for Risk Analysis (SRA) has another and finally the Project Management Institute (PMI) has a third one. This is the one used for the following sections and is shown in Figure 1..

Figure 1 - Risk management process with risk assessment steps marked with the blue area and risk analysis with the green area.

Figure 1 shows the seven steps in the risk management process, furthermore it outlines the sub-steps that make up the risk assessment process. Then again part of the risk assessment are the risk analysis which can be either quantitative or qualitative. Thereby the risk analysis is the backbone of the risk assessment which is the base for the whole risk management process. Following the risk management process the project manager can enforce countermeasures to reduce risks. A strong basis for making these decisions is vital and therefore choosing a good risk assessment tool plays an important role in helping the project manager successfully manage the risks.

Risk assessments

An example from the world of security management gives a pretty good explanation of what risk assessments is: “A risk assessment is a quantitative, qualitative, or hybrid assessment that seeks to determine the likelihood that an adversary will successfully exploit a vulnerability and the resulting impact (degree of consequence) to an asset.” . It is the very same process that is used to make a risk assessment in a project management environment. The only difference is that the adversaries are the things that can go wrong, and the asset is the project. So, to sum up, a risk assessment is one or more methods/tools that are used to discover what can go wrong and how big the impact will be if it happens.

Risk analysis

Multiple types of risk assessments are used for many different industries and purposes, but all of them build on either a quantitative, qualitative - or hybrid risk analysis, and all seek to identify risks and implement countermeasures. The qualitative approach relies heavily on the skills and knowledge of the people performing the analysis. It is very useful to identify and gain an understanding of individual risks and their probability of occurring. As it is not a number-based approach it therefore also leaves a subjective result, however it is the fastest of the methods to perform. The quantitative approach is evidence based and relies on numbers to evaluate and quantify project risks. This however is only possibly if the risks have already been identified using the qualitative approach. Therefore the qualitative analysis is alway performed first. As the quantitative methods are time consuming and resource intensive, they are usually only applied to the most critical risks. The output from the analysis will then be a number e.g., a monetary value, time delay or similar .

FMEA and Risk Assessment

The FMEA method is a qualitative risk assessment method . The tool consists of mostly subjective input and the ranking of the identified risks is dependent on the person using the tool, therefore it is qualitative. However, some places it is mistakenly described as quantitative, as the tool delivers a numerical output this might be where the confusion arises. As the qualitative analysis can be skipped the FMEA method thereby covers all main steps in the risk assessment process shown in Figure 1. In brief the steps and their purpose are:

1. Identify potential risks/failures including their causes and effects.
2. Qualitative risk analysis: Evaluate and prioritize identified risks/failure modes since not all risks are equally important
3. Plan risk response: Find and suggest actions that can reduce or eliminate the chance that the potential risk/failure occurs

The FMEA does this with a systematic approach to analyze potential failure modes, aimed at preventing the failures before they happen and thus reduce the risk of a project, product, service, or system failing. This is intended to be a preventive action and thus the method functions best when applied early on in a project. Furthermore, using a highly skilled and diverse team consisting of team members from many different areas with multidisciplinary educational backgrounds such as engineering, design, management, manufacturing etc. with a knowledge of the project type (be it product development, construction, system implementation etc.) yields the most effective FMEA that will identify corrective actions required to prevent failures/risks from reaching the project/customer/end user etc. depending on what case the FMEA is applied to.

The way of evaluating the risk is using the RPN number that is based on ranking the severity (the consequence of a failure happening), occurrence (the probability/frequency of a failures occurrence) and detection (the likelihood that a failure is detected before the impact of it happens) of each failure mode on a 1-10 scale and then calculating the RPN as:

$Risk Priority Number = Severity \cdot Occurrence \cdot Detection$

The ranking of each element is based on the subjectivity of the individual ranking it. To counter this a standard scale is presented in the application section. However, the newest studies from 2021 show that performing a fuzzy hierarchical FMEA (F-FMEA) can improve the reliability of the model, as it makes the method able to work with subjectivity in the data and evaluation process thus making the ranking of the individual risks more precise. . Using the fuzzy approach is however much more complex and therefore the standard approach is described first in the application section, however the idea of the fuzzy method is also described.

Application

How to apply the FMEA

Applying the FMEA is a proactive methodology that usually contains the following steps:

1. Assemble a team of carefully selected people with different job responsibilities and levels of experiences. The purpose is to bring a range of perspectives and experiences to the project.
2. Review the process steps (This could be stage gates in a project, steps in product development, parts of a service journey etc.)
3. Brainstorm and list potential failure modes.
4. List potential effects/consequences, causes and detection systems for each failure mode.
5. Assign severity, occurrence, and detection ratings for each effect.
6. Calculate the RPN for each effect.
7. Prioritize the failure modes using the RPN and recommend actions to reduce or eliminate the failure mode.

The result is a worksheet containing the RPN along with failure modes. In total it consists of 10 columns and should be filled out from left to right. An example of how the worksheet can look is shown in Figure 2.

Figure 2 – FMEA worksheet with explanations.

Furthermore, in Figure 2 there is an explanation on how to fill out each column. This might lead to questions about the scale used for the severity, occurrence, and detection rating. Usually, the rating is made using the scaling shown in Figure 3, 4 and 5.

Figure 3 - Description of the ranking of the severity ratings from 1 to 10 - based off

Figure 4 - Description of the ranking of the occurence ratings from 1 to 10 - based off

Figure 5 - Description of the ranking of the detection ratings from 1 to 10 - based off

As these scales are not absolute, they are affected by subjectivity. Thus, even though the RPN is a number it does not make the FMEA a quantitative method. The scaling definitions might also defer slightly depending on what sort of context the FMEA is applied to, whether it is a product development project, construction project or other sort of project. However as seen from the scaling in Figure 3, 4 and 5 the FMEA method is often used to analyze failures in product and system development projects.

If it were used to manage a project at "top-level" the effect scaling might simply be altered to cope with the nature of a projects progress instead e.g., possible threats to the projects schedule and cost. This must be decided upon at project startup. The definitions of the effect criteria could then instead depend on the total delay of the project schedule or budget overrun.

Diversity as a strength

The final RPN depends so much on the subjectivity of the team, that it is of outmost importance that the project manager knows how to assemble a skilled and diverse team, as this first step is vital for the outcome of the analysis. Therefore, when performing the first step of gathering the team the project manager should be aware of how a team can be diverse.

In general there are three different ways a team can be diverse: education diversity, skills diversity, and gender diversity. This can be divided into two types of diversity: surface-level diversity and deep level diversity. The first type is the biological and physical differences as age, gender, race and ethnicity and the latter type is psychological characteristics, such as cognitive abilities, attitudes, values, knowledge, and skills . For the FMEA both types are important but as the psychological characteristics is the diversity that includes knowledge and skill this is most important as this is attributes that can help the team to identify risks and failure modes in a project. The knowledge and skill of course must be relevant to the actual project, meaning the team should not just be assembled for the sake of diversity. However, one should not neglect the importance of biological and physical differences in diversity as these also have an impact on team cohesion, ideas, and performance.

When to apply FMEA

There are many cases in which a FMEA risk assessment will yield a beneficial outcome. There are however three main categories where FMEA’s are used.

1. process development
2. product development
3. service development

As process, product and service development usually takes form as projects the tool is a great help for the project manager to help analyze potential risks in the form of failures. The important thing here is however also where in the process it is applied. As mentioned FMEA is designed to be used before a change is being implemented. This means in the context of a project, the FMEA should be made in the early phases/initialization of the project. This makes sense as mistakes in the beginning of a project become vastly more difficult and costly to change in the later stages of the project. However, at the same time the information available is much larger in the end of a project. Therefore, once the FMEA risk assessment has been created in the beginning of the project it must be revisited and revised with new knowledge as the project moves along. Depending on the case the FMEA might also yield useful preventive actions after a project is done this is however a rare use case. This could for example be in a product development project where preventive actions are made should there be an issue when the product hits the market. Hopefully this should however already have been resolved using the FMEA in the early stages of the project.

Example of FMEA application

The FMEA can as mentioned be applied in a variety of cases. To make an example of how to use the tool a case is shown where a construction company is presented with the task of preparing a plot of land for the construction of a new apartment building. Their task is simply to prepare for the foundation of the building that is to be constructed in a bushy urban area. The following FMEA can then be made for the main tasks in this project.

Figure 6 – FMEA worksheet with example.

Ideally the construction company would make the FMEA using a team of people with a variety of skills and experiences within all the sub task that are to be performed, (such as civil engineers, construction workers and planners). This team would then sit together and brainstorm potential failures/risks in all steps of the process. Finally, when they are done, the RPN is calculated and it shows that the machinery maintenance should be prioritized as this has the highest RPN. As the example shows the process steps to include in the FMEA might be very trivial but it helps people across organizations, companies, or management teams to realize the risk that are present in a particular process step and respond to them.

Fuzzy FMEA

As the example shows the RPN is a multiplication of the severity, occurrence and detection rating which means the FMEA does not give attention to the importance of each input parameter. Furthermore, the assessment of each parameter is subjective and qualitative which sometimes can lead to the RPN number being inaccurate in terms of not assigning the largest value to the highest risk. Therefore, another more precise way to find the RPN is using the Fuzzy Logic approach, specifically the Mamdani method, to determine the values of S, O, and D thereby obtaining a more accurate FRPN (Fuzzy Risk Priority Number). However, one must know how to use the Fuzzy logic technique, a technique that is usually used to manage uncertainty in data. The method is based on mathematical principles in the form of degrees of membership functions. Fuzzy logic involves determining the degree of membership in a set. This means the member variable can not only be binary, where 0 represents a non-member and 1 represents a member, but a value ranging from 0 to 1 representing an increase in membership level. For the FMEA this means that the 1-10 ranking of the S, O, and D is divided into linguistic membership groups, it could be the five following: Very Low (VL), Low (L), Medium (M), High (H), and Very High (VH). This can be seen in Figure 7.

Figure 7 – Linguistic groups for S, O and D - based on

What the fuzzy logic then does is for example if a rank is 2 it is not just a member of one group but could have a 0.7 membership of the group Low and a 0.3 membership of the group Very Low. The Fuzzy logic method then uses something called a rule base that aggregates the membership levels of the 3 parameters into one final fuzzy output. The rule base essentially combines the fuzzy S, O and D values into 4 linguistic membership groups for risk:

1. Action is Unnecessary (R1)
2. Action is Suggested (R2)
3. Action is Needful (R3)
4. Action is Very Needful (R4)

Again, partial membership exists (e.g., a 0.2 membership of R1 and a 0.8 membership of R2) which is then defuzzied into the final FRPN value that is used to rank the risks. The explanation here is however just the main idea of F-FMEA, for a full explanation on how the Fuzzy logic and membership function math works see the referenced material as this process is quite complicated.

Limitations

For any tool or methodology there is a need to think critically of its usage and the same applies with the FMEA tool. Naturally there are more than one limitation, but one could argue that the biggest strength of the FMEA is also its biggest weakness: the team performing the analysis. They are the factor that has the largest influence on how well the risks/fault modes are identified, ranked, and acted upon. Therefore, they are also the tools biggest limitation. As mentioned, having a diverse team is important however one must also consider that diversity can reduce team performance by negatively affecting cohesion, decision-making quality, and members commitment to the team . The problem with having a bad team (e.g., one that are not skilled enough or not committed to the task because of diversity issues) is the risk of “falsely approving” the project that the FMEA is applied to. This could happen if the analysis team, believed to be experts, overlooks, or underestimates big risks in the project, leading to management having a false sense of security. They might then give the green light to projects that is going to fail. A project failing is of course not good for any company, however falsely approving a project could even lead to the project delivering bad products that are harmful or hazardous to the user. Therefore, keeping this potential flaw in mind when performing the analysis and assembling the team is important. To avoid this two independent teams could make the analysis simultaneously, this however doubles the resources required.

Other limitations of the FMEA method that should be considered when using it is:

1. Multiple failures: It is not possible to consider the relation between multiple failures that occur at once.
2. It is time consuming and people dependent: It might be very time consuming to list all potential failure modes and it relies heavily in the expertise of the people identifying them.
3. Updates required frequently to make sure all risks are evaluated
4. Potential waste of resources: Conversely, it might be that the analysis is too time consuming compared to what is gained from it and resources thereby are wasted.

As is also mentioned in the PRINCE2 guide the risk management approach should be appropriate for the size of the project. Therefore, the size scale and complexity of the project should be taken into consideration when applying the FMEA so that it does not create undue burden and bureaucracy. However, this applies both ways. If the FMEA is the basis of a risk assessment in a very large and expensive project the project manager could consider applying the Fuzzy-FMEA. This would remove some of the limitations of the normal FMEA regarding subjectivity in the Severity, Occurrence and Detection ratings and make the ranking of the largest risks, based on the RPN values even more precise. The cost of doing this however is increased resource spending as the Fuzzy-FMEA is a rather complicated method.

Annotated Bibliography

• Nuchpho, P. (2014). Risk Assessment in the Organization by Using FMEA Innovation: A Literature Review.'

- This article explains some of the basics of the FMEA method along with the standard way of calculating the RPN number and some of the limitations of the method. This includes some of the critique against the RPN number that this article finds in literature. The article concludes that fuzzy approaches are a new way to handle these limitations.

• Ben-Daya, M. (2009). Handbook of Maintenance Management and Engineering

- This book gives a thorough explanation on how to do a FMEA and in what cases it is applied. The book includes a full guide with useful examples on the traditional FMEA method and also provides standard rankings of the S, O and D. Furthermore it describes how the FMEA might be used and what value it provides. Some of the perspectives of the book is in relation to production systems but gives a good introduction to the FMEA and its history.

• Project Management Institute Inc (PMI), (2019). Standard for Risk Management in Portfolios, Programs, and Projects

- This is the standard for risk management and gives a more high-level explanation of the risk management process used for both project, program, and portfolio management. The standard is from the project management institute and provides the structure used for the risk management process in this article. Also provides good explanations on the difference between risk analysis, assessment and management.

• Damnjanovic, I. (2019). Project Risk Management Fundamentals. Springer

- Another book on managing risk that goes a bit more in depth than the PMI standard and also explains the difference between qualitative and quantitative analysis. The risk management process is not identical to the PMI as this book is based on the ISO standard, but the overall logic is the same. Very useful to dive deeper into the individual steps of the process.

• L. Pokoradi, S. Kocak and E. Toth-Laufer, Fuzzy Hierarchical Failure Mode and Effect Analysis,

- Article that provides state of the art research on improvements to the FMEA method using fuzzy approaches that removes some of the issues with the RPN number that has been criticized earlier. The article first gives an example from the automotive industry of the standard FMEA application and then the fuzzy FMEA. The article does not really dive into the Fuzzy logic math here. Therefore to read more about that see this article