Quality Gates in Project Management

Revision as of 19:05, 28 February 2021

Figure 1: 5Ws and 1H of Quality Gates (own figure)

Abstract

Projects are temporary and unique endeavors and require active management of their integration, scope, schedule, cost, quality, resources, and communication according to the Project Management Institute. ^[1]

One tool to control project management is the Stage Gate approach that divides projects into stages with gates and tracks their progress and success. In the 1990s a modification to focus on especially schedule, cost, and quality was established known as Quality Gates (QG). A quality gate represents a checkpoint of predefined criteria set by the project manager to evaluate the current progress of a project. Hereby a focus lies on the three aspects of time and cost, both resource-related and quality (2). After assessing the specified benchmarks and standards, an evaluation of passing or failing the gate is made.

This article sets the quality gate model into the picture with other phase gate models and points out its differences, benefits, and limitations. More weight lies on the quality criteria and its control in project management and its lifecycles. Besides, how four main tenets set, assess, and complete gates together with intermediate milestones. ^[2] The article also explains the required steps a project manager needs to take in order to introduce quality gates for projects. A proposal of applying the quality gates by segmenting projects into the three stages of front-end, middle, and back-end gates for different project stages ^[2][3] is suggested with a detailed successive explanation. Finally, the concern of resource-intensity in quality gate processes (QGP) is discussed whereby more focus on scoping and lean management for projects is recommended. Also, the development from sequential towards iterative quality control is elaborated along with expanding the application from product-only towards service-oriented sectors.

Keeping Projects on Track with Quality

The technique of applying quality gates and their control process after each phase is a useful tool and is often applied by project managers or owners and process managers. What is does is designing, embedding and checking quality criteria after the development of a project reaches a new phase, e.g. from developing an IT software to its deployment. Before deploying the software a gatekeeper, for instance the project manager, assesses whether the quality of the previous phase satisfies the quality gate criteria. The process of defining, embedding and checking quality criteria is underlying to the entire project’s lifecycle.
Quality gates reach from initiation of a project until its final closing after deployment. Most applications take place in industrial circumstances where development takes place, such as in new product development (NPD), production (manufacturing and assembly) and also very commonly in IT software development. Often quality gates are the chosen tool to assess these processes because it enables practitioners to gain control over the quality development, its status and the current project’s state of the art in terms of quality. This is done by defining the quality criteria in the initiation and planning of a project, setting milestones and gates assessing the set criteria along the way.

Phase Gate Models

Quality gates are a comparable form of the stage gate model and also other phase gate models. It is suggested to separate projects in terms of quality gates into the four different categories of planning, design, development and deployment.
Also in the stage gate model projects are separated into phases because the outcome is often unknown and controlling can therefore take place after each phase. Setting gates after each phase, which must be passed before continuing to the next, allows control. Further, it enables early identification of potential risks and provides increased visibility.

Figure 2: Quality Gates after each Phase of a Project

According to PMI a project has the dimensions of integration, scope, schedule, cost, quality, resources, and communication and Cooper’s Stage Gate model intends to include all aspects in its model. Contrary, the Quality Gate model focuses specifically on the factors of time (schedule), cost and quality – all three of them are essential for a project’s quality outcome. In this way, it differs to the Stage Gate model. The gates are set timely independent and flexible, but in a sequential order – whereas its criteria to pass a gate are defined very specifically. More information about the stage gate model can be obtained by another DTU wiki called The Stage-Gate Model.

Quality gates as a project management tool have been established because practitioners felt that in our rapidly changing world and disrupting time there is less focus on quality due to a downside in order to increase time-to-market. Scholars therefore point out that “Quality gates […] are supposed to increase the confidence stakeholders have in a release” ^[4]. It is one of the solutions to introduce more quality control in project management surroundings.

Quality Management & Control

Figure 3: 2x2 Comparison Matrix of Project and Quality Management Tools (own figure)

Quality management consists of four basic pillars, its planning, assurance, control and improvement ^[5]. In project management, quality is of relevance in order to achieve the intended outcome and function of the endeavour. Already Cooper pointed out in his stage gate model in 1990 ^[6] that many newly launched products lack quality inspection processes. To achieve control over quality management in projects several tools evolved over the years and gained publicity.

Total quality management (TQM): "A management approach of an organisation centred on quality, based on the participation of all its members and aiming at long term success through customer satisfaction and benefits to all members of the organisation and society." ^[7] TQM aims for establishing an organisational culture where quality is the main criteria for success and requires cultural implementation. More information on the topic can be found in the DTU wiki Total Quality Management.

European Foundation for Quality Management (EFQM) model: “The EFQM Model is a globally-recognised management framework which allows organisations to achieve success by measuring where they are on the path towards transformation, helping them understand the gaps and possible solutions available, and empowering them to progress and significantly improve their organisation’s performance.” ^[8]. It helps to define purpose, create culture, forge strong leaders, transform organisation, foster agile practices, address unique organisational challenges, forecast the future. More information on the topic can be found in the DTU wiki under EFQM and the European Quality award.

(Lean) six sigma: The combination of the principles from lean principles/management and six sigma is called lean six sigma. Lean management focuses on continuous process improvement based on small, incremental changes with the overall goal to improve efficiency and quality. Together with six sigma it aims for improving processes and performance by narrowing resources and variations down to the minimum required. More information on the topic can be found in the DTU wiki under Lean Six Sigma.

Compared to quality management tools quality gates focus on specific quality aspects at specific times, the gates, in contrast to non-stop iterative quality management perspectives. It sets and later assesses quality criteria and intervenes when errors occur in the project’s development. Thus, it becomes obvious that quality gates are rather different to prevalent and common quality management and control techniques. Using these argumentations project managers can justify its presence and application for endeavours where quality-heaviness is key. Thus, quality gates is one of the solutions to control quality in the way of a phase gate model. It takes the advantages of quality management and combines it with the upsides of continuous project control by applying the project management techniques from phase gate processes.

Airbus Case Study

"A350 First Flight" by Matthieu Luna is licensed under CC BY-NC-ND 2.0

One major industrial player adopting the quality gate is Airbus. The multinational aerospace company incorporates quality gates approximately since 2015 and implements it as a quality management tool to handle and control the quality of supplier products. End-to-end (E2E) quality gates are nowadays an essential part of Airbus’ Quality Management System. ^[9]

Airbus implemented quality gates from the designs over parts up to a final aircraft and therefore considers them end-to-end gates. Every single time a project or product from a supplier is supposed to proceed to the next stage Airbus assesses the quality specifications and decides on the proceeding. They have implemented these gates because lack of quality concerns their Final Assembly Lines (FAL) and have found that these concerns especially arise from parts by external supplier. The quality requirements stated at the gates has even become part of their Customer Supplier Agreements and since then their supply chain efficiency improved since the amount of incomplete products decreased together with cost and visibility, cooperation and on-time delivery increased. Airbus and the supplier degree on targets and deliverables which they can measure, control and review at handover meetings.

In particular one example turned into a success story and show cased the strength quality gates have. Airbus had one supplier who supplied their production facility in St. Nazaire who always lacked documentation when delivering parts that were also in poor quality. This supplier and his low quality led to required manual corrections by Airbus in their FAL. Thus, customer and supplier integrated end-to-end quality gates for these particular sub-parts. The first time, the products came in the same quality again with no documentation, however Airbus refuse to accept the delivery. Causing delivery delay and financial inconveniences at the supplier, they reviewed the products and documents and delivered at the requested quality three days later. Since then there have been no more issues with the supplier and products and documentation come at the quality level asked for.

Application

Quality Gate Process

Figure 4: Quality Gate Process, own figure modified from Salger et alii ^[10]

Operating quality gates is an iterative management and an ongoing process. According to Salger et alii ^[10] it can be grouped into the conceptual and operative level and is described in the QGP. Figure 4 illustrates the iterative application of the processes.

Conceptual level: On the conceptual level quality gate management takes place. First, the initial definition of quality gates and its criteria are required. Further, coaching and support of the project in the three aspects (cost, time and quality) takes place. Afterwards the criteria defined for the quality gates is gathered and consolidated and results in the lessons learned. In this stage a gate is ready for the pass or fail assessment. Last but not least adjustment and controlling of the quality gates takes place when assessing the criteria and deciding to go or not go into the next project phase.

Operative level: The operative level describes the operations executed by the project manager along the project’s process. First is the initialisation where quality gates are set up for the project. Next is their active application of the initialized gates. The project manager is responsible for the active execution on the project’s deliverables. Last is the finalization where results are defined and lead to lessons learned.

Setup Advisory for Project Managers

To successfully implement and assess quality gates four tenets are of essential relevance which are performance standards and gate governance, criteria - benchmarks and standards, criteria control and tracking.

Performance standards and gate governance: To reach the quality gates, milestones have to be completed successfully otherwise a quality gate will very likely not be passed at the gate deadline.
These milestones already inhabit single criteria relevant for later quality gates.

Figure 5: Modern Gate Governance, own figure modified from Olechowski et alii ^[11]

An assessment of the project’s status is evaluated based on the actual performance of intermediate milestones. Each criteria must be met successfully by each party. When a quality gate is not met, the project cannot proceed to its successive milestones and gates. A project evaluation is required based on the governance decisions traditional gate decisions of go, kill, recycle and hold. In a go decision, the criteria is met and a quality gate is passed, thus the project proceeds to the next phase. The opposite, when gate criteria is not fulfilled, is the kill decision, it implies the failure of required quality criteria to proceed from phase X to phase X+1. The kill decision further implies the stop of the whole project. An option to work with insufficient and not met criteria at a quality gate but still not killing the whole project is the recycle decision. In this case the gatekeeper, e.g. the project manager, decides to revise the project and return to phase X and re-align. This decision is made when a project is still considered meaningful and worthwhile to rework, the criteria why the gate was not passed should not be of significance. The last option is the hold decision, which comes into play when a project is put on hold. This decision is useful when other project-dependent processes are not at the current stage to be sufficient for a gate go. A hold decision can end up in recycling or killing a project.

In more recent research there is however also other suggestions for realistic options at gates which this article proposes and strongly encourages project manager to apply. These gate decisions include the options of go, waiver, waiver with re-view, back-up plan, kill and delay ^[11]. The go option implies the same as in the traditional way and proceeds into the project’s next phase. The same traditional decisions from above apply to the kill and delay option and granting a waiver at a quality gate is similar to the recycle decision. But the waiver with re-view and back-up plan decisions are novel. A waiver with a re-view introduces a review of the project at an interim date that is before the next quality gate is set. This approach assesses and controls the lack of quality, but allows procedure to the next phase while not delaying the project. The back-up plan decision is made when a project is either uncertain in its outcome and an alternative is required. When evaluating a quality gate this back-up plan can be made until its risk, development and quality is assessed and a decision to either go or kill is finite, these plans are only temporary. Figure 5 represents the up-to-date available decisions for quality gates.

Criteria – benchmarks and standards: Quality gates and milestones require evaluation criteria. Benchmarks and standards are utilized in quality gates to make the project’s status measurable and assessable. However, criteria must not necessarily be numbers but can also be of Boolean (Yes/No) character. Some good examples for quality indicators that can be used throughout the project stages are given by PMI^[3]. They include the number of errors found on drawings, defects found in design reviews, requests to fix bugs, customer field failures, customer complaints and failures during test. Further, PMI mentions the rate at which requests to make fixes or changes come into the system and at which fixes or changes are being discharged. Lastly, the institute also suggests a criteria on warranty costs
These examples above represent the three axes of time, cost and quality respectively, provide good examples for project managers with less knowledge about the field, and provide a decent head start. However, it shall be noted that the suitable quality indicators differ from the application field, e.g. manufacturing, IT and finance.

Criteria control: Quality gates require implementation and control during the whole project life cycle. Every gate and several milestones should continuously assess and track the three aspects of project time, cost and quality. Every development project has planned and precisely scheduled quality gates, which is a project manager’s responsibility. Introducing a quality gate criteria checklist or matrix is beneficial in this case, an example is given here.

Tracking: Along the project development and thus its milestones and quality gates, a project manager is obliged to track its progress. When assessing the milestone and gate criteria, deductions in terms of potentially necessary resource shifts are necessary. Thus, the information requires not only evaluation in terms of whether the project can proceed or not, inspection for possibly needed changes and alterations of the current or upcoming milestone and gate criteria is mandatory.

Phasing Quality Gates and Objectives

Quality gates can, nevertheless the total quantity, be grouped into three main categories, which are described by the following. These groups provide orientation to phase quality gates and suggest topics to cover quality management and control successfully. They follow the suggestions of PMI ^[2].

Front-end Gates: For this stage investigation of the project, its internal commitment, investing issues (time and money) and general project design are determined. Relevant criteria for milestones and the gates for this topic must be set. Later in gates, their status is evaluated accordingly. During this phase stakeholder, agree to the scope of the project as well. This stage is also known as project planning or verification. At least one quality gate is suggested to cover these aspects.

Middle Gates: This phase covers implementation, integration and validation readiness. It usually requires first prototyping and actual product/service development processes. Here project requirements are tested against their potential solutions. When the milestones and gates are passed one can assume all stakeholder are satisfied with the project’s first solution. The phase covers the aspects of both solution and readiness acceptance. In other words, the project’s design and development take place here. At least two quality gates are suggested to cover the two main aspects of design (solution) and development (readiness).

Back-end Gates: Finally, the field trial, controlled introduction and the general availability (release) of the project are covered in the back-end gates. For passing this gate the project manager needs to assure all stakeholders agree the project is complete and the project’s execution is now compliant with all requirements and faces no more risk. This is thus the project’s deployment. The gate is also known as the Go-Live phase. It is suggested to have at least one quality gate covering the final deployment check.

Limitations and Challenges

The following section describes the quality gate tool's limitations by the means of practical examples. It points out that its concrete gate decisions can have a serious negative impact on a company's performance, the technique is less applicable to smaller projects and that endeavors with high uncertainty are less desirable application fields for quality gates.

Rewinding to the Airbus example from earlier, the company mentions that the quality gate model is not necessarily applicable to every situation, sometimes requires modification and is definitely not the solution to every problem a company might face. Airbus mentions they have to manufacture ten aircrafts per FLA per week and if a gate is not passed, a delay in terms of project hold (stop and fix) is not acceptable. They mention that only in 2% of their cases a stop and fix decision at a gate has been made ^[9], however this has already had a huge impact, how much in particular is not stated though. Nevertheless, these tough decisions that imply passing or failing a gate apply to all phase gate models and their processes. Phase gate models are not the perfect and or one and only solution to every faulty process. Quality gates are thus not an exemption but rather the rule when it comes to their decisional impacts.

Other scholars and practitioners mention that the project at which quality gates should be applied to must have a certain size and be meaningful. Thus, quality gates are not necessarily applicable to smaller projects, for instance in manufacturing. Studies from practice show from two cases, one at a job-shop manufacturing and another in clocked assembly line, both placed in industrial surroundings, quality gates were beneficial for processes when they had a certain capacity. There is little to no experience for production setups with smaller batch sizes. This is mainly due to the fact that the benefits of quality gates increase when the process happens more often since otherwise the continuous improvements and learnings from the technique cannot be implemented and become meaningless instead ^[12]. It is further mentioned that phase gate models accordingly find implementation in OEMs such as General Motors or Daimler. The researchers from the case studies summarize that “another limitation is that whereas the model seems suitable for productions with sufficient batch sizes in order to allow continuous update processes and learning, the benefit for small batch productions has to be investigated.” ^[12]

The application of quality gates requires an early on definition of criteria and benchmarks. Therefore, for projects with very high uncertainty of the outcome or a desired unplanned development quality gates are not the favourable technique to apply. Often the process of quality gates is thus implemented in software development where the outcome of a particular piece of software is known from the very beginning on ^[13]. FedEx for example started to implement quality gates to assess and further develop their enterprise IT production process. They have defined the early on targets in the very beginning and set requirements for sub-targets throughout the whole process. They have formed an internal department named the “Software Quality Assurance” team, which was responsible to develop key quality standards for software development lifecycles. This highlights again that the implementation of quality gates requires very early target setting and devotion by a team that aims for a particular goal. It is not beneficial to apply the process of quality gates into projects where deliverables are uncertain.

To summarize, it is important to understand that the decisions made at quality gates can have a huge impact. When specific requirements are demanded the no go or a stop and halt decision at a quality gate can e.g. affect a production company’s output and lead-time significantly. Project managers should wisely consider where the implication of quality gates is reasonable and applicable. Also, the quality gate process is not necessarily adoptable to smaller projects. Its relevance in consideration of the continuous improvement of processes becomes more meaningful for larger endeavours. Last but not least, the implementer should have a very clear and concise goal for his or her project. When the desirables and the outcome are not clear it is hard to define quality criteria, which is necessary to control at the quality gates and must be defined when setting up a project.

Tips and Tricks for Practitioners

For the successful application of Quality Gates, McKinsey & Company [40] and others have defined some best practices, which project managers are strongly encouraged to practice as well:

I. Pay special attention to the positioning of deliverables, milestones and gates The correct positioning of quality criteria especially in the development process is key for successful quality gate application. II. Apply E2E quality gate architecture Figure 6: Parallel Milestones in Different Funtional Groups to Quality Gate Representation (own figure) Ensure that quality gates cover the whole project lifecycle and do not only cover a particular part. The example from Airbus showed that their end-to-end architecture was the vital part to improve the supplier quality and decrease FLA downtime. III. It is suggested to call milestones according to the quality gate For instance, the milestones to achieve QG 1 should be named M1.1, M1.2 and the like. IV. Add actual names to the quality gate checklist and not only a department. Put the responsible people, e.g. the department’s manager (John Doe, Morten Menigmand, Erika Mustermann) into the column for deliverable responsible instead of the department (HR, IT, R&D). Otherwise, afterwards discussion can come up when a target is not met and who was responsible for it. V. The amount of four quality gates is advised for small to medium-sized projects However always assess the amount of quality gates needed and required according to the project lifecycle, its characteristics, scope and the company’s general procedure and quality standard. VI. You cannot manager what you cannot measure It is desirable to have measurable criteria, however this is not mandatory. It can also be in the Boolean form of yes or now. Nevertheless, all sub-criteria must be met before passing on in quality gates. VII. One quality gate for each transition from one project phase to another See section Phasing Quality Gates and Objectives VIII. Milestones can also be in parallel When different departments/functional teams are working together and its work is required for, achieving a common QG it is beneficial to have department-dependent milestones in parallel. Compare to Figure 5.

Annotated Bibliography

(1) PMI Guide - Project Management

(2) PMI Symposium – Quality Control with Quality Gates

(3) Quality Gates Concept in Battery Production

References

1. ↑ Project Management Institute, Inc.. (2017). Guide to the Project Management Body of Knowledge (PMBOK® Guide) (6th Edition). Project Management Institute, Inc. (PMI). Retrieved from https://app.knovel.com/hotlink/toc/id:kpGPMBKP02/guide-project-management/guide-project-management
2. ↑ ^{2.0 2.1 2.2} Achieving Total Project Quality Control Using The Quality Gate Method, Presented at the 1993 PMI Annual Symposium;, John M. Aaron, Cesare P. Bratta, D. Paul Smith
3. ↑ ^{3.0 3.1} Quality Management for Battery Production: A Quality Gate Concept Joscha Schnell, Gunther Reinhart Procedia CIRP 57 ( 2016 ) 568 – 573 49th CIRP Conference on Manufacturing Systems (CIRP-CMS 2016) Institute for Machine Tools and Industrial Management (iwb), Technical University of Munich (TUM)
4. ↑ G. Schermann, J. Cito, P. Leitner and H. C. Gall, "Towards quality gates in continuous delivery and deployment," 2016 IEEE 24th International Conference on Program Comprehension (ICPC), Austin, TX, USA, 2016, pp. 1-4, doi: 10.1109/ICPC.2016.7503737
5. ↑ Rose, Kenneth H. (July 2005). Project Quality Management: Why, What and How. Fort Lauderdale, Florida: J. Ross Publishing. p. 41. ISBN 978-1-932159-48-6
6. ↑ Cooper, Robert G. (1986). Winning at new products. Addison-Wesley. ISBN 0201136651
7. ↑ Pfeifer, Tilo (2002), Quality Management: Strategies, Methods, Techniques, Munich, Germany: Carl Hanser Verlag, p. 5, ISBN 9783446220034
8. ↑ https://www.efqm.org/index.php/efqm-model/
9. ↑ ^{9.0 9.1} Sciannamea, I. (2017). Quality Gates. In Supply Chain Integration Challenges in Commercial Aerospace (pp. 193-207). Springer, Cham.
10. ↑ ^{10.0 10.1} Salger, Frank & Bennicke, Marcel & Engels, Gregor & Lewerentz, Claus. (2008). Comprehensive Architecture Evaluation and Management in Large Software-Systems. 5281. 205-219. 10.1007/978-3-540-87879-7_13
11. ↑ ^{11.0 11.1} Olechowski, A., Eppinger, S., & Joglekar, N. (2017). Assessment of back-up plan, delay, and waiver options at project gate reviews. In DS 87-2 Proceedings of the 21st International Conference on Engineering Design (ICED 17) Vol 2: Design Processes, Design Organisation and Management, Vancouver, Canada, 21-25.08. 2017 (pp. 317-326).
12. ↑ ^{12.0 12.1} Wuest, T., Liu, A., Lu, S. C. Y., & Thoben, K. D. (2014). Application of the stage gate model in production supporting quality management. Procedia Cirp, 17, 32-37.
13. ↑ Ambartsoumian, V., Dhaliwal, J., Lee, E., Meservy, T., & Zhang, C. (2011). Implementing quality gates throughout the enterprise it production process. Journal of Information Technology Management, 22(1), 28-38.