Quality assurance methods in project management

From apppm
(Difference between revisions)
Jump to: navigation, search
(EFQM and the European Quality award)
 
(28 intermediate revisions by one user not shown)
Line 1: Line 1:
Within project management, quality is a key parameter governing the success of a firm. Quality is defined as “The degree to which a set of inherent characteristics fulfils requirements” (Winch 2010), the specific requirements to be fulfilled are themselves hard to define accurately and poses a problem in construction. An attempt to accurately define goals on how a building is well-built have been made by Vitruvius, which state that a building should consist of “Utilitas, Firmitas” and “Venustas”, which is translated to “Utility, Strength” and “Beauty”, utility meaning that a building should be usable, strength meaning it should be durable and Beauty meaning buildings should be aesthetically pleasing. Using these as guidance, quality could be interpreted as the interested parties’ ability to fulfill these three goals. After goals are specified, assurance that conformance of quality level will be obtained is beneficial for ensuring trust between stakeholders, and is therefore an important factor cultivating, successful, collaboration.
+
''Developed by Joachim Skjærup Bach''
  
 +
 +
Within project management, quality is a key parameter, governing the success of a firm. Quality is defined as “The degree to which a set of inherent characteristics fulfills requirements” <ref name="Winch">Winch, Graham M. 2010, “MANAGING CONSTRUCTION PROJECTS”, 2’th edition, WILEY-BLACKWELL, ISBN 978-1-4051-8457-1</ref>, the specific requirements to be fulfilled are themselves hard to define accurately which poses a problem in construction. An attempt to accurately quality has been made by Vitruvius, which stated that a building should consist of “Utilitas, Firmitas” and “Venustas”. This is translated to “Utility, Strength” and “Beauty” utility meaning that a building should be usable, strength meaning it should be durable and Beauty which is self-explanatory. Using these as guidance, quality performance could be interpreted as the interested parties’ ability to fulfill these three goals. After goals are specified, assurance that conformance of quality level will be obtained is beneficial for stakeholders, since it enhances trust. Quality assurance is therefore an important factor cultivating successful, collaboration.
 +
[[File:Quality_assurance.jpg|thumb|3200px| '''Figure 1:''' For image see: http://www.spgl.eu/images/img_construction.jpg]]
 
= Quality management systems =
 
= Quality management systems =
  
Line 9: Line 12:
 
* Total Quality Management (TQM)
 
* Total Quality Management (TQM)
  
Where quality control defines processes to be conducted in order to ensure that quality products and services are produced. This can be done in a variety of ways including statistical quality control, inspection and performance evaluation. These are all post construction control methods, other modern methods include preventative quality control measures, such as reevaluation of design and last planner management including all interested parties to an inclusive dialog, thereby minimizing errors due to bad communication. Quality control measures are crucial for ensuring project quality, for detailed information see [[Quality Control]]. The quality control measures can in most cases stand alone in a project, but in many cases quality assurance is a good complement.
+
Where quality control defines processes which ensure quality products and services. These processes could be inspection, stop-the-line, or performance evaluation. These are all post construction control methods. Other modern methods, include preventative quality control measures, such as reevaluation of design and last planner management. Last planner management is arranging meetings where all interested parties and stakeholders participating in an inclusive dialog, facilitating improved communication. Quality control measures are crucial for ensuring project quality, for detailed information see [[Quality Control]]. The quality control measures can, in most cases, stand alone in a project. However, quality assurance is an important contributor for cultivating successful collaboration. Total quality management is management of all quality aspects including QC and QA, for more information see [[ Total Quality Management]].
 
+
Total quality management is the absolute management of all quality aspects including QC and QA. It works as the overall management process.
+
  
The governing principles behind quality management systems are determined precisely in <ref name="ISO9001"/>. The following is a summery of those.
+
The governing principles behind quality management systems, are determined precisely in <ref name="ISO9001"/>. The following is a summery of those.
  
 
* Customer focus
 
* Customer focus
Line 23: Line 24:
 
* Relationship management of interested parties
 
* Relationship management of interested parties
  
These principles are general for all QMS systems regardless of size. With this being said, companies may adopt subsets within these different principles focusing more on specific purposes or explaining in detail aspects of some principles.
+
These principles are general for all QMS systems, regardless of size. With this being said, the list is incomplete and companies may adopt subsets within the above mentioned principles focusing more on specific purposes.
  
 
= Quality Assurance management =
 
= Quality Assurance management =
Quality assurance, (QA), management is the part of general quality management systems, (QMS), which revolves around guaranteeing quality requirements are met. The reason for QA, is to develop procedures and routines which manages quality control methods ensuring quality of the project.   
+
Quality assurance, (QA), management is the part of general quality management systems, (QMS), which revolves around guaranteeing quality requirements are met. The reason for QA, is to develop procedures and routines which manages QC methods ensuring quality of the project.   
  
 
== Why is quality assurance important? ==
 
== Why is quality assurance important? ==
[[File:Golf tdi.JPG|thumb|400px|Volkswagen Golf TDI 2.0 at a car exhibit, displaying the clean diesel brand]]
+
[[File:Golf tdi.JPG|thumb|400px|'''Figure 2:''' Volkswagen Golf TDI 2.0 at a car exhibit, displaying the clean diesel brand <ref> Figure from wikicommons, link https://commons.wikimedia.org/wiki/File:VW_Golf_TDI_Clean_Diesel_WAS_2010_8981.JPG </ref>]]
In modern society quality products are becoming more available to the public. This increases the customer expectations with regards to quality and performance. Requirements for performance can have a variety of characteristics, such as environmental, resource or safety related. The capability of organizations to fulfill these requirements, is challenged. When failure to conform to quality requirements is present, organizations reputations and trust is decreased. These two factors, customer trust, and organizational reputation plays an increasingly important role for organizations. To give an example, consider the Volkswagen emission scandal in 2015.
+
In modern society quality products are becoming more available to the public. This increases the customer expectations with regards to quality and performance. Requirements for performance can have a variety of characteristics which is related to environmental, resource or safety. The capability of organizations to fulfill these requirements, is challenged. When organizations fail to conform to quality requirements, reputations and trust are decreased. These two factors, customer trust, and organizational reputation, plays an increasingly important role in a globalized market. To give an example, consider the Volkswagen emission scandal in 2015.
  
=== Volkswagen case ===
+
=== Volkswagen emission scandal ===
Volkswagen is a German car brand which produces cars, and was in a period of years awarded multiple environmental awards for their low CO2 emission diesel cars. When the Environmental Protection Agency ran experiments on diesel cars from Volkswagen, they couldn’t replicate the emission rates Volkswagen was informing, leading to investigators demanding an explanation. This lead to the discovery of software implemented in the cars, designed to report lower false emission rates of the cars, such that they seem more environmentally friendly. When the EPA released their results documenting the systematic cheat to the public, a drop 20 % the first day, and 17 % the day after in Volkswagen shares was observed (cite Volkswagen article), showing just how important customer trust, reputation and quality conformance is for the company.   
+
Volkswagen is a German car brand which produces cars, and was in a period of years awarded multiple environmental quality awards for their low CO2 emission diesel cars. When the Environmental Protection Agency (EPA) ran experiments on diesel cars from Volkswagen, they couldn’t replicate the emission rates Volkswagen was informing leading to investigators demanding an explanation. This lead to the discovery of scam software which was implemented in cars, designed to report fake emission rates, such that they seem more environmentally friendly. When EPA released their results documenting the systematic cheat to the public, a drop 20 % the first day, and 17 % the day after in Volkswagen shares was observed <ref name="Volkswagen"> Website article: “Volkswagen the scandal explained” 10. December 2015, BBC News, Russel Hotten, http://www.bbc.com/news/business-34324772 seen 08/06/2017. </ref>, showing just how important customer trust, reputation and quality conformance is for the company.   
Customer complaints are therefore critical to take care of and organizations often hire people just to answer customer complaints. This practice of continually evaluating quality management systems, through customer satisfaction, is one of the aspects identified in ISO-9001:2015 as a mandatory procedure for assessing performance and detecting faults or risks inherent in the company. Suggestions of performance evaluation models are given In section XX.
+
In a similar manner, customer claims are important to assess in order to avoid bad publicity. In order to improve quality practice it is possible to continually assess the quality management systems, through customer satisfaction surveys. This is one of the aspects identified in ISO-9001:2015 as a mandatory procedure for assessing performance and detecting faults or risks inherent in the company.
The previous example of Volkswagen shows how important quality conformance is for companies, and why effective quality assurance methods are necessary to ensure customer requirements and legislative requirements to quality are fulfilled by organizations.
+
  
 
== Three main types of systems ==  
 
== Three main types of systems ==  
[[File:CCM Logo.jpg|thumb|400px|Certified Construction Manager or short CCM is the only certification accredited by the American National Standard Institute, (ANSI) under ISO-17024]]
+
[[File:CCM Logo.jpg|thumb|400px|'''Figure 3:''' Certified Construction Manager or short CCM is the only certification accredited by the American National Standard Institute, (ANSI) under ISO-17024]]
 
Quality assurance systems can be characterized into three main categories.
 
Quality assurance systems can be characterized into three main categories.
 
* First-party system
 
* First-party system
 
* Second-party system
 
* Second-party system
 
* Third-party system
 
* Third-party system
Which are outlined in the following.
+
These are outlined in the following.
 
=== First-party system ===
 
=== First-party system ===
First party systems, are systems where company themselves are responsible for quality conformance alone. This way only the company itself regulates and optimizes the quality of produced services and products, without intervention of external organizations.
+
First-party systems, are systems where companies themselves are responsible for quality conformance. This way, only the company itself regulates and optimizes the quality of produced services and products, without intervention of external organizations. This provides an conflict of interests since companies want to present themselves in the most favorable way which can influence it's quality control protocols, and skew results.
  
 
=== Second-party system ===
 
=== Second-party system ===
Second-party systems, where suppliers follow the strongest marketplace standard which hasn’t been accepted through an independent organization such as ISO. An example of a second party system is the advent of software such as windows, which becomes a de-facto standard for other companies to follow, even though this service’s quality hasn’t been approved by an independent organization.
+
Second-party systems is where suppliers follow the strongest marketplace standard which hasn’t been accepted through an independent organization such as ISO. An example of a second party system is the advent of software such as windows, which becomes a de-facto standard for other companies to follow, even though this service quality hasn’t been approved by an independent organization.
  
 
=== Third-party system ===
 
=== Third-party system ===
Third-party systems are when quality of services or products are controlled by independent organizations, thereby ensuring no conflict of interest. This approach of conformance with quality requirements, is determined best practice. <ref name="Winch">Winch, Graham M. 2010, “MANAGING CONSTRUCTION PROJECTS”, 2’th edition, WILEY-BLACKWELL, ISBN 978-1-4051-8457-1</ref>. An example of a third-party system is the ISO system, which certifies companies which fulfill criteria regarding quality. Another company which certifies construction managers is the Construction Management Association of America, (CMAA), see <ref name="CMM"> http://cmaanet.org/about-cmaa, website explaining in detail CMAA's goals and ethics, seen 12/06/2017 </ref>. They can certify construction managers which fulfill requirements outlined in American National Standards Institute, (ANSI) which is following the international ISO:17024 standard <ref name="CMM"> </ref>. A more detailed description of the ISO-9001:2015 standard is outlined in the following.
+
Third-party systems are when quality of services or products are controlled by independent organizations, thereby ensuring no conflict of interest. This approach of conformance with quality requirements, is determined best practice <ref name="Winch"></ref>. An example of a third-party system is the ISO system, which certifies companies fulfilling international recognized criteria regarding quality. Another organization which certifies construction managers is the Construction Management Association of America, (CMAA). For more information see; <ref name="CMM"> http://cmaanet.org/about-cmaa, website explaining in detail CMAA's goals and ethics, seen 12/06/2017 </ref>. They certify construction managers which fulfill requirements outlined in the American National Standards Institute, (ANSI) which is following the international ISO:17024 standard <ref name="CMM"> </ref>. A more detailed description of the ISO-9001:2015 standard is outlined in the following.
  
 
== ISO-9001:2015, a third-party system ==
 
== ISO-9001:2015, a third-party system ==
  
Since the transition into the post-modern construction era, where projects are "unique" and planned continuously throughout the project life cycle using systems such as the Last Plannar system <ref name="Rolf Simonsen"> "Lean Construction & Last Planner System", S. Rolf, DTU, January 10, 2017, internal link: https://cn.inside.dtu.dk/cnnet/filesharing/download/cc9fa450-000e-41b0-a037-c6d6bdc16f15 </ref>, focus on quality requirements plays an increasing role. The third party independent organization ISO, has therefore been established with the goal of setting quality standards. Companies can be granted a certification from ISO-standards if they fulfill these requirements. In order to fully understand terms and definitions used in DS/EN ISO-9001:2015, the reader is directed to definition standard DS/EN ISO-9000:2015 <ref name="ISO9000"> DS/EN ISO 9000:2015, “Quality management systems – Fundamentals and vocabulary (ISO 9000:2015)”,  Technical Committee ISO/TC 176, approved by CEN, Göteborg plads 1 DK-2150 Nordhavn. </ref>. If a company has fulfilled all regulatory specifications defined in <ref name="ISO9001"> DS/EN ISO 9001:2015, “Quality management systems – Requirements (ISO 9001:2015)”, Technical Committee ISO/TC 176, approved by CEN, Göteborg plads 1 DK-2150 Nordhavn. </ref>, then it can be certified. This certification means that the customer is assured that requirements set, were met thereby providing assurance that quality is up to standard.  
+
Since the transition into the post-modern construction era, where projects are "unique" and planned continuously throughout the project life cycle. Systems such as the Last Plannar system <ref name="Rolf Simonsen"> "Lean Construction & Last Planner System", S. Rolf, DTU, January 10, 2017, internal link: https://cn.inside.dtu.dk/cnnet/filesharing/download/cc9fa450-000e-41b0-a037-c6d6bdc16f15 </ref> arises which focuses on quality requirements. This is because quality conformance play an increasingly important role.  
  
This way of assuring quality services and products through certification, is effective because of the independence of the ISO-cooperation. Which otherwise would be deemed untrustworthy, an attribute devastating for the entire building process. The independence therefore cultivates trust between customer and contractor, and is the main reason that third-party systems are deemed best practice.
+
The third party independent organization ISO, has therefore been established with the goal of setting quality standards. Companies can be granted a certification from ISO-standards if they fulfill these requirements. These requirements are determined in various standards. One of the defining standards in quality project management is DS/EN ISO-9001:2015. To fully understand terms and definitions of the full article, the reader is directed to definition standard DS/EN ISO-9000:2015 <ref name="ISO9000"> DS/EN ISO 9000:2015, “Quality management systems – Fundamentals and vocabulary (ISO 9000:2015)”,  Technical Committee ISO/TC 176, approved by CEN, Göteborg plads 1 DK-2150 Nordhavn. </ref>. If a company has fulfilled all regulatory specifications defined in <ref name="ISO9001"> DS/EN ISO 9001:2015, “Quality management systems – Requirements (ISO 9001:2015)”, Technical Committee ISO/TC 176, approved by CEN, Göteborg plads 1 DK-2150 Nordhavn. </ref>, then it can be certified. The specification of requirements are determined using other standards or where those aren't available, best practices.
 +
 
 +
This way of assuring quality services and products through certification, is effective because of the independence of the ISO-cooperation. Which otherwise would be deemed untrustworthy, an attribute devastating for the entire building process. The independence cultivates trust between customer and contractor, and is the main reason that third-party systems are deemed best practice.
  
 
== Commissioning as quality assurance ==
 
== Commissioning as quality assurance ==
Commissioning originates from quality assurance of ships. Commissioning is conducted on all mayor ships before venturing out on the open sea, in order to ensure quality standards are met. In recent years, commissioning have been tested on building projects as an attempt to lower costs arising from late delivery, budget overruns, fault correcting, etc. Furthermore, it goals was to optimize customer satisfaction levels, together with more accurate budget predictions.  
+
[[File:Commissioning.PNG|thumb|400px|'''Figure 4:''' Illustrative sketch of commissioning effects and maintenance costs with and without commissioning. x-axis shows years after the building has been taken into use, while the y-axis shows maintenance costs. The light blue color indicates regular costs without performing commissioning, dark blue without commissioning but with some usage and light pink with commissioning. Sketch taken from <ref name="Commissioning"> "Commissioning process", Vaerdibyg.dk 2013, in collaboration with organizations; Dansk byggeri, Bat-kartellet, Bygherre foreningen, Danske Arkitekt Virksomheder, DI Byg, Foreningen af Rådgivende ingeniører, TEKNIQ installatørernes organisation, link: http://www.vaerdibyg.dk/index.php?option=com_docman&task=doc_view&gid=156&Itemid= </ref>. ]]
  
The process of commissioning is fairly simple, it evolves gathering main stakeholders from all parts of a specific construction including e.g. engineers, entrepreneurs, carpenters, builders, users, architects, etc. together continuously throughout the project phases on a weekly basis, such that dialog and collaboration increases. This ensures initial quality requirements are met and unforeseen problems are dealt with, before continuing construction, thereby minimizing needs for rebuilds.  
+
Commissioning originates from quality assurance of ships. Commissioning is conducted on all mayor ships before venturing out on the open sea, in order to ensure quality standards are met. In recent years, commissioning have been tested on building projects as an attempt to lower costs arising from late delivery, budget overruns, fault correcting, etc. It was also implemented to optimize customer satisfaction levels, together with more accurate budget predictions.  
  
Commissioning takes time and costs money. This should be recognized and taking into account in the tender phase and preferably written into suppliers contracts thereby ensuring all parties will attend meetings. This is crucial, since in many cases, systems are cross-correlated with each other, plumbing is correlated with heating which is connected with walls and isolation, and so-on and so-forth. For the procedure to be effective all relevant parties must attend the meetings and communicate otherwise quality will deteriorate.
+
The process of commissioning is fairly simple, it evolves gathering main stakeholders from all parts of a specific construction including e.g. engineers, entrepreneurs, carpenters, builders, users, architects, etc. together throughout the project phases on a weekly basis going through design and completed work. As a side benefit, dialog and collaboration will also increase as an effect of the increased communication. This ensures initial quality requirements are met and unforeseen problems are dealt with, before continuing construction. This minimizes the need for expensive rebuilds.
 +
 
 +
Commissioning takes time and costs money. This should be recognized and taking into account in the tender phase and preferably written into suppliers contracts. This ensures that all parties will attend the meetings. This is crucial, since systems are cross-correlated with each other, plumbing is correlated with heating which is connected with walls and isolation, and so-on and so-forth. For the procedure to be effective, all relevant parties must attend the meetings and communicate.
 +
 
 +
= Culture of Improvement =
 +
The following provides an outline of two similar methods for facilitating a culture of improvement. By providing the framework for a culture of improvement, companies can identify problems and rank themselves against each other. This provides the incentive for improvement of performance. As a result, customers are provided with better services and companies are continually improving themselves. 
  
 
== EFQM and the European Quality award ==
 
== EFQM and the European Quality award ==
[[File:Basicmodel.PNG|thumb|400px|European Foundation of Quality Management, the basic model, illustrating focus points for achieving high quality performance, figure from <ref name="efqm.org"> The basic EFQM excellence model, seen on http://www.efqm.org/efqm-model/efqm-model-in-action/strategy on 21/06/2017. </ref>]]
+
[[File:Basicmodel.PNG|thumb|400px|'''Figure 5:''' European Foundation of Quality Management, the basic model, illustrating focus points for achieving high quality performance, figure from <ref name="efqm.org"> The basic EFQM excellence model, seen on http://www.efqm.org/efqm-model/efqm-model-in-action/strategy on 21/06/2017. </ref>]]
 +
 
 +
The European Quality Award is an award granted to companies, organizations, firms and other professionals excelling in producing high quality services or products. The award is one part of the European Foundation for Quality Management, which is an organization providing the framework for establishing criteria suitable for determining which organizations is worthy of the award. There are multiple more or less detailed versions of this framework, for illustrative purposes the following outlines the freely available basic model, see figure 5, which provides an insight into how quality assurance through certification, can be achieved.
 +
 
 +
The basic model containing the major points organizations should focus on, can be seen on figure 5. The figure is an excerpt from <ref name="efqm.org"> </ref>, which is a website containing detailed information about the points displayed. Furthermore, spreadsheets and questionnaires are also available for assessing the individual clients performance. This kind of performance assessment, can help identify problems, thus initiating the first steps of improvement needed in order to obtain certification. This way of helping organizations through transparent questionnaires and other methodologies, improves the integrity of the awards and certifications. It is an important aspect, since trust in the certification is the single most important aspect.
 +
 
 +
== Project excellence ==
 +
[[File:EFQM.PNG|thumb|400px|'''Figure 6:''' International Project management association, (IPMA) basic model for ranking organizations against each other. Taken from <ref name="Winch"> </ref> page 335, figure 12.8]]
 +
The figure to the right shows a schematic overview of the project excellence award. Organizations are primarily judged on two different aspects, project management and results. The following outlines an explanation of the weighing and what aspects are covered.
 +
 
 +
Project management consists of five different aspects of enablers. Enablers are resources which drive the project forwards towards fulfilling customer needs. This can be leaders of a team, suppliers, collaborations, etc. All important for reaching project goals. The weighing on the enablers are different, with processes and project objectives scoring the highest. This indicates, that processes and reaching project objectives are a critical part of performance, and organizations therefore need to direct attention towards these two for ensuring best performance.
  
The European Quality Award is an award granted to companies, organizations, firms and other professionals excelling in producing high quality services or products. The award is one part of the European Foundation for Quality Management, which is an organization providing the framework for establishing criteria suitable for determining which organizations is worthy of the award. The basic model containing the major points organizations should focus on, can be seen on figure XX, the figure is an excerpt from <ref name="efqm.org"> </ref>, which is a website containing detailed information about the points displayed. Furthermore, spreadsheets and questionnaires are also available for assessing the individual clients performance. This kind of performance assessment, can help identify problems, thus initiating the first steps of improvement towards certification.
+
The other part which is evaluated is Project Results. Here the weighing is further skewed, with almost the entire scoring dependent upon "Key performance and project results" and "Customer Results", covering aspects like; efficiency, conformance, customer satisfaction, project costs. This is then influencing the management aspects and visa versa. Organizations are scored which cultivates improvement throughout the organization.  
  
== International Project Management Association ==
+
The project excellence model can be used on the building industry by assessing the performance of all contributing parties including, engineers, contractors, entrepreneurs, architects and suppliers. All of them can be scored against similar firms, thus ensuring each company is performing to the fullest of their capabilities. This is valuable for the customers receiving the final product and is also preventative in the formation of cartels, since the ranking is performed by an independent organization.
[[File:EFQM.PNG|thumb|400px|Internation Project management association, (IPMA) basic model for ranking organizations against each other. Taken from <ref name="Winch"> </ref> page 335, figure 12.8]]
+
  
 
= Advantages =
 
= Advantages =
The main advantage of general quality assurance is increased trust between parties that best practices are upheld. The increased trust, facilitates improved efficiency throughout the building process and enables collaborative efforts for improving overall performance. This can result in a positive feedback loop where quality is continuously improved together with trust. Some positive effects resulting from the increased quality are; a minimization of the number of disputes leading to lawsuits, improved customer satisfaction, improved reputation of contractors, entrepreneurs, suppliers, etc. and reduction of costs associated with poor performance. Quality assurance is therefore an important aspect of quality management and should be put under focus, especially during early stages of construction. Advantages of the previously outlined methods are given in the following sections.  
+
The main advantage of general quality assurance is increased trust between parties, that best practices are upheld. The increased trust, facilitates improved efficiency throughout the building process and enables collaborative efforts for improving overall performance. This can result in a positive feedback loop where quality is continuously improved together with trust. Some positive effects resulting from the increased quality are; a minimization of disputes leading to lawsuits, improved customer satisfaction, improved reputation of contractors, entrepreneurs, suppliers, etc. and reduction of costs associated with poor performance. Quality assurance is therefore an important aspect of quality management and should be put under focus, especially during early stages of construction. Advantages of the previously outlined methods are given in the following sections.  
  
 
== Third-party system ==
 
== Third-party system ==
Advantages of the third party system, include among others, an independent assurance that regulations and agreements are upheld and in conformance with international and national standards. Another advantage is that it allows customers to easily identify companies renown for providing quality services with certainty that independent organizations without interests involved have done the certification.
+
Advantages of the third party system include, an independent assurance that international/national regulations and practices are upheld, it enables customers to easily identify companies renown for providing quality services. These are valuable, in an industry where the general public opinion is negative.
 +
 
 +
The advantages of the EFQM framework is that it enables firms to identify weaknesses and strengths. It enables companies to enter into a positive feedback loop of continual improvement. This is necessary in a world where what is excellent today is mediocre tomorrow. It provide tools for companies to tackle the complexity of large projects by dividing the overall tasks into smaller boxes such that performance is easily quantified and compared with other projects.
 +
 
 +
Advantages of the project excellence model is that it is a quantitative way of ranking companies against one another, thereby cultivating a competitive environment. The model can be used on different scales, as a big scale tool of evaluating large projects, or as a small scale optimization of a particular supplier.
  
 
= Challenges =
 
= Challenges =
  
Quality Assurance is difficult in post-modern construction for a variety of reasons including; communication, project uniqueness, social status, interest crossings, etc. The main challenge is thorough communication between partners and collaboration, since studies have shown that poor communication is a leading contributor to project failures which can result in reworks, or potentially accidents and loss of life.
+
Quality Assurance is difficult in post-modern construction for a variety of reasons including; communication, project uniqueness, social status, interest interference, etc. The main challenge is thorough communication between partners and collaboration. Communication also plays a critical role in safety conformance.  
  
Another important challenge for especially commissioning is economy, as the customer needs to be convinced that resources should be spent on cultivating a commissioning process.
+
== Commissioning ==
 +
Another important challenge for especially commissioning is economy, as the customer needs to be convinced that resources should be spent on cultivating a commissioning process, even thou immediate rewards aren't readily apparent. This is an aspect, which the industry is improving on, since the publishing of studies comparing maintenance costs of buildings, with and without commissioning. Lastly, an important challenge of commissioning is mediating interested parties when non-conformance is observed. This involves assuring stakeholders, that the commissioning agent is not a quality police, but rather, a partner helping with achieving peak performance.  
  
Similarly, the last planner system also faces economic challenges since meetings are costly, also with respect to time. Especially the time aspect of the last planner system proves problematic, since suppliers, contractors and other interested parties often work on multiple projects in parallel on different geographic locations, complicating the situation. In order to combat this, it could be beneficial to incorporate in contracts the requirement to attend these meetings with economic penalties if members are not attending, they would therefore have an incentive other than pure quality.
+
== Last Planner System ==
 +
The last planner system faces economic challenges since meetings are costly, also with respect to time. Especially this time aspect of the last planner system, proves problematic, since suppliers, contractors and other interested parties often work on multiple projects in parallel. This complicates the logistics of arranging meetings. In order to combat this, one could make meetings an obligatory part of the collaboration with economic penalties to members not attending. Thus providing another incentive to attend meetings rather than pure quality performance.
 +
 
 +
== Third-party system ==
 +
 
 +
The main problems with third-party certification systems, are that of clearly defining quality standards which on one hand, is broad enough to be applicable to all projects, but at the same time quantifiable such that performance can be evaluated. Inconsistencies, miscommunication, vague specification and costly procedures, are all factors complicating certification of companies. Furthermore, when companies which are deemed trustworthy defaults and get bad publicity, e.g. Pihl and Søn, then the entire foundation of customer trust is undermined.
  
 
= Conclusion =
 
= Conclusion =
 +
Quality assurance is an important contributor to total quality management systems. It ensures a trustworthy bond between client and organization that quality is in conformance with expectations. Reasons for why quality assurance is important have been outlined and used in an example case of the Volkswagen emission scandal. Building trust and creating a culture of improvement are some of the main benefits.
 +
 +
In modern time, commissioning has been tried out as an quality assurance method. The effects of commissioning, are a lowering of maintenance costs which is a substantial part of the lifetime cost of a building. On the down side, commission is an expensive procedure without any immediate benefits. This makes it difficult to sell. Furthermore, commissioning takes time and requires a collaboration between interested parties which can prove difficult to reach when multiple projects are worked on simultaneously.
 +
 +
The certification system is outlined and the third-party system is deemed best practice, because of the independent judgement. The ISO standard 9001:2015 have been outlined which provides the general guidelines for companies to follow. Clear specification of quality requirements prove difficult and the assurance requires quality control and methods for conducting this have been linked to. In similar manner, the american CCM certification, EFQM and IPMA frameworks have been explained. The systems provided the framework which cultivates a competitive culture of improvement within the company. This could result in positive feedback loops where quality and trust are continually increased.
  
 
= Abbreviations =
 
= Abbreviations =
Line 100: Line 131:
 
* TQM – Total Quality Management
 
* TQM – Total Quality Management
 
* EFQM - European Foundation of Quality Management
 
* EFQM - European Foundation of Quality Management
 +
* EPA - Environmental Protection Agency
 +
* IPMA - Internation Project Management Association
  
 
= Annotated Bibliography =
 
= Annotated Bibliography =
Line 114: Line 147:
  
 
''http://cmaanet.org/about-cmaa, website explaining in detail CMAA's goals and ethics, seen 12/06/2017'' - Website explaining the principle behind CCM certification of people, together with an detailing of the responsible organization "CMAA's" goals, ethics and code of conduct.
 
''http://cmaanet.org/about-cmaa, website explaining in detail CMAA's goals and ethics, seen 12/06/2017'' - Website explaining the principle behind CCM certification of people, together with an detailing of the responsible organization "CMAA's" goals, ethics and code of conduct.
 +
 +
''"Commissioning process", Vaerdibyg.dk 2013, in collaboration with organizations; Dansk byggeri, Bat-kartellet, Bygherre foreningen, Danske Arkitekt Virksomheder, DI Byg, Foreningen af Rådgivende ingeniører, TEKNIQ installatørernes organisation, link: http://www.vaerdibyg.dk/index.php?option=com_docman&task=doc_view&gid=156&Itemid='' - A folder made for practitioners shortly describing how and why commission should be used. The folder consists of two main chapters, first chapter is explaining the commissioning process in detail, while the second chapter explains the reasons why commissioning should be conducted. The folder is in Danish, but an English version should be available in the near future.
  
 
= References =
 
= References =
 
<references/>
 
<references/>

Latest revision as of 14:30, 19 November 2018

Developed by Joachim Skjærup Bach


Within project management, quality is a key parameter, governing the success of a firm. Quality is defined as “The degree to which a set of inherent characteristics fulfills requirements” [1], the specific requirements to be fulfilled are themselves hard to define accurately which poses a problem in construction. An attempt to accurately quality has been made by Vitruvius, which stated that a building should consist of “Utilitas, Firmitas” and “Venustas”. This is translated to “Utility, Strength” and “Beauty” utility meaning that a building should be usable, strength meaning it should be durable and Beauty which is self-explanatory. Using these as guidance, quality performance could be interpreted as the interested parties’ ability to fulfill these three goals. After goals are specified, assurance that conformance of quality level will be obtained is beneficial for stakeholders, since it enhances trust. Quality assurance is therefore an important factor cultivating successful, collaboration.

Contents

[edit] Quality management systems

Quality management is comprised of three main parts:

  • Quality Control (QC)
  • Quality Assurance (QA)
  • Total Quality Management (TQM)

Where quality control defines processes which ensure quality products and services. These processes could be inspection, stop-the-line, or performance evaluation. These are all post construction control methods. Other modern methods, include preventative quality control measures, such as reevaluation of design and last planner management. Last planner management is arranging meetings where all interested parties and stakeholders participating in an inclusive dialog, facilitating improved communication. Quality control measures are crucial for ensuring project quality, for detailed information see Quality Control. The quality control measures can, in most cases, stand alone in a project. However, quality assurance is an important contributor for cultivating successful collaboration. Total quality management is management of all quality aspects including QC and QA, for more information see [[ Total Quality Management]].

The governing principles behind quality management systems, are determined precisely in [2]. The following is a summery of those.

  • Customer focus
  • Leadership
  • Engagement of people
  • Process approach
  • Improvement
  • Evidence-based decision making
  • Relationship management of interested parties

These principles are general for all QMS systems, regardless of size. With this being said, the list is incomplete and companies may adopt subsets within the above mentioned principles focusing more on specific purposes.

[edit] Quality Assurance management

Quality assurance, (QA), management is the part of general quality management systems, (QMS), which revolves around guaranteeing quality requirements are met. The reason for QA, is to develop procedures and routines which manages QC methods ensuring quality of the project.

[edit] Why is quality assurance important?

Figure 2: Volkswagen Golf TDI 2.0 at a car exhibit, displaying the clean diesel brand [3]

In modern society quality products are becoming more available to the public. This increases the customer expectations with regards to quality and performance. Requirements for performance can have a variety of characteristics which is related to environmental, resource or safety. The capability of organizations to fulfill these requirements, is challenged. When organizations fail to conform to quality requirements, reputations and trust are decreased. These two factors, customer trust, and organizational reputation, plays an increasingly important role in a globalized market. To give an example, consider the Volkswagen emission scandal in 2015.

[edit] Volkswagen emission scandal

Volkswagen is a German car brand which produces cars, and was in a period of years awarded multiple environmental quality awards for their low CO2 emission diesel cars. When the Environmental Protection Agency (EPA) ran experiments on diesel cars from Volkswagen, they couldn’t replicate the emission rates Volkswagen was informing leading to investigators demanding an explanation. This lead to the discovery of scam software which was implemented in cars, designed to report fake emission rates, such that they seem more environmentally friendly. When EPA released their results documenting the systematic cheat to the public, a drop 20 % the first day, and 17 % the day after in Volkswagen shares was observed [4], showing just how important customer trust, reputation and quality conformance is for the company. In a similar manner, customer claims are important to assess in order to avoid bad publicity. In order to improve quality practice it is possible to continually assess the quality management systems, through customer satisfaction surveys. This is one of the aspects identified in ISO-9001:2015 as a mandatory procedure for assessing performance and detecting faults or risks inherent in the company.

[edit] Three main types of systems

Figure 3: Certified Construction Manager or short CCM is the only certification accredited by the American National Standard Institute, (ANSI) under ISO-17024

Quality assurance systems can be characterized into three main categories.

  • First-party system
  • Second-party system
  • Third-party system

These are outlined in the following.

[edit] First-party system

First-party systems, are systems where companies themselves are responsible for quality conformance. This way, only the company itself regulates and optimizes the quality of produced services and products, without intervention of external organizations. This provides an conflict of interests since companies want to present themselves in the most favorable way which can influence it's quality control protocols, and skew results.

[edit] Second-party system

Second-party systems is where suppliers follow the strongest marketplace standard which hasn’t been accepted through an independent organization such as ISO. An example of a second party system is the advent of software such as windows, which becomes a de-facto standard for other companies to follow, even though this service quality hasn’t been approved by an independent organization.

[edit] Third-party system

Third-party systems are when quality of services or products are controlled by independent organizations, thereby ensuring no conflict of interest. This approach of conformance with quality requirements, is determined best practice [1]. An example of a third-party system is the ISO system, which certifies companies fulfilling international recognized criteria regarding quality. Another organization which certifies construction managers is the Construction Management Association of America, (CMAA). For more information see; [5]. They certify construction managers which fulfill requirements outlined in the American National Standards Institute, (ANSI) which is following the international ISO:17024 standard [5]. A more detailed description of the ISO-9001:2015 standard is outlined in the following.

[edit] ISO-9001:2015, a third-party system

Since the transition into the post-modern construction era, where projects are "unique" and planned continuously throughout the project life cycle. Systems such as the Last Plannar system [6] arises which focuses on quality requirements. This is because quality conformance play an increasingly important role.

The third party independent organization ISO, has therefore been established with the goal of setting quality standards. Companies can be granted a certification from ISO-standards if they fulfill these requirements. These requirements are determined in various standards. One of the defining standards in quality project management is DS/EN ISO-9001:2015. To fully understand terms and definitions of the full article, the reader is directed to definition standard DS/EN ISO-9000:2015 [7]. If a company has fulfilled all regulatory specifications defined in [2], then it can be certified. The specification of requirements are determined using other standards or where those aren't available, best practices.

This way of assuring quality services and products through certification, is effective because of the independence of the ISO-cooperation. Which otherwise would be deemed untrustworthy, an attribute devastating for the entire building process. The independence cultivates trust between customer and contractor, and is the main reason that third-party systems are deemed best practice.

[edit] Commissioning as quality assurance

Figure 4: Illustrative sketch of commissioning effects and maintenance costs with and without commissioning. x-axis shows years after the building has been taken into use, while the y-axis shows maintenance costs. The light blue color indicates regular costs without performing commissioning, dark blue without commissioning but with some usage and light pink with commissioning. Sketch taken from [8].

Commissioning originates from quality assurance of ships. Commissioning is conducted on all mayor ships before venturing out on the open sea, in order to ensure quality standards are met. In recent years, commissioning have been tested on building projects as an attempt to lower costs arising from late delivery, budget overruns, fault correcting, etc. It was also implemented to optimize customer satisfaction levels, together with more accurate budget predictions.

The process of commissioning is fairly simple, it evolves gathering main stakeholders from all parts of a specific construction including e.g. engineers, entrepreneurs, carpenters, builders, users, architects, etc. together throughout the project phases on a weekly basis going through design and completed work. As a side benefit, dialog and collaboration will also increase as an effect of the increased communication. This ensures initial quality requirements are met and unforeseen problems are dealt with, before continuing construction. This minimizes the need for expensive rebuilds.

Commissioning takes time and costs money. This should be recognized and taking into account in the tender phase and preferably written into suppliers contracts. This ensures that all parties will attend the meetings. This is crucial, since systems are cross-correlated with each other, plumbing is correlated with heating which is connected with walls and isolation, and so-on and so-forth. For the procedure to be effective, all relevant parties must attend the meetings and communicate.

[edit] Culture of Improvement

The following provides an outline of two similar methods for facilitating a culture of improvement. By providing the framework for a culture of improvement, companies can identify problems and rank themselves against each other. This provides the incentive for improvement of performance. As a result, customers are provided with better services and companies are continually improving themselves.

[edit] EFQM and the European Quality award

Figure 5: European Foundation of Quality Management, the basic model, illustrating focus points for achieving high quality performance, figure from [9]

The European Quality Award is an award granted to companies, organizations, firms and other professionals excelling in producing high quality services or products. The award is one part of the European Foundation for Quality Management, which is an organization providing the framework for establishing criteria suitable for determining which organizations is worthy of the award. There are multiple more or less detailed versions of this framework, for illustrative purposes the following outlines the freely available basic model, see figure 5, which provides an insight into how quality assurance through certification, can be achieved.

The basic model containing the major points organizations should focus on, can be seen on figure 5. The figure is an excerpt from [9], which is a website containing detailed information about the points displayed. Furthermore, spreadsheets and questionnaires are also available for assessing the individual clients performance. This kind of performance assessment, can help identify problems, thus initiating the first steps of improvement needed in order to obtain certification. This way of helping organizations through transparent questionnaires and other methodologies, improves the integrity of the awards and certifications. It is an important aspect, since trust in the certification is the single most important aspect.

[edit] Project excellence

Figure 6: International Project management association, (IPMA) basic model for ranking organizations against each other. Taken from [1] page 335, figure 12.8

The figure to the right shows a schematic overview of the project excellence award. Organizations are primarily judged on two different aspects, project management and results. The following outlines an explanation of the weighing and what aspects are covered.

Project management consists of five different aspects of enablers. Enablers are resources which drive the project forwards towards fulfilling customer needs. This can be leaders of a team, suppliers, collaborations, etc. All important for reaching project goals. The weighing on the enablers are different, with processes and project objectives scoring the highest. This indicates, that processes and reaching project objectives are a critical part of performance, and organizations therefore need to direct attention towards these two for ensuring best performance.

The other part which is evaluated is Project Results. Here the weighing is further skewed, with almost the entire scoring dependent upon "Key performance and project results" and "Customer Results", covering aspects like; efficiency, conformance, customer satisfaction, project costs. This is then influencing the management aspects and visa versa. Organizations are scored which cultivates improvement throughout the organization.

The project excellence model can be used on the building industry by assessing the performance of all contributing parties including, engineers, contractors, entrepreneurs, architects and suppliers. All of them can be scored against similar firms, thus ensuring each company is performing to the fullest of their capabilities. This is valuable for the customers receiving the final product and is also preventative in the formation of cartels, since the ranking is performed by an independent organization.

[edit] Advantages

The main advantage of general quality assurance is increased trust between parties, that best practices are upheld. The increased trust, facilitates improved efficiency throughout the building process and enables collaborative efforts for improving overall performance. This can result in a positive feedback loop where quality is continuously improved together with trust. Some positive effects resulting from the increased quality are; a minimization of disputes leading to lawsuits, improved customer satisfaction, improved reputation of contractors, entrepreneurs, suppliers, etc. and reduction of costs associated with poor performance. Quality assurance is therefore an important aspect of quality management and should be put under focus, especially during early stages of construction. Advantages of the previously outlined methods are given in the following sections.

[edit] Third-party system

Advantages of the third party system include, an independent assurance that international/national regulations and practices are upheld, it enables customers to easily identify companies renown for providing quality services. These are valuable, in an industry where the general public opinion is negative.

The advantages of the EFQM framework is that it enables firms to identify weaknesses and strengths. It enables companies to enter into a positive feedback loop of continual improvement. This is necessary in a world where what is excellent today is mediocre tomorrow. It provide tools for companies to tackle the complexity of large projects by dividing the overall tasks into smaller boxes such that performance is easily quantified and compared with other projects.

Advantages of the project excellence model is that it is a quantitative way of ranking companies against one another, thereby cultivating a competitive environment. The model can be used on different scales, as a big scale tool of evaluating large projects, or as a small scale optimization of a particular supplier.

[edit] Challenges

Quality Assurance is difficult in post-modern construction for a variety of reasons including; communication, project uniqueness, social status, interest interference, etc. The main challenge is thorough communication between partners and collaboration. Communication also plays a critical role in safety conformance.

[edit] Commissioning

Another important challenge for especially commissioning is economy, as the customer needs to be convinced that resources should be spent on cultivating a commissioning process, even thou immediate rewards aren't readily apparent. This is an aspect, which the industry is improving on, since the publishing of studies comparing maintenance costs of buildings, with and without commissioning. Lastly, an important challenge of commissioning is mediating interested parties when non-conformance is observed. This involves assuring stakeholders, that the commissioning agent is not a quality police, but rather, a partner helping with achieving peak performance.

[edit] Last Planner System

The last planner system faces economic challenges since meetings are costly, also with respect to time. Especially this time aspect of the last planner system, proves problematic, since suppliers, contractors and other interested parties often work on multiple projects in parallel. This complicates the logistics of arranging meetings. In order to combat this, one could make meetings an obligatory part of the collaboration with economic penalties to members not attending. Thus providing another incentive to attend meetings rather than pure quality performance.

[edit] Third-party system

The main problems with third-party certification systems, are that of clearly defining quality standards which on one hand, is broad enough to be applicable to all projects, but at the same time quantifiable such that performance can be evaluated. Inconsistencies, miscommunication, vague specification and costly procedures, are all factors complicating certification of companies. Furthermore, when companies which are deemed trustworthy defaults and get bad publicity, e.g. Pihl and Søn, then the entire foundation of customer trust is undermined.

[edit] Conclusion

Quality assurance is an important contributor to total quality management systems. It ensures a trustworthy bond between client and organization that quality is in conformance with expectations. Reasons for why quality assurance is important have been outlined and used in an example case of the Volkswagen emission scandal. Building trust and creating a culture of improvement are some of the main benefits.

In modern time, commissioning has been tried out as an quality assurance method. The effects of commissioning, are a lowering of maintenance costs which is a substantial part of the lifetime cost of a building. On the down side, commission is an expensive procedure without any immediate benefits. This makes it difficult to sell. Furthermore, commissioning takes time and requires a collaboration between interested parties which can prove difficult to reach when multiple projects are worked on simultaneously.

The certification system is outlined and the third-party system is deemed best practice, because of the independent judgement. The ISO standard 9001:2015 have been outlined which provides the general guidelines for companies to follow. Clear specification of quality requirements prove difficult and the assurance requires quality control and methods for conducting this have been linked to. In similar manner, the american CCM certification, EFQM and IPMA frameworks have been explained. The systems provided the framework which cultivates a competitive culture of improvement within the company. This could result in positive feedback loops where quality and trust are continually increased.

[edit] Abbreviations

  • ANSI - American National Standards Institute
  • ISO - International Standard Organization
  • CMAA - Construction Management Association of America
  • CCM - Certified Construction Manager
  • QMS - Quality Management Systems
  • QA – Quality Assurance
  • QC – Quality Control
  • TQM – Total Quality Management
  • EFQM - European Foundation of Quality Management
  • EPA - Environmental Protection Agency
  • IPMA - Internation Project Management Association

[edit] Annotated Bibliography

Winch, Graham M. 2010, “MANAGING CONSTRUCTION PROJECTS”, 2’th edition, WILEY-BLACKWELL, ISBN 978-1-4051-8457-1. - Management book explaining principles, methods, and methodologies for planning and management in construction. Chapter 12 - Managing conformance, is used extensively throughout the article. The chapter explains structure of Quality management, definition, methodologies and presents case studies exemplifying the outlined theory. Especially quality control methods such as statistical, performance and inspection methods are explained, together with problems arising in general quality management.

DS/EN ISO 9000:2015, “Quality management systems – Fundamentals and vocabulary (ISO 9000:2015)”, Technical Committee ISO/TC 176, approved by CEN, Göteborg plads 1 DK-2150 Nordhavn - ISO - standard presenting fundamentals and definitions associated with other quality management standards, and general ideas behind other similar standards of quality management.

DS/EN ISO 9001:2015, “Quality management systems – Requirements (ISO 9001:2015)”, Technical Committee ISO/TC 176, approved by CEN, Göteborg plads 1 DK-2150 Nordhavn. - Iso standard explaining general requirements needing to be met in order to obtain the certification. It it not explicitly specifying requirements, but instead is presenting the framework from which specific criteria can be specified.

“Volkswagen the scandal explained” 10. December 2015, BBC News, Russel Hotten, http://www.bbc.com/news/business-34324772 seen 08/06/2017. - Article containing general details about the Volkswagen scandal, including a timeline.

"Lean Construction & Last Planner System", S. Rolf, DTU, January 10, 2017, internal link: https://cn.inside.dtu.dk/cnnet/filesharing/download/cc9fa450-000e-41b0-a037-c6d6bdc16f15 - Powerpoint slideshow, explaining the details around the Last Planner System, together with principles of LEAN construction and examples. Lastly it contains examples last planner meetings together with an explanation of the 7 flows, which state when an activity is ready to be initiated.

http://cmaanet.org/about-cmaa, website explaining in detail CMAA's goals and ethics, seen 12/06/2017 - Website explaining the principle behind CCM certification of people, together with an detailing of the responsible organization "CMAA's" goals, ethics and code of conduct.

"Commissioning process", Vaerdibyg.dk 2013, in collaboration with organizations; Dansk byggeri, Bat-kartellet, Bygherre foreningen, Danske Arkitekt Virksomheder, DI Byg, Foreningen af Rådgivende ingeniører, TEKNIQ installatørernes organisation, link: http://www.vaerdibyg.dk/index.php?option=com_docman&task=doc_view&gid=156&Itemid= - A folder made for practitioners shortly describing how and why commission should be used. The folder consists of two main chapters, first chapter is explaining the commissioning process in detail, while the second chapter explains the reasons why commissioning should be conducted. The folder is in Danish, but an English version should be available in the near future.

[edit] References

  1. 1.0 1.1 1.2 Winch, Graham M. 2010, “MANAGING CONSTRUCTION PROJECTS”, 2’th edition, WILEY-BLACKWELL, ISBN 978-1-4051-8457-1
  2. 2.0 2.1 DS/EN ISO 9001:2015, “Quality management systems – Requirements (ISO 9001:2015)”, Technical Committee ISO/TC 176, approved by CEN, Göteborg plads 1 DK-2150 Nordhavn.
  3. Figure from wikicommons, link https://commons.wikimedia.org/wiki/File:VW_Golf_TDI_Clean_Diesel_WAS_2010_8981.JPG
  4. Website article: “Volkswagen the scandal explained” 10. December 2015, BBC News, Russel Hotten, http://www.bbc.com/news/business-34324772 seen 08/06/2017.
  5. 5.0 5.1 http://cmaanet.org/about-cmaa, website explaining in detail CMAA's goals and ethics, seen 12/06/2017
  6. "Lean Construction & Last Planner System", S. Rolf, DTU, January 10, 2017, internal link: https://cn.inside.dtu.dk/cnnet/filesharing/download/cc9fa450-000e-41b0-a037-c6d6bdc16f15
  7. DS/EN ISO 9000:2015, “Quality management systems – Fundamentals and vocabulary (ISO 9000:2015)”, Technical Committee ISO/TC 176, approved by CEN, Göteborg plads 1 DK-2150 Nordhavn.
  8. "Commissioning process", Vaerdibyg.dk 2013, in collaboration with organizations; Dansk byggeri, Bat-kartellet, Bygherre foreningen, Danske Arkitekt Virksomheder, DI Byg, Foreningen af Rådgivende ingeniører, TEKNIQ installatørernes organisation, link: http://www.vaerdibyg.dk/index.php?option=com_docman&task=doc_view&gid=156&Itemid=
  9. 9.0 9.1 The basic EFQM excellence model, seen on http://www.efqm.org/efqm-model/efqm-model-in-action/strategy on 21/06/2017.
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox