Biases in Project Management

From apppm
Jump to: navigation, search

Developed by Eva Rún Arnarsdóttir

Contents

Abstract

The human mind is an effective and powerful tool. However, it is not without its faults and has some limitations which can cause biases. Biases occur in everyday activities and decisions, it is the brains' way of dealing with uncertainty and complexity, often in wrong ways. There are so many seemingly trivial decisions made on the basis of biases, whether it is choosing a cola drink or which car to buy.

In this article cognitive biases are examined, with most emphasis on Optimism Bias since it is a very important factor in project management, specifically in cost and risk assessments. Cognitive Bias also includes other topics such as Gender Bias, Groupthink and Confirmation Bias, that will be touched upon in this article. The notion of biases has evolved through time and the understanding of them has been steadily increasing. These biases are very important in a team setting and therefore fall under the realm of project management. It can be found in project management literature when team building is discussed e.g. in Guide to the Project Body of Knowledge (PMBOK® Guide) where Interpersonal and Team Skills or Expert Skills are mentioned. [1] This topic is mainly related to project management but can also be found in program and portfolio management.

Project managers have a tendency to overestimate benefits and underestimate cost i.e., be too optimistic. This is known as Optimism Bias and is widely accepted as a key reason for overruns in projects, especially in large infrastructure projects. [2]

Being aware of these biases is crucial for all project managers in order to be able to offset them. By acknowledging biases and applying appropriate measures, it is possible to counter the effects. These measures include simple solutions such as finding a mutual ground within teams and more complex solutions such as Reference Class Forecasting, which is explained in this article.

In this article some of these biases related to Project Management are examined in more detail. How these biases can be seen in project management and measures to counter them are presented as well as how they can be applied and when. Finally, some limitations are considered and topics for further reading recommended.

The Big Idea: An Introduction

When it comes to decision making the human mind tends to be biased. In past years project estimates have often not been satisfactory and people believe that it is due to biases in many cases. This causes a lot of problems especially for the project owner. In public projects a cost overrun is unfortunate for the national treasury and will eventually be bad for the citizens. The project might end up delivering worse results on account of low available capital or having to speed up the project which diminishes the quality. [2]

Limited information, limited time and limited minds are all causes for bad decision making. In many cases we rely on heuristics instead. Heuristics are approximate methods used to simplify decision making, often choosing a satisfactory solution instead of the optimal one. Sometimes heuristics lead to good decisions, but they can also lead to inaccurate conclusions. [3] Heuristics and biases are often linked together as the concepts are very interconnected where biases lead to people making decisions based on heuristics.

This section contains definitions on biases, both the general definition as well as some important types of biases in project management.

What is Bias?

The definition of bias in the Oxford dictionary is split in four meanings, two of whom are relevant in project management and will be addressed in this article:

  1. “A strong feeling in favour of or against one group of people, or one side in an argument, often not based on fair judgement.” [4]
  2. “The fact that the results of research or an experiment are not accurate because a particular factor has not been considered when collecting the information.” [4]

The first definition is tied to people and communications between either team members or stakeholders. The latter can be more related to uncertainty and risk management.

Cognitive Bias

The concept of cognitive bias was coined by Amos Tversky and Daniel Kahneman in the 1970s. Cognitive biases are a result of your mind trying to simplify things when the human mind is forced to make a decision and deal with complexity or uncertainty. This can lead to incorrect judgement and as a result a miscalculation of the situation or project at hand. Kahneman and Tversky found that these errors were often systematic rather than random and therefore they are sometimes referred to as Systematic biases. Systematic biases are frequent distortions in the human mind, often contrary to rational thought. [5] Cognitive biases can be an advantage when a quick respond is more valuable than an exact right solution. In some situations, it is crucial to make decisions timely for example when in life threatening situations. Project managers are usually not faced with those situations in their professional life and accuracy is often preferred over quick responses so biases are more often seen as disadvantages in project management.

Wikipedia has a page dedicated to different types of biases titled “List of cognitive biases” which contains 185 different types, so it is a very wide concept. [6] Therefore only a few will be mentioned here and only Gender Bias and Optimism Bias will be explored in depth. The following section introduces these biases with some basic definitions.

Gender Bias is quite self-explanatory, people are misjudged based on gender. Typically, women are not recognized as equals to men in fields that have been male dominated through the years. [7]

Confirmation Bias happens when people look for confirmation or evidence to support what they already belief and ignore information that contradicts it. [3]

Conservatism is when team members is will not take into consideration new information or any negative feedback but rather stick to what has always been. [8]

Groupthink occurs then team members think alike, and they do not accept evidence that proves otherwise. [8]

Misconceptions of Chance / The Gambler's Fallacy is when you look at independent outcomes and try to guess the next for example when flipping a coin. If you get 6 heads in a row you think the next one is going to be tails even though the odds are still 50/50. Or a gambler who has been on a losing streak feels he is due to win soon even though each game is independent of the other. [5]

Optimism Bias is a result of project managers' tendency to be too optimistic when calculating the benefits of projects and downplay the costs. A large number of recent projects have had cost overruns and/or demand shortfalls. It is widely accepted that Optimism Bias is to blame for these miscalculations. This is most evident in large infrastructure projects, especially in the public sector where politics play a big role. [2]

Overconfidence Effect occurs when the team or an individual team member is overconfident without any evidence supporting their belief. [8]

Recency Illusion is when much emphasis is put on recent data, often older data is more relevant. [8]

Application

How are these biases above applied in project management and what can managers do to combat them? First an overview of how biases apply in teams is provided as well as some information on Gender Bias. Then Optimism Bias is discussed and finally a method known as Reference Class Forecasting is introduced which can help combat Optimism Bias.

Biases in Teams

Failure rates among projects is definitely too high and poses a concern, so studying why failures happen is becoming more common and important. How a project turns out is the result of many factors e.g. leadership, cultural and behavioral factors where biases and human emotions play important roles. Culture relates to values and beliefs of a group that is adopted at early stages of life and therefore hard to change. Individual cultures and leadership both influence the organizational or project culture. Project managers must try to foster a project culture where everyone feels safe and respected. Biases can have a huge effect on the project culture and team spirit. Gender Bias, Confirmation Bias, Conservatism and Groupthink can influence people’s actions and feelings. These types of biases can be toxic in a team setting. It is up to the project manager to acknowledge these biases and make sure the team is not affected by them for example by finding mutual ground among team members in the beginning of a project. [8]

Purvis et. al. (2004) proposed 8 tactics to battle biases and minimize the probability of a failure. First a formal kick-off event where previous projects are discussed, what went wrong and how is it possible to prevent it happening again? To make sure decision processes are based on objective data i.e. not falling for the Recency Illusion, to clearly specify methods for planning, to look at both positive and negative sides of the project and to inform the team members about the project management process, specifically how to address unexpected events. They also mention the importance of constructing more than one alternative approach to the project, to institute a committee to oversee the project and to formulate procedures intended for positive feedback to employees. [9]

Gender Bias

Gender discrimination has been around for ages and affected lives in many ways. Through time, Gender Bias has diminished but there is still a long way to go. In the workforce women are still overlooked for management positions. Women only comprise 15.2% of corporate boards of Fortune-500 companies, even though there is no evidence they are any less qualified. Many studies show that people tend to believe the gender stereotypes and associate some adjectives and attributes to either men or women. Stereotypical attributes of men are competent, ambitious, independent, decisive, logical, and dominant. However, attributes to describe the stereotypical woman are kind, caring, collaborative, obedient, understanding, and considerate. These stereotypes regarding the genders are used to make quick decisions when forming impressions about people. It is wired into our brain that this is in fact the truth even though it is not rational. When women break out of the typically female persona they are often met with negative reactions and disapproval and can be seen as cold or mean. [10]

Men are often thought of as “benefiters” from Gender Bias, this is not always the case. Gender Bias can also affect men in the workplace when trying to break into women dominated fields such as teaching or nursing. There is in fact another type of bias that can negatively affect men, called the “women are wonderful effect”, when women are preferred over men. [11]

Gender Bias is often intercorrelated with Attractiveness Bias, where more attractive people are favored over others. Marlowe et. al. (1996) found clear evidence of Gender and Attractiveness Bias in a conducted research. They also found that more experienced managers seemed to be less inclined to be influenced by these biases. [12]

Biases take time to change as they are a result of a complex psychological process. Attributes for a good leader have been changing and now female attributes such as those described above are valued more in management. This is a result of leadership evolving to foster a good work environment and sharing responsibilities. The female stereotype is also evolving and people are more likely to think that women are as intelligent and competent as men. This will hopefully translate to more women getting top management positions and decrease the pay gap between the genders. [10]

Optimism Bias in planning

Optimism Bias usually manifests in planning for projects where uncertainty and complexity is high. In such projects the risk management is a vital part of the process. Many methods exists for a risk assessment and to calculate the cost and benefits of a project. Recently studies have proposed that the best way to do a cost estimate and a risk assessment is with the Reference Class Forecasting Method, which will be discussed in this section along with information about Optimism Bias in general. Bent Flyvbjerg of Oxford University has researched Optimism Bias extensively, both causes and possible remedies which include the Reference Class Forecasting. He has published a wide range of articles on the subject which provide the basis for this section. [13] [14] [15] [16] [17]

Illustration of uplifts
Figure 1: Inaccuracy in project estimations. Adapted from Flyvbjerg (2006), page 5. [17]

Flyvbjerg et. al. (2002) examined the inaccuracy of transport project estimates and found that costs were underestimated in almost 9 out of 10 projects. Actual costs were 28% higher on average than the estimations and this seemed to be a global phenomenon. They noticed that the cost overrun was different depending on the type of transport project. They examined 258 projects, split into Rail, Road and Fixed Links (Bridges and Tunnels). Rail projects had average cost escalation of 44.7%, the fixed link projects had average cost escalation of 33.8% and road projects had the lowest average of 20.4%. Cost underestimation had not decreased in 70 years when Flyvbjerg and colleges wrote this paper. It would be interesting to see newer numbers concerning Optimism Bias and to see if recent methods such as Reference Class Forecasting could be diminishing the project overruns. Flyvbjerg based the Reference Class Forecasting method on the classification of projects from this paper. [13]

Flyvbjerg wrote a paper about megaprojects in 2014 where he lists many past projects with cost overruns and how large the overrun was. Five of these projects have a cost overrun of over 1000%. The highest overrun is the Suez Canal with a cost overrun of 1900%. These numbers are unacceptable since the methods and technology to predict these numbers is available. Project managers must put more emphasis on cost and benefit estimations.[14]

Possible causes for wrong estimations in planning

Flyvbjerg categories the causes of Optimism Bias into 4 categories. First is technical causes which includes limited data and scope changes. Second is the psychological aspect, the tendency for the brain to favour optimism. Third and four categories are economic and political causes where interest of a third party causes deliberately wrong estimates. [15]

Make and Preston (1998) documented 21 sources of error and biases in transport project appraisal e.g. double counting or interactions that are not taken into consideration in models, unclear objectives, and incorrect definitions of study area, base or assumptions to name a few. All these factors contribute to the last one, Optimism Bias. Benefits are sometimes counted more than once, quantifiable costs excluded, and the asset life is overestimated. In their paper they call it Appraisal Optimism and say it is the greatest problem of all. They suggest three solutions, an in-house group to ensure honest appraisals, more transparency to the public and extra emphasis on ex-post evaluation. [18]

Political influence can have an effect on projects, especially public projects. Project managers and/or project owners have been known to downplay the costs and overhype the benefits in order to get the project approved. They are deliberately applying Optimism Bias; they know the estimation is way too optimistic, but they believe the project will be cancelled if they present the real numbers to the people in charge. This is called strategic misrepresentation but is out of the scope of this article and will not be explored in more detail here since it is an intentional bias, and therefore essentially not a bias. [16]

Reference Class Forecasting

Reference Class Forecasting (RFC) is a method to improve the reliability of project cost estimates. It originated with the aforementioned psychologists Tversky and Kahneman whom felt this method could compensate for the cognitive bias of decision makers. Kahneman developed the theoretical framework which earned him a Nobel Prize in economics. Reference Class Forecasting focuses on including historical data as a reference point and therefore taking an “outside view”. Instead of only looking at the specific problem at hand, similar projects are analyzed and information from them transferred to the current problem. The data collected is on how well the projects delivered the planned benefits and if they were on time and on budget, if not than by how much. By doing this it is possible to learn from past mistakes and the estimates become more realistic and accurate. Though the theories were originally introduced by Kahneman he did not develop the practical use of the method. Bent Flyvbjerg, a Danish economist and professor at Oxford University, in association with COWI, developed the practical method for use in planning projects.
Illustration of uplifts
Figure 2: Required uplifts for each class when using the RCF method. Adapted from Flyvbjerg and COWI (2004), page 32. [15]
American Planning Association (APA) recommended the use of Reference Class Forecasting for project planning in 2005 and since then it has grown in popularity. [17]

The Reference Class Forecasting method has three steps. First a relevant reference class needs to be found. Secondly, a probability distribution needs to be selected for the reference class which means finding data for other projects in the same reference class and using it to make an assertion about the projects in this class. Finally, the projects have to be compared to the distribution from the reference class and adjusted. [17]

Flyvbjerg examined 260 infrastructure projects and put in a database. He documented similarities between projects and classified into reference classes. The three main groups were, Roads, Rail and Fixed Links, which includes bridges and tunnels. [17]

By examining former projects and their cost overruns, a probability distribution was created for each class. Each class has a required uplift that is needed in step 3 to adjust the new project to the distribution. Managers and project owners need to establish what the accepted risk of overrun and find the appropriate uplift. As the risk gets lower the uplift gets higher. That is if the project owners are willing to accept a 30% chance of a overrun for a railway project the uplift would be 51%, however of the allowable risk is only 10% the uplift required according to RCF is 68%. The required uplift for each class and different risks is shown in Figure 2. [15]

Studies on the effectiveness of RCF

Batselier and Vanhoucke performed an experiment in 2016 to test what is indeed the best forecasting method. The project in question was a real-life construction project regarding the finishing touches on the interior of an office building. They did estimates with Earned Value Management, Baseline estimates, Monte Carlo simulation and RCF. The project was then executed, and the estimates compared to the actual cost of the project. Their findings show that RCF did deliver the best estimate out of all methods and they describe RCF as the most user-friendly method because it does not require a lot of detailed information. [19]

However, Tim Neerup Themsen (2019) suggests that the Reference Class Forecasting method is not as good as people belief. He bases his research on a Danish Megaproject, Signalling Programme, that did not deliver the promised results despite RCF being used for estimating. The project had cost overruns, was delayed and had to reduce the scope. Themsen believes that the estimation experts showed signs of biases and that RCF does not prevent optimism bias. No outside stakeholders questioned the application of RFC because they were made to believe it was superior to all other methods. Themsen poses the question of how many projects have to fail in order for people to start questioning RCF. He does not propose that people stop using RCF, but to focus on the conditions that the estimations were made under and to have an impartial reviewer. It must be noted that it was the first big Danish public project to use the RCF method so perhaps it was not carried out correctly and future projects could be executed better. [20]

Limitations and conclusions

There will always be some biases that we are not able to prevent and we can never completely be free of biases but by being aware of them can possibly diminish the effects. Project estimations will never be completely accurate but they could be a lot better. In addition, some events cannot be predicted and therefore never included in estimates, for example pandemics. Many projects have had delays and unexpected costs in relation to Covid 19 but that is inevitable in these unprecedented times.

The main limitations of Reference Class Forecasting is in acquiring a useful and reliable dataset for the reference class and to choose the right class for the project. [17] In some cases there is no similar collection of projects readily available so it is not possible to acquire a realistic estimate using RCF. [19] As the method gets more popular, the number and variety of reference classes will expand.

Optimism Bias has mostly been researched in regard to transport and infrastructure projects. It is possible to transfer the findings of these problems onto other types of projects, but it might deliver different results. Researching Optimism Bias in more general project management aspect could be a relevant topic for further research.


To be able to deliver effective estimates and essentially quality projects, project managers need to have great managerial skills and that starts with being aware of our limitations and applying appropriate methods when needed. Flyvbjerg proposes four measures to deal with Optimism Bias in addition to the uplifts mentioned before. These measures include an independent appraisal, penalizing cost overruns, formalized requirements for risk assessment in the planning stage and an emphasis on realistic budgeting as an ideal. [15] Battling biases in project management requires transparency, outside expert judgements as well as a better monitoring and control system for projects. Most importantly project managers need to be aware of the fact that the mind is limited and try to rely on facts and rational thought over the simplified version that biases support.

Annotated bibliography

The following resources are the key resources used for this article, and can provide basis for further and deeper studies on the topic.

1. Flyvbjerg, B. and Cowi. (2004). Procedures for Dealing with Optimism Bias in Transport Planning: Guidance Document. London: The British Department for Transport.

A comprehensive guidance document regarding Optimism Bias and ways to combat it. It introduces the idea of reference class uplifts as well as discusses causes of optimism bias and possible cures. The article explains the different reference classes and how the corresponding uplifts were calculated, with examples of previous projects' cost overruns.

2. Flyvbjerg, B. (2006). Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice. European Planning Studies, 16(1), 3-21.

An article explaining the Reference Class Forecasting Method and its origin within psychology with Kahneman's Planning Fallacy. It documents the inaccuracy in infrastructure projects over the years and tries to explain the reasons behind the inaccuracy. The first instance of Reference Class Forecasting is studied and an example presented of how to use the Reference Class method in a large project, the Edinburgh Tram project. Finally it details the future possibilities of RCF.

3. Tversky, A. and Kahneman, D. (1974). Judgement under Uncertainty: Heuristics and Biases. Science, New Series, 185(4157), 1124-1131.

This article was written by two psychologists in 1974 and was revolutionary within the field of psychology. It introduced the idea of Heuristics and Cognitive Biases and provided basic examples. Tversky and Kahneman wrote many other articles about this subject, all of whom are interesting and relevant to biases. The article has many examples but focuses on three heuristics employed in decisions under uncertainty, namely Representativeness, Availability of instances and Adjustments from an anchor. This article explains these heuristics and biases in very simple terms and is therefore a good reading for those interested in the psychology aspect of biases.

4. Heilman, M. E. (2012). Gender stereotypes and workplace bias. Research in Organizational Behavior, 32, 113-135.

An article about Gender Bias and Gender Stereotypes, both descriptive and prescriptive. Why women have a harder time advancing their careers and attributes linked to a good managers versus attributes linked to the genders is examined as well as what happens when people ignore these stereotypes.


References

  1. Project Management Institute, Inc.(PMI). (2017). Guide to the Project Management Body of Knowledge (PMBOK® Guide) (6th Edition). Retrieved on February 9th 2021 from https://app.knovel.com/hotlink/toc/id:kpGPMBKP02/guide-project-management/guide-project-management.
  2. 2.0 2.1 2.2 Leleur, S., Salling, K.B., Pilkauskiene, I. and Nicolaisen, M.S. (2015). Combining Reference Class Forecasting with Overconfidence Theory for Better Risk Assessment of Transport Infrastructure. The European Journal of Transport and Infrastructure Research (EJTIR), 15(3), 362-375. Retrieved on February 10th 2021 from https://www.researchgate.net/publication/275213953_Combining_Reference_Class_Forecasting_with_Overconfidence_Theory_for_Better_Risk_Assessment_of_Transport_Infrastructure_Investments
  3. 3.0 3.1 Virine, L., Trumper, M. and Virine, E. (2018). Heuristics and Biases in Project Management. PM World Journal, 7(1), 1-11. Retrieved on February 15th 2021 from https://pmworldlibrary.net/wp-content/uploads/2018/01/pmwj66-Jan2018-Virines-heuristics-and-biases-in-project-management.pdf
  4. 4.0 4.1 Oxford University Press. (2021). bias noun. Retrieved from https://www.oxfordlearnersdictionaries.com/definition/english/bias_1?q=bias on February 9th 2021.
  5. 5.0 5.1 Tversky, A. and Kahneman, D. (1974) Judgement under Uncertainty: Heuristics and Biases. Science, New Series, 185(4157), 1124-1131. Retrieved on February 10th 2021 from http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=4F92E2FFA38970D381524DF81AF1D10F?doi=10.1.1.207.2148&rep=rep1&type=pdf
  6. Wikipedia. List of cognitive biases. Retrieved on February 10th 2021 from https://en.wikipedia.org/wiki/List_of_cognitive_biases
  7. Pinto, J. K., Taranakul, P. and Pinto, M. B. (2017). “The aura of capability”: Gender bias in selection for a project manager job. International Journal of Project Management, 35(3), 420-431. Retrieved on February 11th 2021 from https://www.sciencedirect.com/science/article/pii/S0263786317300297?casa_token=eTCIzCrkRNcAAAAA:fNB7WE_DQurrkYOPS6EukGS3VC7Uelk63TKAOgGiuEtawgXtKMq4mZacbm8nvoq9178g0MnnvA#bb0225
  8. 8.0 8.1 8.2 8.3 8.4 Shore, B. (2008). Systematic Biases and Culture in Project Failures. Project Management Journal, 39(4), 5–16. Retrieved on February 14th 2021 from https://journals.sagepub.com/doi/pdf/10.1002/pmj.20082
  9. Purvis, R. L., McCray, G. E. and Roberts, T. L. (2004). Heuristics and Biases in Information Systems Project Management, Engineering Management Journal, 16(2), 19-27. Retrieved on February 14th 2021 from https://www-tandfonline-com.proxy.findit.dtu.dk/doi/pdf/10.1080/10429247.2004.11415245?needAccess=true
  10. 10.0 10.1 Heilman, M. E. (2012). Gender stereotypes and workplace bias. Research in Organizational Behavior, 32, 113-135. Retrieved on February 12th 2021 from https://www-sciencedirect-com.proxy.findit.dtu.dk/science/article/pii/S0191308512000093
  11. Krys, K. et.al. (2018). Catching up with wonderful women: The women-are-wonderful effect is smaller in more gender egalitarian societies. Internation Journal of Psychology, 53(1), 21-26. Retrieved on February 12th 2021 from https://www.researchgate.net/publication/331639154_Catching_up_with_wonderful_women_The_women-are-wonderful_effect_is_smaller_in_more_gender_egalitarian_societies
  12. Marlowe, C. M., Schneider, S. L. and Nelson, C. E. (1996). Gender and Attractiveness Biases in Hiring Decisions: Are More Experienced Managers Less Biased? Journal of Applied Psychology, 81(1), 11-21. Retrieved on February 12th 2021 from http://web.a.ebscohost.com.proxy.findit.dtu.dk/ehost/detail/detail?vid=0&sid=b6497522-254a-4e9b-9d5d-828ce6cd7149%40sessionmgr4006&bdata=JnNpdGU9ZWhvc3QtbGl2ZQ%3d%3d#AN=12428449&db=buh
  13. 13.0 13.1 Flyvbjerg, B., Holm, M. S. and Buhl, S. (2002). Underestimating Costs in Public Works Projects: Error or Lie? Journal of the American Planning Association, 68(3), 279-295. Retrieved on February 17th 2021 from https://www-tandfonline-com.proxy.findit.dtu.dk/doi/pdf/10.1080/01944360208976273?needAccess=true
  14. 14.0 14.1 Flyvbjerg, B. (2014). What You Should Know About Megaprojects and Why: An Overview. Project Management Journal, 45(2), 6-19. Retrieved on February 12th 2021 from https://journals-sagepub-com.proxy.findit.dtu.dk/doi/abs/10.1002/pmj.21409
  15. 15.0 15.1 15.2 15.3 15.4 Flyvbjerg, B. and Cowi. (2004). Procedures for Dealing with Optimism Bias in Transport Planning: Guidance Document. London: The British Department for Transport. Retrieved on February 17th 2021 from https://poseidon01.ssrn.com/delivery.php?ID=044090101066109125071078086000091088034056090036079024023025127111066004108030084075060056032028047044115026097031014085098116112075028080092082020120103025007110110026011016025027120109086092083028031073004095017068115099071026100010031005070100008084&EXT=pdf&INDEX=TRUE
  16. 16.0 16.1 Flyvbjerg, B. (2007). Policy and planning for large-infrastructure projects: problems, causes, cures. Environment and Planning B: Planning and Design, 34, 578 – 597. Retrieved on February 17th 2021 from https://journals.sagepub.com/doi/pdf/10.1068/b32111
  17. 17.0 17.1 17.2 17.3 17.4 17.5 Flyvbjerg, B. (2006). Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice. European Planning Studies, 16(1), 3-21. Retrieved on February 15th 2021 from https://www-tandfonline-com.proxy.findit.dtu.dk/doi/full/10.1080/09654310701747936
  18. Make, P. and Preston, J. (1998). Twenty-one sources of error and bias in transport project appraisal. Transport Policy, 5(1), 1-7. Retrieved on February 10th 2021 from https://www.sciencedirect.com/science/article/pii/S0967070X98000043
  19. 19.0 19.1 Batselier, J. and Vanhoucke, M. (2016). Practical Application and Empirical Evaluation of Reference Class Forecasting for Project Management. Project Management Journal, 47(5), 36–51. Retrieved on February 18th 2021 from https://journals.sagepub.com/doi/pdf/10.1177/875697281604700504
  20. Themsen, T. N. (2019). The processes of public megaproject cost estimation: The inaccuracy of reference class forecasting. Financial Accountability and Management, 35(4), 337-352. Retrieved on February 17th 2021 from https://onlinelibrary.wiley.com/doi/full/10.1111/faam.12210?casa_token=CL_6AsSd0xMAAAAA%3AcAI44_N_LdQKMR_GSb0xH7OFo_-JLzl1KbevboJSGMqBpBFwZdQ_2PagUKMTxZD2Ksitur9dI9oSFCs
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox