Optimism bias, Strategic Misinterpretation and Reference Class Forecasting (RCF)

Revision as of 15:26, 1 March 2019

Abstract

Most major projects around the world are facing the problems regarding planning and underestimating the costs of a project in the initial phases. These setbacks can cost an enormous sum of money and cause major setbacks of the planned schedule. The initial planning and make a reliable budget are some om the main organizational managerial skills for a project manager. ^[1].

But unfortunately, it is today seen that most projects will not fulfill the initial plans and will not met the initial budget goals. It is logically a utopian idea of predicting the unpredictable (known unknowns) or even the things that is impossible to foreseen. And that is why we, as project managers need tools and learn from the past and others.

The Danish professor Bent Flyvbjerg did research into cost overrun of major projects. By sub-dividing the broader aspect of this into the two topics: Optimism Bias and Strategic Misrepresentation Flyvbjerg explored and explained more about the cost and benefit shortfall of major projects. And through further investigation, Flyvbjerg came up with a possible solution: the use of Reference Class Forecasting ^[2]. The Reference Class Forecasting approach provides a more general overview and “is beneficial for non-routine projects” ^[3] .

Through reading this article, an explanation of the strategies and ideas mentioned above will be discussed. Furthermore, the idea of Reference Class Forecasting in larger projects will be covered as well as the studies' limitations. Examples of causal factors for cost overruns will be provided followed up by specific tools for project managers.

Word definitions

Optimism Bias

The term optimism bias was invented by the Nobel prize winner Daniel Kahneman describing the idea that most people consider themselves less likely to experience something unpleasant. This leads to the effect of overconfidence in personal judgements, leading to project managers misjudging the outcome of their decisions. Risks tend to be seen lower and own capabilities better although previous experience shows the opposite. This can result in project managers undertaking projects with too optimistic expectations about budget, risks and the project scope, which will most likely not be met. ^[4].

Strategic Misrepresentation

You have a great idea for a book, and you have found a publisher who is willing to pay. However, he needs to know more about the time perspective. "When can I have the first manuscript? Can you have it done by the end of next month?", he asks. You lower your eyes and gulp. "Of course, no problem" you answer. You have never managed that kind of timeline before, but you are aware that if you tell the truth, the publisher will not go with your idea. You do not feel like you just lied - and in official social terms, you did not, you performed the behavior of Strategic Misrepresentation and sorted out some of the truth. In larger project this Strategic Misrepresentation can result in underestimating costs, pre-determining and quite likely also overestimating the potential clients' benefits. Applying additional pressure and strain on individuals through manipulation, competing for scarce funds or jockeying for a position all qualify for the same over-arching category. ^[5]. Note that Strategic Misrepresentation is a form of bias as well, but is used more intentional, and more likely to be a form a technique. When project planners present their cases, they often brighten the numbers regarding the risk and the benefits of their projects. Regarding to Flyvbjerg, they are deliberately deceiving the decision makers, since the projects that look the best on paper will be approved. One of the things that can make a project more vulnerable to Strategic Misinterpretation is the end date is a few years down the road ^[6]. No one knows more about large-scale projects than professor Bent Flyvbjerg. Why are cost and schedule overruns so frequent? Because it is not certain that it is the best offer or project that wins. It is the project which looks best on the paper that wins.

Reference Class Forecasting

Go back to the previous example with the book. How would you decide how much time you actually needed? Reference Class Forecasting (RCF) could be a method to determine so. The problem can occur when to determine which class to place your project in. Would you compare with all books written? All books within your subject? Books at the same educational level as yours? This is the idea and challenge of RCF. RCF is a method of looking to future events by taking relatable situations and their previous outcomes. This approach aims to give a much less biased view on a specific event. The advantages and downsides of Reference Class Forecasting will be further clarified in this article.

Application

Background

Risk contingency "is expected to be expended [and is] an amount added to an estimate to allow for items, conditions, or events for which the state, occurrence, or effect is uncertain and that experience shows will likely result, in aggregate, in additional costs" ^[7]. There are various methods that can be used to quantify the risk contingency, such as Reference Class Forecasting (RCF), the conventional contingency approach and risk-based estimating. This article focuses on the RCF. The cost estimates produced at different stages of the same project carries different levels of uncertainties and risks and thus different estimation accuracies. As a project progresses through its lifecycle, more information about the project's scope, design, and specifications become available, which enable the estimation team to more accurately estimate the quantity and price of material and resources. As a result, generally more risk contingency is applied at the earlier stages than in later stages.

Project cost performance and causal factors for cost overruns

Tools for project managers

It is possible to use other biases to limit the effects. This bias is particularly important for decision-makers creating health or safety products, where the dangers of being overly optimistic can lead to dreadful outcomes. There are two researched ways of reducing the Optimism Bias:

1. Make past bad events more easily retrievable from one’s memory
2. Highlight losses that are likely to occur because of these bad events

Bringing negative events to our mind before we have the option to act can be a great technique to change people's behavior. The aim is to make the negative effects of a certain action clear to the individual, and offer a clear. It is important to negate the potentially huge costs of the positivity bias when estimating the expected time to complete a task or project. As project manager, remember to factor in a proportional optimism Bias multiplier into estimations given. Governments have this problem so consistently that there are even detailed documents outlining a 5-step approach of how to factor the bias into the planning of large projects. Unfortunately, past cost overruns in mega-projects have resulted in scandalous errors with projects finishing significantly higher than original estimated budgets. The literature abounds with examples. In general, these studies agree that nine out of ten projects exceeded budget. "Overruns of 50% are common; cost overruns over 50% are not uncommon" ^[8]. Flyvbjerg suggests the major causes for the differences in budget and actual are biased estimates of costs and benefits. Unfortunately, its adoption in large projects has been incredibly limited.

Required uplift as a function of maximum acceptable level of risk for cost overruns

In collaboration with the Danish consultancy COWI, Flyvbjerg developed guidelines for project planners to remediate their estimations. The application for a fixed link project will serve as an example. Figure 1 shows the required uplift of the initially estimated budget regarding the acceptable chance of a cost overrun. Such a tradeoff is the outcome of the second step of the RCF method. If a planner now wants to correct his estimations about budgeting, he can calculate the required uplift regarding a possible risk of a cost overrun.

As an example, when the maximum acceptable chance for a cost overrun is set to be 20 %, the corresponding uplift of the budget is around 50 % in this case, illustrated by Figure 1. It has to be mentioned that the RCF method comprises simple looking steps on the first view. Nevertheless, the setting of the reference class is a difficult task. Gathering empirical data from the business sector the specific project is embedded in, selecting only comparable projects and creating the probability distribution are complex tasks.

Limitations of the Reference Class Forecast method

Although Reference Class Forecasting (RCF) can largely be quite an accurate and effective way to limit the bias in a situation, sometimes it is virtually impossible to limit the biases even with the use of RCF. A better illustration of this statement is an example of colossal-sized projects, like the Olympics. In terms of the management of such an event, with relation to RCF, the issue for an organization undertaking such as high-status event, is that it is almost a certainty that it will the companies first time in charge of such a project. With never large projects never done before. There has not been an Olympic games that has met or been even close to its initial budget with an average of 51% cost overrun for the hosting country throughout recent history of the event)[8]. With this knowledge, although the Olympics can be considered a very niche event, it can be demonstrated that it even with RCF in place, tasks with such significance make it an increasingly difficult task to refer to RCF as it is a first-time event for the country and organizations involved. As every other method, the RCF method is facing limitations. There seem to be three major issues limiting the application of the RCF method. These are:

• Definition of frequentism
• Subjectivity of reference classes
• Acquisition of data (database)

The first one is more a philosophical problem, whereas the second one concerns the subjectivity of the method. The last one is about gathering the empirical data. Nevertheless, they go hand in hand.

Frequentism

Something about Frequentism

Subjectivity of reference classes

Something about subjectivity of reference classes

Acquisition of data (database)

Something about acquisition of data

Annotated bibliography

Bib

References

1. ↑ PMI:Project Management Institute,Project Management: A guide to the Project Management Body of Knowledge (PMBOK guide), 6th Edition 2017 , Table 1-2.
2. ↑ Flyvbjerg, Bent, Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice, (European Planning Studies, 2008), 16. 3-21.
3. ↑ Flyvbjerg, B., From Nobel Prize to project management: getting risks right, (Paper presented at PMI® Research Conference: New Directions in Project Management, Montréal, Québec, Canada. Newtown Square, PA: Project Management Institute, 2006).
4. ↑ Kahnemann, Daniel, Thinking fast and slow, New York: Farrar, Straus and Giroux, 2013.
5. ↑ Flyvbjerg, Bent,Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice .
6. ↑ Hot Air - Meaning Ring, http://meaningring.com/2016/05/30/strategic-misrepresentation-by-rolf-dobelli/
7. ↑ Liu, L., Wehbe, G., & Sisovic, J. (2010). The accuracy of hybrid estimating approaches? Case study of an Australian state road & traffic authority. Paper presented at PMI® Research Conference: Defining the Future of Project Management, Washington, DC. Newtown Square, PA: Project Management Institute.
8. ↑ Clifford, Gray, Revisit of Reference Class Forecasting (RCF): Estimating Costs of Infrastructure Projects