Optimism bias, strategic misinterpretation and reference class forecasting

Developed by Alexander Baizanos

Multi-National corporations completing large-scale projects face potential high cost overruns and possible overestimations on initial budgets. Through these, sometimes unrealized setbacks, the corporations are not operating at maximum possible efficiency. This can happen through a number of factors such as a poor allocation of resources, lacking a sufficient standard of managerial skills, Optimism Bias and Strategic Misinterpretation.

A Danish geographer by name of Bent Flyvbjerg did research into cost overrun and benefit shortfall of major projects, with additional studies into hypothetical solutions. Through his investigation, Flyvbjerg realised that the cost and benefit shortfall of major projects could be better understood by filtering the wide topic into two sub-topics, Optimism Bias and Strategic Misrepresentation. Through further analysis into the study area, a possible solution to this problem was first found through the use of the Reference Class Forecasting approach. The Reference Class Forecasting approach helps by providing a less skewed and opinionated view on a specific subject by using the 'Outside View' instead of the commonly applied 'Inside View'. The over-arching goal is to reduce cost overrun and benefit shortfall by engaging all aspects to increase forecasting accuracy.

Through reading this article, a clear and succinct explanation of the strategies and ideas mentioned above will be discussed. A brief analysis will be given on how it is used in business practice to increase efficiency and a detailed paragraph of the studies' limitations as a theory will also be covered.

Optimism Bias

Optimism bias by definition is a form of cognitive bias. This bias refers to the belief that there are reduced negative externalities about a specific situation. Factors that contribute to Optimism bias is if an individual or corporation has a more informative view of the desired end state of the project, the cognitive mechanisms in use, the information the individual or company has to begin with and the overall mood of the project scope. All these factors significantly contribute to the bias, potentially skewing judgement on a project. Optimism bias can be both positive and negative, for example, an overoptimistic overall outcome of a project's benefits can lead to poor planning, budgeting and other factors in the early stages of the project portfolio. However, an increased Optimism Bias can be seen as quite a positive aspect of a business plan. An over-confidence in the desired outcome would increase expectations of an individual or company as it is falsely made to look that it is easier to achieve. Although this sounds negative if not met, increased expectations help people in a corporation and the business its self, grow considerably as the company will continuously strive for potential new feats. Not only are potential higher target reached, but self-satisfaction is drastically increased no matter the outcome as noted by Psychologists Margaret Marshall and John Brown's theory of Optimism Bias, presented by Associate Professor of Cognitive Neuroscience Tali Sharot in a TED talk . The study found that when individuals succeed, they attribute their success to their personal traits. This increased satisfaction boosts continual drive for success, even with mishaps, the mentality of individuals has been changed to a higher level of output and thinking.

Strategic Misepresentation

Figure 1: Comparing the relationship between Strategic misrepresentation (deception) and optimism bias (delusion) through factors of knowledge and corpooration and governmental power

In corporations, whether largely publicized or not strategic misrepresentation occurs and is almost inevitable. Through various factors may it be political, economic, social or all three cohesively implemented in unity, strategic misrepresentation is often used to seize a potential advantage on another company. This advantage can be exploited when one party or organization knows less information than the other or is relying on the other organizations' information for their own benefit.

When compiling a set of outcomes for a project or plan, one party can purposely increase the number of deliverables or strategically underestimate the costs of the proposed scheme, pre-determining and quite likely also overestimating the potential clients' benefits. Politically speaking, in an organization, strategic misrepresentation can also be witnessed easily. Applying additional pressure and strain on individuals through manipulation, competing for scarce funds or jockeying for a position all qualify for the same over-arching category. Through this additional pressure, people have now been exposed to a form of bias and are swayed to a specific side for whatever reasoning.

It should be strongly noted that although Optimism bias and Strategic misrepresentation are both forms of biases, there is a sizeable difference between the two executions. Strategic misrepresentation, of course, a form of bias, is more a deception technique rather than an Optimism bias which refers to individuals using deceit techniques on one's self. As mentioned prior, through a greater knowledge of a given topic increases the strategic misrepresentation with respect to pressure, may it be political or organizational. A better demonstration of this theory can be visualised through figure 1, a graph of explanatory power with respect to political and organizational pressure. Furthermore, the same graph clearly shows the Optimism bias has a polar opposite reaction to Strategic misrepresentation when comparing using these two categories.

Reference Class Forecasting

Reference Class Forecasting is a method of looking to future events by taking relatable situations and their previous outcomes. This approach aims to give a much less biased view on a specific event. A study conducted by Daniel Kahneman and Amos Tversky in 1979 shows that when a project is in its beginning stages, judgement of the overall risks of any given events in the project, the total cost and the length of time it will take to complete, is quite often biased, sometimes even without the intention of bias. This phenomenon is called 'The Planning Fallacy'.This sensation was further investigated also by 2002 Nobel Prize winner Daniel Kahneman. The bases of this experience is that without any real means of desire, humans have a tendency to underestimate the time and resources that you will need to achieve your goals, as a corporation or as an individual. To put this into perspective, the construction of the Sydney Opera House in Australia was originally estimated to cost a grand total of $7.0 million. At the productions' end, the accumulated spending exceeded $102.0 million. On completion of the Sydney Opera House, the project not only surpassed its budget by approximately $90.0 million but it was over ten years over the initially predicted arrival date.

The investigation from Kahneman and Tversky proved that it is entirely the norm to have an unplanned Optimism bias about future events. It has to be also noted that the same study found the there had been a deliberate bias towards costs of materials and labour as a company by previous project managers in an attempt to produce a lower final cost, lower than their opposing firms to ensure that the company would get the contract.

In 2003, Kahneman completed another investigation on the benefits of using the 'Outside View' in Reference Class Forecasting rather than using the 'Inside View'. Taking the 'Inside View' instead is combining favouritism to an idea by having a tendency to tilt more to it therefore obviously incorporating Optimism bias to the situation and having inaccurate readings and forecasts as a result. Furthermore, Kahneman stated that the 'Outside View' attempts to limit the amount of confidence and assurance to a specific situation by targeting the narrowmindedness and aims to eliminate it.

Professor Bent Flyvberg, in 2007, constructed another detailed overview of Reference Class Forecasting and incorporating both the 'Outside View' and the 'Inside View'. In doing this, Flyvberg created a basic yet efficient system to also further eliminate the overconfidence present in the 'Inside Views' reasoning. The Reference Class Forecasting process three-step approach states that;

• It is critical to identify a Reference Class of a past similar project.

• Establishing a probability distribution for a selected Reference Class for the parameters that are being forecasted is vital.

• Determining the outcome with the highest potential for the plan by comparing to previous works in the same field of category is fundamental for success.

Setbacks of Reference Class Forecasting

Figure 2: Inaccuracies of rail, Bridge and tunnel, and road projects with respect to cost forecasting

Although extensive research has been completed, in many forecasting techniques, there has seemed to be no improvement in cost and demand forecasting for a project. This information has seemingly stayed drastically inaccurate for over 70 years when reports for cost data started to be documented. An example of this is through figures from construction projects within the time-span. These large infrastructure plans were consistently off budget by approximately 45% for railworks, 34% for bridges and tunnels and 20% for road schemes even though the extensive background research in the matter and the Reference Class Forecasting technique available for use . Almost identical results are for demand data analysis, even though research in the field has only been for a 30 year sector.

When combining cost and demand data, this information is more often than not, used for a cost-benefit analysis of a specific project to deem if the project should move forward in the early stages of development to potential construction. However, if the demand and cost data is not accurate, the cost-benefit analysis is somewhat incorrect and possibly misleading. Not only does the cost-benefit ratio rely on the accuracy of the data, but so too does environmental and even socioeconomic factors. For this information to be misleading to such a degree will undoubtedly impact on the planning and policy-making of the given scheme in a negative light. By using these forms of information that have been proven to be consistently wrong over an extended period of time, it increases the risk of any project immensely.

Business Practise

Figure 3 : 50th and 80th percenticles of errors for various construction projects

In October of 2004, the first recorded use of an organization attempting to debias a situation occurred in Edinburgh, Scotland with the planning of the Edinburgh Tram line 2. The Scottish Parliament's Edinburgh tram Bill Committee employed Ove Arup and Partners Scotland as the caretakers of the project. The two enterprises were also employed to provide an extensive review of the construction of the Edinburgh Tram line 2 by Transport Initiatives Edinburgh ( A private company managed by the Edinburgh City Council to complete project promotion for the state on major transportation projects).

Through assessment of the upcoming construction of the Edinburgh Tram Line 2, Ove Arup initially priced the total plan at £255.0 million, carefully considering the Optimism bias costs of the build. Ove Arup also further implemented a 25% additional Optimism bias for total capital spent on the tram line, adding approximately £64.0 million to its initial budget compiling to £320.0 million as an estimated gross spend. When calculating the grand total of the business venture, Ove Arup incorporated Reference Forecasting to aid their judgement. The company, through extensive research into previous database costs, comparison to other similar Uk light rail plans and other related schemes added an additional £46.0 million to climax £320.0 million. Of that £46.0 million, Ove Arup estimated a need for £26.0 million for future development costs through renewals and also replacements of the system. A further £20.0 million was added to cover the potential of risks of revenue shortfalls. This additional £46.0 million increased the total budget by 14.4%, and this increase was based entirely on forecasting and attempting to reach bias neutrality.

During calculations, Ove Arup used variance and Standard Deviation for their proposed budget with the assistance of the UK Department for Transport. Ove Arup calculated the 80th percentile for total capital costs, the likelihood of staying within budget was 80%, at £400.0 million. The 50th percentile for staying in the budget for total capital costs, the likelihood was 50%, bringing the budget to £357.0 million. The combined companies all took a conservative stance on the financials involved because the Edinburgh Tram Line 2 had not yet reached any significant business case, potentially adding extra risks to the matter.

After completion of the Edinburgh Tram Line 2 scheme, the project and many aspects (i.e. the costs and time schedule) were deemed a great success and through the exposure of the Reference CLass Forecasting was implement in a further project, London’s £15 billion Crossrail scheme. Subsequently, through the success of projects in the United Kingdom, nowadays countries like Switzerland, Denmark and the Netherlands are heavily implementing these types of procedures.

Limitations

Although Reference Class Forecasting can largely be quite an accurate and effective way to limit the bias in a situation, sometimes it is virtually impossible to limit the biases even with the use of Reference Class Forecasting. A better illustration of this statement is an example of colossal-sized projects, like the Olympics. In terms of the management of sucha an event, with relation to Reference Class Forecasting, the issue for an organization undertaking such as high-status event, is that it is almost a certainty that it will the companies first time in charge of such a project. To further add, with many projects such as a light rail project or an expansion to a multi-lane highway, although these plans have a tight deadline date, if need be the company can usually extend construction time for whatever reason (although usually very ill-advised). An issue is that the Olympics (or an equivalent spectacle) is that it has a very strict deadline day as the Opening Ceremony and games are planned to be on specific dates and times and must be met with no alternatives. Not only does this make it extremely complex for the organizing committee, but to be completing such an event with no prior experience in such massive schemes, cost overrun is almost expected (there has not been an Olympic games that has met or been even close to its initial budget with an average of 51% cost overrun for the hosting country throughout recent history of the event).

With this knowledge, although the Olympics can be considered a very niche event, it can be demonstrated that it even with Reference Class Forecasting in place, tasks with such significance make it an increasingly difficult task to refer to Reference Class Forecasting as it is a first-time event for the country and organizations involved.

Alternatives

After extensive time and research observing both the Optimism bias and the Strategic misrepresentation, it can be noted that both methods undoubtedly rely heavily on Reference Class Forecasting for substantial adjustment and correction to minimalize bias.Through further readings on the topic of bias and possible alterations to theories and ideas already vastly researched, the Socratic method was revealed. Socrates was an ancient Greek philosopher and was said to be on of the finding fathers of Western philosophy. As many people in the ancient civilization of Greece said that he was the wisest humans alive, he insisted he did not understand anything. As a matter of fact, he questioned everything to better comprehend a specific situation, to eliminate gaps of uncertainty about a topic and acquire more background knowledge, improving his incite on the issue, however, not refuting the dilemma. This approach worked almost as an improvement mechanism for the matter of contention at hand, striving to get the most knowledge he could out of the situation. Socrates then further analyzed the given topic to make sense of the information presented and to fathom what he knew and what he did not of the subject before making any substantial judgements on the issue.

As demonstrated, the philosopher took a 'less biased' approach to a specific contention. Through questioning every aspect of the theory he better understood the theories flaws and where it needed amending, but also unintentionally decreased the skewness and bias of the situation by a much-improved view by encompassing all background information.

Relating this theory to the corporate world, if more background knowledge and further understanding of the issue or project is adhered to, then a potential increase in accuracy in costs for constructions and time schedules could be achieved easier as the boost to the comprehension of the situation would significantly reduce the bias in a likely problem.

Conclusion

Through studies conducted beginning in 1979, from 2002 Nobel Prize winner Daniel Kahneman in the field of economics and colleague Amos Tversky, the pair instigated research into biases in management and project portfolios. They better understood how people make judgements of a specific situation in and out if the corporate world. The two researchers found that bias, has two subcategories, optimism bias and strategic misrepresentation. Once a more succinct comprehension of the two brackets of bias were explored, further research of how to eliminate the effect were highly sort after especially in large organizations.

Although when further broken down, Optimism bias almost inevitably does exist no matter what the techniques and strategies to combat it are in place at the present time. Once an acceptance of this phenomenon is appreciated, then the business or individual can limit the impact it has on the company, the output of work or even one's self through potential capabilities of the Reference Class Forecasting System.

In terms of Strategic misrepresentation, this bias more often than not incurred when either a lack of knowledge and understanding of the data or information occurs, subsequently creating an avoidable set of skewed data, therefore almost certainly creating cost and demand data issues. Secondly, information can further be misrepresented if one party or organization has any incentive to alter result (i.e. the possibility of financial gain). Although both examples are quite damning, the mitigation of bias and the correct data analysis training, strategic misrepresentation is avoidable.

Professor Bent Flyvberg, in 2007 added to and finalized the works of both Kahneman and Tversky through Reference Class Forecasting. With the use of this theory, Flyvberg could better understand bias in situations and how to combat the skewness of the potential bias by attempting to severely decrease it from situations. With the use of strategic misrepresentation, although proved not to be immensely effective in all areas (i.e. cost and demand data models) it has demonstrated an increased success with decreased bias in the situation through the elimination of unknown factors and further background knowledge on a given field.

Annotated Bibliography

Flyvbjerg, B. (2006). From Nobel Prize to project management: getting risks right. Paper presented at PMI®

• Helped to better understand setbacks in Reference Class Forecasting
• Clearly described 'The Planning Fallacy'
• Showed how Reference Class Forecasting has been used in companies, both effective and ineffectively

Batselier, J. & Vanhoucke, M. (2016). Practical Application and Empirical Evaluation of Reference Class Forecasting for Project Management. Project Management Journal, 47(5), 36–51.

• Demonstrated Reference Class Forecasting and how to reduce bias out of a situation
• Depicted the effects of skewing data through Optimism bias and/or Strategic Misrepresentation
• Outlined Optimism bias, Strategic misrepresentation and hot it relates to Reference Class Forecasting

Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice

• Thoroughly showed how Optimism bias works and how you can potentially counteract the effect
• Showed how Strategic misrepresentation can affect a cost overrun and cost and data analysis
• Illustrate a clear resemblance between Reference Class Forecasting and the two sub-categories