uncertainty Archives - Markus Schwarzer

This post investigates the impact of Optimism Biases on Capital projects. And how to ensure positive outcomes are more likely.

Capital Project Underdelivery

Most of us would have seen capital projects behaving as below.
The original business case promised much. The actual (under) performance showed in real-time.

Progress is slow and ends slowly, with acceleration in the middle periods. Underperforming capital projects are stretching time and costs. Pressure on cash flows is guaranteed.

Too Many Uncertainties

Capital projects do change frequently in scope. From idea to reality during the project cycle. Uncertainty dominates the early project stages.
Technical specifications, cost estimates, procurement delays are common.
The early project stages are often testing grounds.

And Time Passes

The real test of project performance is how much time and effort it takes to correct started processes.
The more time passes, the harder it becomes to make steering adjustments. More and more resources are needed to rectify problems.
Incremental changes increase uncertainty. At times uncertainties are simply ignored. Systemic underreporting becomes reality.

The Iron Law

Significant cost and schedule overruns become normal.
Project leaders and investors are facing underperforming capital projects. 50% delays in project execution are common.¹

Research has shown this lousy underperformance is so consistent that the project management expert Bent Flyvbjerg coined it the iron law of megaprojects: capital projects are over budget, over time, over and over again.

“Such ignorance or underestimation of risks is often called optimism.” ²

Capital Projects Uncertainty

Uncertainty is not ‘unknown’ yet often not fully recognized.
The problem arises from too much optimism at the planning stage.
Overconfidence develops, unintentionally.
The Optimism Bias creeps in.

What is Optimism Bias?

The tendency to be ‘overly positive’ during forecasting and planning activities:³

“One puzzle of optimism is thus that people maintain overly positive expectations despite a lifetime of experience with Reality.”⁴

Although we are aware that divorce rates are nearing 50% in the Western World, couples who are about to get married estimate their own likelihood of divorce as negligible.

Research supports that most of us show an optimistic bias that arises in a variety of circumstances.

“People tend to underestimate how long a project will take to complete and how much it will cost”⁵

Unrealistic Expectation

Some economists blame optimism bias for the financial downfall of 2008. Unrealistic expectations of financial analysts and government officials. A belief that the market would continue to grow. Despite evidence to the contrary, unfounded optimism contributed to the collapse.

There appears to be a correlation between uncertainty and optimism bias.
We display the largest bias in situations with the greatest unknowns.

Risk assessment

One way to prevent capital project underperformance is to strengthen the risk assessment process. Improve the quantitative risk analysis.

Most quantitative risk analysis is based on expert knowledge.
Expert knowledge and judgment can be prone to optimism bias. Particularly, if based on the ‘inside’ view of assessing risk.⁶

Inside View

The Inside View involves making predictions based on your in-house understanding of the details of the process.

This inside view is prone to error. Relying purely on in-house expertise to predict the future of the project is risky.

Outside View

An Outside View ignores in-house knowledge. It utilizes estimates based on a class of similar previous cases.

This is called reference class forecasting.

It is a systematic control mechanism. To mitigate the impact of time and cost overruns on projects.⁷

What is Reference Class Forecasting (RCF)?

Reference class forecasting is a method for systematically taking an outside view of planned actions.

Nobel Prize winners Daniel Kahneman and Amos Tversky developed the theories behind reference class forecasting.
RCF reduces the optimism bias in cost and time to forecast capital projects with more confidence in success.

It is a way of predicting the future. RCF examines similar past situations and their outcomes.

Reference class forecasting…” essentially ignore the details of a case at hand and do not attempt any detailed forecasting of the case’s future. Instead, they focus on the performance of a reference class of cases chosen because they are similar to the one proposed.” ⁸

How to Reference Class Forecast?

RCF follows three basic steps:

Identify a group of past, similar projects. 20-30 projects are enough to get started. The more projects the better
Establish the risk of the variable in question based on these projects. Relate directly to the cost overruns of these projects
Adjust the current estimate. Check if the current project is (more or less) riskier than projects in the reference class. Resulting in an adjusted uplift factor.

Alert: Unless the review is based on objective evidence, optimism bias might creep back into these estimates.

Most of the underestimation of costs and overestimation of benefits of capital projects is the result of people taking what’s called an “inside view” of their forecasts.

Benefits of Reference Class Forecasting

RCF replaces (biased) assumptions. It assumes adverse events and uncertainty in estimates, with data.
In principle, RCF assumes that a capital project performs no better or worse than past similar projects.

Although, the adjustments could potentially re-introduce the bias of assumptions back into the forecast.
The key is to base assumptions on strong evidence.

Compare Cash Flows

The system generates data and graphs based on past capital works cashflows.
All current and past class projects submit their actual cashflows.
To date and planned cashflows for the remainder of the project.
Target this data to identify any unusual patterns. Probe for weaknesses in the assumptions.
There are 3 types of patterns that then trigger different follow-up actions.

Green Zone

The project cash flow is in the green zone. No follow-up action for projects with ‘normal’ performance. The data of these projects will be stored for future use.

Amber Zone

Raises an alert. Amber project cash flow requires deeper assessments. There is the potential risk of cost overrun and program delay.
Resulting in taking appropriate recovery measures for the planning and implementation phases.

Red Zone

Flags a strong warning. Cashflows within the red zone force the project team to stop, think and carry out a major review.

In Summary

Overtly optimistic capital project estimates lead certainly to underperformance. And disappointment for investors.

Optimism bias severely impairs decision-making.

Project managers and sponsors must tackle problems early.
Seek efficient, effective, and timely intervention.
Risky projects need tackling before things have gone wrong and require substantial costly remedies.
Monitoring tools and early-warning -sign systems can help in project supervision.

Increased deep, rational thinking. Hard, tested data. Reference class forecasting. All help to reduce optimism bias.

(1) De-bias early project estimates in a data-driven way

(2) Set out strong contingency that explicitly considers the risk appetite of decision-makers, and

(3) Test the robustness of the reference class forecast through international benchmarking.

Sources
1 AI in Action, Oxford Global Projects, August 2022

2 Flyvbjerg, Bent. (2011). Over Budget, Over Time, Over and Over Again: Managing Major Projects.

3 Siemiatycki, Matti. “Managing Optimism Biases in the Delivery of Large-Infrastructure Projects: A Corporate Performance Benchmarking Approach.” 2008 First International Conference on Infrastructure Systems and Services: Building Networks for a Brighter Future

4,5 Sharot, T. (2011). The optimism bias. 

6 The planning fallacy: why we underestimate how long a task will take

7 B Flyvbjerg and A Budzier, Report for the Edinburgh Tram Inquiry, February 2018

8 McKinsey on Finance, Better forecasting for large capital projects

Often capital projects run over schedule, over budget and under the financial forceast.

The promised value is elusive. Obviously, some decisons were not as good as they could have been.

Bolster Decision making with the 4 Vs

Dr James Abdey developed the 4 Vs method. A neat thinking tool.
It helps me on my decision making path. An excellent process to start one’s own decision process. It gets the thinking into starting blocks. And it works well in the capital project space.

Is Data is the new Oil?

Clive Humby coined the phrase “Data is the new oil”.
Oil has no intrinsic value per se. It only becomes valuable when converted into materials. Such as petrol, jet fuel or plastics.
Although, one might argue oil is a commodity and data an asset.
Data needs to back and boost good decision making.

Purpose of Data

Generally, data helps us to back up our decisions. We use it to monitor, review and analyse:

trends
patterns
rate of change
outliers – which can be the story or a red herring

4 Vs

James Abdey recommends the 4 Vs to inspire data-based decision making:

One V builds onto the next V into a decision funnel. Helping to gain a true 360° data view of the situation.

1 Volume

Often we are overwhelmed with the sheer volume of data. It is daunting to walk through the data jungle. We want to cut out the noise. Disregard the irrelevant. Home in on the essential data.
Dismiss distractions.
Experience tells us. Evidently, decision making without the correct data is likely to deliver unsatisfactory results.

2 Velocity

During the initial diagnostic process, data keeps changing like sand dunes.
Data generation and collection speed are critical to reach meaningful conclusions.
Static data remains normally unchanged e.g., an operating manual for a capital item such as a truck. Dynamic data is in constant flux. Such as operational data e.g., mileage, fuel use, breakdowns etc.
This might ring a bell for spreadsheet users. Spreadsheets are great for static data sets. However, capturing changing data becomes increasingly more complex and challenging to maintain.

3 Variety

Data can be collected from a variety of diverse sources in order to answer specific questions. A truck normally has an in-built, custom-made information system that relates directly to vehicle related data (e.g., fuel efficiencies).
Combined with an fleet tracking GPS device can capture specific asset (driving analytics, e.g., road user charges) related data.
To be relevant in the decision-making process, data needs to be fit for purpose.

4 Value

Data is a precious business asset. Simply put, it costs less to ensure quality inputs than it does to correct flawed outputs.
For example: manufacturers can cut maintenance costs and minimize downtime with the use of real-time sensor data and predictive analytics. Through anticipating equipment failure, one can plan timely repairs or replacements.
Vibration Analysis (VA) is a good example. A powerful diagnostic tool that can detect current and future equipment failure. VA compares normal versus irregular vibration.
Just like a doctor listen to the sounds coming out of our chest. VA data can diagnose the ‘health’ of a machine by ‘listening’ to the vibrations that it generates.
Data needs to increase the probability of success and boost decision-making value.

Data Integrity

Given the current environment of heightened uncertainty, it is vital that data can be trusted.
Data source reliability and data precision are fundamental for the analytical process.
How trustworthy is the accumulated data of an asset through its life cycle? Error checking assists in preserving the validity and accuracy of data. Seeking affirmation is a universal method for ensuring data integrity.

Data Communication

Data tells stories. Stories help us to bridge the gap from the unknown to the known. Consquently, it guides us to fact-based decision-making.
Communicating data in simple language bolsters the interpretation of data and our decison making.

Data Visualisation

Given these points, data visuals (charts, graphs and maps) are useful tools.
This is something I use all the time.
The old saying ‘A picture paints a thousand words’ applies.
Visuals paint the trends, outliers, and patterns.

Another great Project Tool

Another great way to improve decision making before making commitments. Try the Project Pre-Mortem.

Credits
Clive Humby, Chief Data Scientist of the consumer insights company Starcount 
James Abdey, Associate Academic Director, The London School of Economics
and Political Science

Markus Schwarzer

Markus Schwarzer

Office Location

Phone Number

Email Address