Anyone who causes harm by forecasting should be treated as either a fool or a liar. Some forecasters cause more damage to society than criminals.
—Nassim Taleb, The Black Swan, 2007
In 1957, the Sydney Opera House was expected to cost $7 million and to take 4 years to be carried out. Finally, it took 14 years to be completed for around $102 million, resulting in a cost overrun of 1,400%! Probably the highest in the whole History.
Cost overruns and delays constitute a constant in engineering works: the Panama Canal, the new headquarters for the European Central Bank, the M-30 tunnels, the Madrid-Barcelona high-speed train, the Pajares turn-off (variante de Pajares), the Barajas Airport T4, the L9 of the Barcelona metro… It is an endless list. After having studied hundreds of works in 20 countries over the last 70 years, Bent Flyvbjerg −Infrastructure Policy Expert and Professor at Oxford University−, concluded that 90% of the projects were not able to meet the budget.
And this does not happen only with big civil works, but also with major engineering projects: the A400M Airbus, the Australian F-100, or the Galileo project intended to replace the GPS, that has accrued 10.000 EUR million in terms of cost overruns and a 15-year delay. Imagine!
What about your projects? Are they more resource-intensive than it was originally budgeted? Do they take longer than expected to be completed?
Why we are so bad estimating costs and deadlines: the tension between inside and outside views
Throw the first stone if you have never been over-optimistic when estimating the time and resource costs needed to complete something!
The psychologists Daniel Kahneman and Amos Tversky coined the term planning fallacy to refer to this paradoxical phenomenon: although all of us have miserably failed time and again when estimating the completion time and execution budget of all type of personal and professional projects where we have been involved, the next time our planning remains absurdly optimistic!
In his book Thinking, Fast and Slow, Daniel Kahneman explains how you can address a problem by using an inside view or an outside view:
- When you use the inside view, you focus on your specific circumstances and search for evidence in your similar experiences, including anecdotal evidences and fallacious perceptions. You imagine idyllic scenarios where everything goes according to plan. Nevertheless, in addition to all the variables that you consider on your mind, there is an incalculable volume of unknown and undeterminable variables: diseases, accidents, security incidents, breakdowns, disagreements within a team, other projects that are more relevant, firings, unforeseen exhausted funds, strikes… so many unexpected things may happen! Unfortunately, it is impossible to anticipate all of them. What is clear is that the likelihood that something goes wrong increases as the project is more ambitious. In summary, we underestimate what we don’t know.
- Conversely, when you use the outside view you are looking beyond yourself —beyond your own experience, beyond the details of a particular case— and paying attention to additional objective information, such as the base rate. Instead of thinking that your project is unique, you look for similar projects and check their data on average budget and duration.
Unfortunately, you tend to discard, even to scorn, the outside vision. When statistical information of similar cases comes into conflict with your personal feelings, you will reject the data. After all, those projects were so long and had such a high cost because other people executed them, but you are different and the same thing might not happen to you, right?
Let’s see it from a different angle, thanks to this piece of dialogue from a real conversation between two parents:
“I’m going to enroll my son at the TAI School.”
“Are you sure? It’s really elitist. The probability to be admitted is quite low, only 8%.”
“Well, the probability of my children will be close to 100%, his grades are excellent.”
This father thinks that his son has a very great probability to be admitted because, indeed, the child is quite intelligent and his grades are excellent… as the hundreds of children that apply for this school every year, of whom only 8 out of 100 are admitted. This father is only using its inside view to evaluate the situation. Sadly, the best probability estimation that his son will be admitted to that school remains 8%, that is, the base rate.
To sum up, the outside view looks for similar situations that may provide a statistical basis to make a decision: Have other people faced similar problems in the past? What did it happen? Since we consider the outside view is an unnatural way of think, we prefer to believe that our situation is unique.
According to Kahneman in Thinking, Fast and Slow:
In the competition with the inside view, the outside view doesn’t stand a chance
Welcome to Lake Wobegon, where all the children are above the average
Acknowledge that your perception of events is asymmetrical. According to a high number of psychology experiments:
- You attribute your successes to your skills and your failures to your bad luck, exactly the opposite of how you apprehend successes and failures of others.
- You consider yourself more intelligent than others.
- You think you are a better driver than others.
- You think your marriage will last longer than others.
In essence, you think you are special. Well, this is another thought error known as optimism bias. This bias is closely related to the illusory superiority. In other words, we believe within ourselves that we are different, when actually we are all quite similar.
The optimism bias plays a critical role in the planning fallacy. As explained by the expert Bent Flyvbjerg in his study mentioned above:
“In the grip of the planning fallacy, managers make decisions based on delusional optimism rather than on a rational weighting of gains, losses, and probabilities. They overestimate benefits and underestimate costs. They involuntarily spin scenarios of success and overlook the potential for mistakes and miscalculations. As a result, managers pursue initiatives that are unlikely to come in on budget or on time, or to deliver the expected returns.”
Not everything is our imperfect thought’s fault
The optimism of planners and those who make decisions is not the only cause of these increases. In Thinking, Fast and Slow Kahneman points out other factors:
“Errors in the initial budget are not always innocent. The authors of unrealistic plans are often driven by the desire to get the plan approved—whether by their superiors or by a client—supported by the knowledge that projects are rarely abandoned unfinished merely because of overruns in costs or completion times.”
In order to obtain funding at all costs, overstating the returns of the coming project and easing the risks and costs is frequent. In such a way, Bent Flyvbjerg provides a formula that stresses this pernicious spiral where the projects that are best presented are the most funded ones:
Underestimated costs + Overestimated benefits = Funding
At the end, people don’t fund real projects, but idealized versions of them with oversized returns, while costs and problems are swept under the rug.
What you can do to better plan your next project
Statistical information on similar projects may save us from the planning fallacy. However optimistic you are, you may expect to find, on average, similar bottlenecks to those faced by similar projects’ teams. Bent Flyvbjerg suggested the method known as Reference class forecasting to set the optimism bias aside when forecasting projects’ completion time and cost.
Reference class forecasting requires the following three steps for your particular project:
- Identify a relevant reference class of past projects. The class must be wide enough to be statistically significant, but also restricted enough to be truly compared with your project. Naturally, the rarer the problem you are dealing with, the more difficult this step will be. In case of common decisions (at least for others, although not for you), reference class identification is immediate.
- Establish a probability distribution for the selected reference class. This requires access to credible empirical data for a sufficient number of projects within the reference class in order to obtain statistically significant conclusions.
- Compare your project with the reference class distribution, in order to establish the most likely outcome for your project.
When evaluating plans and forecasting, we tend to focus on what is different, leaving behind that the best decisions often focus on identical factors. Even if the situation you are dealing with seems to be a little different, at the moment of truth it is almost always the same. However painful it may be, let’s assume we are not as special as we think.
Learn from ancient stoic philosophers to think about what may go wrong: project premortem analysis
It is attributed to Seneca the meditation exercise that Stoics called premeditatio malorum: something like a reflection on what may go wrong before undertaking an enterprise. For instance, before undertaking a sea journey, what may go wrong? A storm may break, the captain may get sick, the ship may be attacked by pirates, etc. In this way, you may get yourself ready for these eventualities. And in case nothing can be done, if the worst happens, at least it will not catch you by surprise.
The American psychologist Gary Klein takes this old idea up and recommends performing a premortem analysis of the project under study:
“A premortem in a business setting comes at the beginning of a project rather than the end, so that the project can be improved rather than autopsied. Unlike a typical critiquing session, in which project team members are asked what might go wrong, the premortem operates on the assumption that the “patient” has died, and so asks what did go wrong. The team members’ task is to generate plausible reasons for the project’s failure.”
If you wish to apply this analysis, try to give the following speech to your team:
“Imagine that a year has passed. We have followed the project to the letter. The project has failed spectacularly. Over the next few minutes those in the room independently write down every reason they can think of for the failure—especially the kinds of things they ordinarily wouldn’t mention as potential problems.”
The reasons that have been pointed out will show you how things might happen. After all, a premortem may be the best way to avoid a painful postmortem.
Gonzalo Álvarez Marañón
@gonalvmar
Innovation and Labs (ElevenPaths)
www.elevenpaths.com
The post Why you are late delivering all your projects and what you can do to address it appeared first on Think Big.