Figure 3: People systematically over- or under-estimate probabilities.
Overconfidence
Overconfidence is another powerful bias. We believe we are more accurate at making estimates than we are. I've often repeated a well-known demonstration to illustrate what I
call the "2/50 rule." People are asked to provide confidence intervals within which they are "98% sure" that various uncertain quantities lie. The quantities for the questions are selected from an
Almanac, for example, "What's the elevation of the highest mountain in Texas?" "Give me low and high values within which you are 98% sure that the actual value falls." When the true value is
checked, up to 50% of the time it falls outside of the specified confidence intervals. If people
were not overconfident, values outside their 98% confidence ranges would occur only 2% of the time.
Overconfidence is also demonstrated by the many examples of people expressing confidence about things that are subsequently proved wrong. For example, British Mathematician Lord
Kelvin said, "Heavier-than-air flying machines are impossible." Thomas Watson, Chairman of
IBM, reportedly said, "I think there is a world market for about five computers." Similarly, surveys show that most drivers report that they are better than average, and the most companies believe their brands
to be of "above-average" value.
Anchoring
Another relevant bias is anchoring. Initial impressions become reference points that anchor subsequent thoughts and judgments. For example, if a salesperson attempts to forecast
next year sales by looking at sales in the previous year, the old numbers become anchors, which the salesperson then adjusts based on other factors. The adjustments are usually insufficient.
Dramatic or easy-to-recall events often become strong anchors. For example, the vividness of the horrible events of September 11 caused many to view airline travel as too risky, but
many experts believe that travel has never been safer.
Motivational Biases
Various motivational biases also lead to forecasting errors. The nature of the effect can depend on the individual. For example, project managers who are anxious to be perceived as
successful may pad cost and schedule estimates to reduce the likelihood that they fail to achieve expectations. On the other hand, project managers who want to be regarded (consciously or
unconsciously) as high-performers may underestimate the required work and set unrealistic goals. Most managers are overly optimistic. For example, when companies collect data on the financial returns
from projects, they almost always find that actual returns are well-below forecasted returns.
Base-Rate Bias
Base-rate bias refers to the tendency people have to ignore relevant statistic data when estimating likelihoods. For example, most people believe they are more likely to die from a terrorist
attack than from colon cancer, even though statistics indicate otherwise. Variations of this bias are
important in business environments, including the tendency people have to be insufficiently conservative (or "regressive") when making predictions based on events that are partially random.
Investors, for example, often expect a company that has just experienced record profits to earn as much or more the next year, even if there have been no changes in products or other elements of the
business that would explain the recent, better-than-anticipated performance.
The tendency to underestimate the effort needed to complete a complex task has been attributed to base-rate bias. Instead of basing estimates mostly on the amount of time it has
taken to do previous similar projects, managers typically take an "internal view" of the current project, thinking only about the tasks and scenarios leading to successful completion. This almost
always leads to overly optimistic forecasts. One manager I know says he always multiplies the time his programmers say will be required to complete new software "by a factor of two," because "that's
what usually happens."
Small Sample and Conjunctive Bias
Small sample bias is another example of inaccurate statistical reasoning—people draw conclusions from a small sample of observations despite the fact that random variations
mean that such samples have little real predictive power. Conjunctive events bias refers to the tendency for events that occur in conjunction with one another to make a result appear more likely. For
example, the possibility that you may die during a vacation (due to any cause) must be more likely than the possibility that you will die on vacation as a result of a terrorist attack. Yet, one study
showed that people are willing to pay more for an insurance policy that awards benefits in the event of death due to terrorism than one that awards benefits based on death due to any cause.
Lack of Feedback
Forecasting errors are often attributed to the fact that most people don't get good feedback about the accuracy of their forecasts. We are all fairly good at estimating physical
characteristics like volume, distance, and weight because we frequently make such estimates and get quick feedback about our accuracy. We are less experienced (and get less verification) when making
forecasts for things that are more uncertain. Weather forecasters and bookmakers have opportunities and incentives to maintain records of their judgments and see when they have been inaccurate. Studies show that they do
well in estimating the accuracy of their predictions.
Advice for improving forecasts and estimates includes:
- Think about the problem on your own before consulting others and getting anchored to their biases.
- Be open-minded and receptive. Seek opinions from multiple and diverse sources. Tell them as little as possible about your own ideas beforehand.
- Tell people you want "realistic" estimates. Ask about implicit assumptions.
- Encourage the estimation of a range of possibilities instead of just a point estimate. Ask for low and high values first (rather than for a middle or best-guess value) so as to create
extreme-valued anchors that counteract the tendency toward overconfidence around a middle value.
- Require project proponents to identify reasons why what they propose might fail.
- Give people who provide you with estimates knowledge of results as quickly as possible.
- Use network diagrams, scenario building, and similar techniques to identify and define the sequencing of component activities. A major value of such techniques is that they help reduce the
likelihood that some necessary, but less visible, activities, such as procurement and training, aren't overlooked.
- Routinely use logic to check estimates. As a simple example, if you have 2 months to complete a project estimated to require 2000 hours, verify that you have a sufficient number of FTE's
available.
|
Printable
version
Heuristics & Judgmental Biases