Home

More Papers

Part 3:  Lack of the Right Metrics

Printable version

Metrics = incentives

The metrics that an organization uses for evaluating projects have a big impact not only on the projects that get chosen but also on the projects that get proposed. "Tell me how you will measure me, and I will tell you how I will behave" [1]. Even if the metrics aren't used to create incentives, managers interpret them as indicating what the organization regards as important. Lack of the right metrics is the third reason organizations choose the wrong projects.

Defining Project Value

The goal when choosing projects should be to maximize the total value to be derived from those projects. But, how do you measure project value? More fundamentally, how do you define value?

A Definition of Value

People have been arguing about the definition of value for centuries. The relevant concept of value for our purposes is termed the "utilitarian concept of value:" The value of a project is a measure of degree to which that project enables the organization to achieve its objectives. This view of value was first articulated in the fourth century BC by Aristotle —the value of something is not an intrinsic property of that thing, but rather is determined by its usefulness to those that want it.

Organizations conduct projects because they believe that the consequences or outcomes of the projects will be useful. Thus, we define the value of a project to be the worth, to the organization, of obtaining the consequences of the project. The project consequences that are desired depend, of course, on the organization's fundamental objectives. Suppose, for example, that an organization's fundamental objective is to create value for its shareholders (more discussion of shareholder value is provided below). Suppose further that the organization is considering a hypothetical project that provides only one consequence: an immediate, one-time cash flow to the company of one million dollars (to be more precise, suppose that the magnitude of the cash flow from the project is such that, after all tax and accounting considerations are addressed, increases the net worth of the company by $1 million). Under the specified assumptions, the value of that project would logically be $1 million.

Note that the value of a project does not depend on its cost. (The decision of whether to conduct a project, does, of course, depend on cost, where a measure such as net value—value minus cost—would be relevant.) An exception to the rule that project value does not depend on project cost would be a case where paying more for a project impacts the organization's ability to benefit from the project. An extreme example would be a project anticipated to produce great project outcomes, but whose cost would bankrupt the company. In such cases, the value of the project would logically be the value of the project consequences taking into account any effects of having to pay for the project. Since we are concerned with organizations that conduct numerous projects, each of which consumes only a portion of the budget, we can often ignore such effects.

Project value is what the organization would be willing to pay to obtain project consequences:  Project A is preferred to Project B if and only if Value[Project A] > Value[Project B]

To be precise, I define the value of a project to be the maximum amount the organization's leaders would be willing to spend in order to obtain the consequences of doing the project (including consideration of risks). In the literature on valuation, this value is termed a "buying indifference price"—it is a buying price because it is an amount the organization would pay to buy the project consequences, and it is an indifference price because it is the price point such the organization is indifferent between passing on the project versus paying the price and obtaining the project consequences.

Note that our definition of project value has an important property: An organization will prefer Project A to Project B if and only if our measure of project value is higher for Project A than it is for Project B. Many common approaches to project prioritization don't use a definition of value that has this essential property (see, for example, the discussion of strategic alignment). Unless project value is defined in a way that maps to the true preferences of the organization, that measure cannot correctly prioritize projects. Our definition of project value is a true measure of the relative attractiveness to the organization of its project alternatives.

How do we determine how much an organization would be willing to pay to obtain the anticipated consequences of conducting various projects? Before addressing that question we need to consider more carefully why organizations conduct projects.

Projects Determine the Evolution of the Business

The business of an organization is always evolving, and the projects that the organization chooses affect that evolution. For example, a new technology might become available that would allow the organization to reduce its costs. If a project is conducted to install the technology, the organization would incur lower operating costs than it would have had it chosen not to do the project.

It is also true that the projects that an organization chooses not to do affect the evolution of the business. For most organizations, standing still means falling behind. There are many reasons for this, including increasing competition, changing customer preferences, and the aging of organizational assets. Thus, to take one example, if the organization chooses not to do projects that maintain or replace aging assets, the service provided by those assets will decline.

Figure 18 illustrates a useful way to think about project value. At the point in time when an organization is considering a new project, it is really facing a choice between two alternative future states. If the project is conducted, that project will, presumably, transform the business to some more desirable state. If the project is not conducted, some other, presumably less-desirable, state will result.

Figure 18:   Project choices determine the future state of the business.

The perspective of Figure 18 provides a basis for computing project value. The value of a project is the difference between the values of the two potential future states of the business:

Some Observations Regarding Project Value

Based on Equation 1, we can conclude some important things about project value.

• To estimate project value, it is necessary to consider what would happen if the project is not conducted as well as what would happen if the project is conducted. Many project scoring systems ignore the implications of not doing projects, which causes some types of projects to be grossly undervalued.
• The same project can have different values to different organizations. For the purposes of estimating project value, therefore, it is not sufficient to consider just the characteristics of the project itself. It is also necessary to consider characteristics of the business that determine how useful that project will be to that business.
• The value of a project can change depending on what other projects are conducted. Because of synergies and economies of scale, for example, the costs and benefits of a project can change based on the other projects that are also conducted. If such dependencies exist, optimizing the project portfolio requires estimating the value of conducting different groupings of the interdependent projects.

For the purpose of identifying metrics for measuring project value, the most important implication of Equation 1 is this: Project value is the contribution that the project makes to the total value achieved by the organization. Because project value is the difference between the value that the organization attains in two alternative states, the methods for estimating project value are essentially the same as the methods for estimating organizational value. This is good news, since management scientists have devoted considerable effort to determining how best to measure the value created by organizations. We can use the concepts and methods that they have devised to enable us to quantify project value.

Inadequacy of Financial Metrics

So, how can you measure the value created by an organization? Interestingly, most businesses get it wrong.

Most businesses use financial metrics computed from cash flows to measure value, for example, return on investment (ROI), internal rate of return (IRR), net present value (NPV), payback period, etc. Using these metrics to evaluate candidate projects requires forecasting the cash flows that would be produced by the project. Some impacts on cash flow may be relative easy to forecast, in particular, the costs to conduct the project and any direct impacts the project will have on the firm's future costs and revenues. However, it is difficult to translate many project consequences into impacts on cash flow. For example, how would a project designed to collect better information about customer preferences impact future cash flows? From a practical standpoint, cash flow analysis will not capture many project impacts.

Thus, at best, financial metrics provide only a partial representation of what is important to a business. According to a study by Research Technology Management, companies that rely mostly on financial metrics obtain "unbalanced portfolios" that are not well matched to the strategy of the firm [2].

The limitations of financial metrics are even more obvious when it comes to evaluating projects in the public sector. Public-sector organizations have social missions and may not even sell goods and services that generate cash flows. Even it they do, earning a financial return may not be a primary objective. For example, a water utility has a mission that includes serving community water needs. A public school has a mission that includes educating students. Cash flow analysis will not do a good job of indicating how well these organizations are accomplishing their missions. Financial metrics fail to measure the value of projects intended to achieve non-financial objectives.

Two Views on Organizational Value

If financial analysis is inadequate, how then can we measure value? There are two competing views on how to measure the value created by an organization, based on the answer to the question, "Value to whom?" The first view is that the purpose of organizations is to produce value for owners (shareholder value). The second view is that the purpose of organizations is to create value not just for owners, but for other stakeholders as well (stakeholder value).

Shareholder Value

As indicated above, most management scientists and U.S. business leaders argue that the fundamental objective of investor-owned organizations is to maximize value for the owners of the business. In the case of a publicly traded company with one type of stock, shareholder value is roughly the number of outstanding shares multiplied by current share price. Dividends, of course, increase the return to shareholders while simultaneously decreasing share price. For both theoretical and practical reasons, increasing market value can be argued to be the ultimate goal of many private-sector businesses.

One argument for shareholder value is the objectivity of market valuations. Market prices are objective and applied consistently across all publicly held companies and markets. If they were not there would be opportunities for arbitrage; that is, one could buy shares of a company at a lower price in one market and simultaneously sell them at a higher price in another market.

Irrespective of theory, seeking higher stock value is clearly a practical objective for senior management. Compensation for top executives is often tied to stock performance. Poor stock performance is a common reason for a company's board of directors to consider replacing its chief executive. A company whose stock is depressed because it is not effectively managing for shareholder value is a candidate for investor takeover.

It is important to understand that managing for shareholder value and managing for profitable cash flows are two very different things. The figure below compares the market values and risk-adjusted discounted value (NPV) of projected future earnings for four companies in the same industry. In each case, the discount rate for computing the NPV is the company's weighted average cost of capital.

Figure 19:   Market value is not the NPV of projected earnings (data for 4 companies in same industry).

As illustrated, a company's market value is usually higher than the NPV of its earnings. In the literature on real options, the additional value is termed "option value." In addition to projected future cash flows, market value depends on how buyers and sellers in the marketplace perceive, among other things, the ability of the business to obtain and respond to future opportunities and avoid future threats.

Also as shown in the figure, it is not uncommon to find two companies in the same industry such that one has a higher NPV of projected returns but the other has a higher total market value. Indeed, some "dot com" companies have substantial market values yet cannot convincingly forecast any profits. Option value does eventually translate into earnings. However, because it is generally difficult or impossible to forecast the exact mechanisms by which this will occur, option value and other indirect sources of value will not be represented in accounting forecasts of cash flows.

The implication of the above is that impacting cash flows is only one of several paths by which projects can influence shareholder value. The alternative paths that may be relevant depend on the nature of the organization's business. For example, a project that generates new knowledge that improves consistency of execution can reduce investment risk. This can have the effect of lowering the cost of capital and increasing the returns on the capital invested. Investors seeking investments likely to increase in value will buy more shares, which will increase shareholder value. As another example, a service company might conduct projects that enable it to meet commitments or promises to customers. This improves customer perceptions and can influence the views and reports of Wall Street analysts, resulting in higher stock prices.

According to the perspective of shareholder value, the appropriate metrics for evaluating projects in the private sector are those relevant to forecasting impacts on the market value of the business. Real options analysis and related methods are designed to do this. Companies that evaluate projects by estimating impacts on profits alone ignore a significant component of market value.

Stakeholder Value

The alternative view, more popular outside the U.S., is that firms should create value for all stakeholders, not just shareholders. Stakeholders are groups of people who have an interest in the business or organization (Figure 20). Typically, important stakeholders are those whose actions can have important impact on the success of the organization. Stakeholder value includes value for employees, suppliers, customers, local community, and society at large.

Figure 20:   Some potentially important stakeholders.

Proponents of more narrow shareholder value argue that organizations pursuing self-interest and economic efficiency do, in fact, serve all stakeholders. However, this argument carries the implicit assumption that the markets within which the firm operates are "perfect," so that stakeholders other than shareholders are unharmed by the firm's focus on benefiting shareholders alone. For example, if employees are laid off, they can immediately get equivalent jobs elsewhere. Similarly, if suppliers or consumers are mistreated, they can take their business to other firms. If there is any impact on stakeholders, the argument goes, it is beneficial. The most successful companies create the products and services that customers want most at the lowest price. Their rising stocks attract more investors, which enables them to satisfy even more customers and hire more employees. The tax system, the argument goes, ensures that successful companies create tax revenues for social ends.

The U.S. economy is well diversified, and many have argued that this makes the "perfect market" assumption a pretty good approximation. However, well-publicized accounting and investment scandals, declines in the U.S. housing market, volatile oil prices, and the global environmental impacts of carbon emissions are raising new questions about the the wisdom and acceptability of absolute reliance on shareholder value. Some critics of shareholder value have claimed that the current recession was largely caused by excessive risk taking fueled by shareholder demand for ever higher levels of short-term gain. Still, defenders of shareholder value have responded by blaming the "principal-agent problem"—the providers of capital and the manager-agents they employ to make decisions inevitably have different incentives and information. The best way to serve the long-term investor, they say, is to tie decisions more closely to shareholder value.

Regardless of one's stance in the debate on shareholder versus stakeholder value, prudence suggests that private-sector companies should give consideration to impacts on stakeholders when evaluating projects. Companies in regulated industries are often required to serve the interests of customers, citizens, and other stakeholders. In any case, it would be foolish for a company to ignore significant concerns of its stakeholders. Recognizing the importance of stakeholder value further weakens the argument for using financial metrics as the sole basis for project prioritization.

Stakeholder value is almost certainly the more appropriate definition of value for non-profit and public-sector organizations. However, even those companies that care only about shareholders need to consider how their actions affect other stakeholders. If they don't, they may fail to conduct projects needed to prevent adversely impacting the perceptions and attitudes of workers, customers, regulators, and local citizens and/or public officials. Careful investors factor considerations such as employee relations, customer satisfaction, regulator attitudes, and community perceptions into their investment decisions. Thus, significant impacts on stakeholders inevitably affect shareholder value.

As is the case with shareholder value, methods are available for quantifying stakeholder value. The appropriate project valuation metrics are those that result from applying these methods. As shown in the next sub-section, metrics may be found by constructing a project-selection decision model that evaluates projects and project portfolios based on the goal of maximizing organizational value (shareholder or stakeholder value).

Improving Prioritization and Part 2 References

Project-Selection Decision Models