Home

More Papers

Part 3:  Lack of the Right Metrics

Printable version

The metrics that an organization uses for evaluating projects have a big impact not only on the projects that get chosen but also on the projects that get proposed. "Tell me how you will measure me, and I will tell you how I will behave" [1]. Even if the metrics aren't used to create incentives, managers interpret them as indicating what the organization regards as important. Lack of the right metrics is the third reason organizations choose the wrong projects.

Defining Project Value

The goal when choosing projects should be to maximize the total value to be derived from those projects. But, how do you measure project value? More fundamentally, how do you define value?

A Definition of Value

People have been arguing about the definition of value for centuries. The relevant concept of value for our purposes is termed the "utilitarian concept of value:" The value of a project is a measure of degree to which that project enables the organization to achieve its objectives. This view of value was first articulated in the fourth century BC by Aristotle —The value of something is not an intrinsic property of that thing, but rather is determined by its usefulness to those that want it.

Organizations conduct projects because they believe that the consequences or results of the projects will be useful. Thus, we define the value of a project to be the worth, to the organization, of obtaining the consequences of the project. Suppose, for example, that an organization is considering a hypothetical project that provides only one consequence: an immediate, one-time cash flow to the company of one million dollars (to be more precise, suppose that the magnitude of the cash flow from the project is such that, after all tax and accounting considerations are taken into account, increases the net worth of the company by $1 million). The value of that project would be $1 million.

More precisely, we define the value of a project to be the maximum amount the organization would be willing to give up, in dollars, in order to obtain the anticipated consequences of doing the project (including consideration of risks).

Note that our definition of project value has an important property: An organization will prefer Project A to Project B if and only if our measure of project value is higher for Project A than it is for Project B. Many common approaches to project prioritization don't use a definition of value that has this essential property (see, for example, the discussion of strategic alignment). Unless project value is measured in a way that maps to the real preferences of the organization, that measure cannot correctly prioritize projects. Our definition of project value is a true measure of the relative attractiveness to the organization of its project alternatives.

How do we determine how much an organization would be willing to give up in order to obtain the anticipated consequences of conducting various projects? One approach would be to simply ask the organization's senior executives. However, answering such questions would be very difficult, and it would be hard to get consistent estimates. A better approach is to use analysis. Various theories and associated models exist for determining how much decision makers should be willing to pay for something, assuming certain assumptions apply. For example, the discounted cash flow model computes a current value for a project that generates a time stream of future payments based on discounting. The discounted cash flow model can be derived from two assumptions: (1) that investors wish to maximize their wealth and (2) that a market exists that will provide a return from investing cash. Other valuation theories exist for valuing projects that produce non-financial as well as financial project consequences.

Before further discussing how to measure project value, we need to consider more carefully what the consequences are of doing projects.

Project Value is the Difference Between the Value of Doing and Not Doing the Project

The business of an organization is always evolving, and the projects that the organization chooses affect that evolution. For example, a new technology might become available that would allow the organization to reduce its costs. If a project is conducted to install the technology, the organization would incur lower operating costs than it would have had it chosen not to do the project.

It is also true that the projects that an organization chooses not to do also affect the evolution of the business. For most organizations, standing still means falling behind. There are many reasons for this, including increasing competition, changing customer preferences, and the aging of organizational assets. Thus, to take one example, if the organization chooses not to do projects that maintain or replace aging assets, the service provided by those assets will decline.

Figure 18 illustrates a useful way to think about project value. At the point in time when an organization is considering a new project, it is really facing a choice between two alternative future states. If the project is conducted, that project will, presumably, transform the business to some improved state. If the project is not conducted, some alternative state will result.

Figure 18:   Project choices determine the future state of the business.

The perspective of Figure 18 provides a basis for computing project value. The value of a project is the difference between the values of the two potential future states of the business:

Some Observations Regarding Project Value

Based on Equation 1, we can conclude some important things about project value.

• To estimate project value, it is necessary to consider what would happen if the project is not conducted as well as what would happen if the project is conducted. Many project scoring systems ignore the implications of not doing projects, which causes some types of projects to be grossly undervalued.
• The same project can have different values to different organizations. For the purposes of estimating project value, therefore, it is not sufficient to consider just the characteristics of the project itself. It is also necessary to consider characteristics of the business that determine how useful that project will be to that business.
• The value of a project can change depending on what other projects are conducted. Because of synergies and economies of scale, for example, the costs and benefits of a project can change based on the other projects that are also conducted. If such dependencies exist, optimizing the project portfolio requires estimating the value of conducting different groupings of the interdependent projects.

For the purpose of deriving metrics, the most important implication of Equation 1 is this: Project value is the contribution that the project makes to the total value created by the organization. Because project value is the difference between the value that the organization attains in two alternative states, the methods for estimating project value are essentially the same as the methods for estimating organizational value. This is good news, since management scientists have devoted considerable effort to determining how best to measure the value created by organizations. We can use the concepts and methods that they have devised to enable us to quantify project value.

Inadequacy of Financial Metrics

So, how can you measure the value created by an organization? Interestingly, most businesses get it wrong.

Most organizations use financial metrics to measure value, for example, return on investment (ROI), internal rate of return (IRR), net present value (NPV), pay-back period, etc. Using these metrics to evaluate candidate projects requires forecasting all of the ways that a project can impact cash flows, which is often very difficult to do. The biggest limitation of using such metrics for selecting projects, however, is that they provide, at best, only a partial representation of what is important. According to a study by Research Technology Management, companies that rely mostly on financial metrics obtain "unbalanced portfolios" that are not well matched to the strategy of the firm [2].

Financial metrics, quite simply, don't capture all of the organization's true objectives. For public-sector organizations, this limitation is obvious. Public-sector organizations have non-financial objectives such as protecting public health and the environment, as well as mission-specific objectives. For example, a water utility has a mission that includes serving community water needs. A public school has a mission that includes educating its students. Financial metrics fail to measure the full value of projects that achieve non-financial objectives.

Likewise, the standard financial metrics do not represent the true objectives of private-sector organizations. Management scientists and U.S. business leaders are nearly unanimous in the opinion that the fundamental objective of investor-owned organizations is to maximize shareholder value. With this view, the appropriate metrics for evaluating projects in the private sector are those relevant to forecasting impacts on the market value of the business.

Two Views on Organizational Value

There are two competing views on how to measure the value created by an organization, based on the answer to the question, "Value to whom?" The first view is that the purpose of organizations is to produce value for owners (shareholder value). The second view is that the purpose of organizations is to create value not just for owners, but for other stakeholders as well (stakeholder value).

Shareholder Value

As indicated above, most management scientists and U.S. business leaders argue that the fundamental objective of investor-owned organizations is to maximize value for the owners of the business. In the case of a publicly traded company with one type of stock, shareholder value is roughly the number of outstanding shares multiplied by current share price. Dividends, of course, increase the return to shareholders while simultaneously decreasing share price. For both theoretical and practical reasons, increasing market value can be argued to be the ultimate goal of businesses.

One argument for shareholder value is the objectivity of market valuations. Market prices are objective and applied consistently across all publicly held companies and markets. If they were not there would be opportunities for arbitrage; that is, one could buy shares of a company at a lower price in one market and simultaneously sell them at a higher price in another market.

Irrespective of theory, seeking higher stock value is clearly a critical objective for senior management. Compensation for top executives is often tied to stock performance. Poor stock performance is a common reason for a company's board of directors to consider replacing its chief executive. A company whose stock is depressed because it is not effectively managing for shareholder value is a candidate for investor takeover.

It is important to understand that managing for shareholder value and managing for profitable cash flows are two very different things. The figure below compares the market values and risk-adjusted discounted value (NPV) of projected future earnings for four companies in the same industry

Figure 19:   Market value is not the NPV of projected earnings (data for 4 companies in same industry).

As illustrated, a company's market value is usually higher than the NPV of its earnings. In the literature on real options, the additional value is termed "option value." In addition to projected future cash flows, market value depends on how buyers and sellers in the marketplace perceive, among other things, the ability of the business to obtain and respond to future opportunities and avoid future threats.

Also as shown in the figure, it is not uncommon to find two companies in the same industry such that one has a higher NPV of projected returns but the other has a higher total market value. Indeed, some "dot com" companies have substantial market values yet cannot convincingly forecast any profits. Option value does eventually translate into earnings. However, because it is generally difficult or impossible to forecast the exact mechanisms by which this will occur, option value and other indirect sources of value will not be represented in accounting forecasts of cash flows.

The implication of the above is that impacting cash flows is only one of several paths by which projects can influence shareholder value. The alternative paths that may be relevant depend on the nature of the organization's business. For example, a project that generates new knowledge that improves consistency of execution can reduce investment risk. This can have the effect of lowering the cost of capital and increasing the returns on the capital invested. Investors seeking investments likely to increase in value will buy more shares, which will increase shareholder value. As another example, a service company might conduct projects that enable it to meet commitments or promises to customers. This improves customer perceptions and can influence the views of Wall Street analysts, resulting in higher stock prices.

Stakeholder Value

The alternative view, more popular outside the U.S., is that firms should create value for all stakeholders, not just shareholders. Stakeholders are groups of people who have an interest in the business or organization (Figure 20). Typically, important stakeholders are those whose actions can have important impact on the success of the organization. Stakeholder value includes value for employees, suppliers, customers, and the local community.

Figure 20:   Some potentially important stakeholders.

Proponents of more narrow shareholder value argue that organizations pursuing self-interest and economic efficiency do, in fact, serve all stakeholders. However, this argument carries the implicit assumption that the markets within which the firm operates are "perfect," so that stakeholders other than shareholders are unharmed by the firm's focus on benefiting shareholders alone. For example, if employees are laid off, they can immediately get equivalent jobs elsewhere. Similarly, if suppliers or consumers are mistreated, they can take their business to other firms. If there is any impact on stakeholders, the argument goes, it is beneficial. The most successful companies create the products and services that customers want most at the lowest price. Their rising stocks attract more investors, which enables them to satisfy even more customers and hire more employees. The tax system, the argument goes, ensures that successful companies create tax revenues for social ends.

The U.S. economy is well diversified, and many have argued that this makes the "perfect market assumption" a pretty good approximation. However, the recent well-publicized accounting scandals have caused politicians and some U.S. business leaders to question absolute reliance on shareholder value. It may be reasonable, therefore, for even a private sector company to want to explicitly consider impacts on all stakeholders when evaluating projects. Companies in regulated industries are often required to serve the interests of customers, citizens, and other stakeholders. Including stakeholder value further weakens the argument for using financial metrics as the sole basis for project prioritization.

Stakeholder value is almost certainly the more appropriate definition of value for non-profit and public-sector organizations. However, even those companies that care only about shareholders need to consider how their actions affect other stakeholders. If they don't, they may conduct (or more likely, fail to conduct) projects that adversely impact the attitudes of workers, customers, regulators, and local citizens and/or public officials. Careful investors factor considerations such as employee relations, customer satisfaction, regulator attitudes, and community perceptions into their investment decisions. Thus, significant impacts on stakeholders inevitably affect shareholder value.

As is the case with shareholder value, methods are available for quantifying stakeholder value. The appropriate project valuation metrics are those that result from applying these methods. The approach involves constructing a project-selection decision model that evaluates projects and project portfolios based on the goal of maximizing organization value (shareholder or stakeholder value).

Improving Prioritization and Part 2 References

Project-Selection Decision Models