Lee Merkhofer Consulting Priority Systems
Implementing project portfolio management

Part 3:  Lack of the Right Metrics


Metrics imply incentives

Metrics = incentives

The metrics that an organization uses for evaluating projects have a big impact not only on the projects that get chosen but also on the projects that get proposed. "Tell me how you will measure me, and I will tell you how I will behave" [1]. Even if the metrics aren't used to create incentives, managers interpret them as indicating what the organization regards as important. Lack of the right metrics is the third reason organizations choose the wrong projects.

Defining Project Value

The goal when choosing projects should be to maximize the total value to be derived from those projects. But, how do you measure project value? More fundamentally, how do you define value?

A Definition of Value

People have been arguing about the definition of value for centuries. The relevant concept of value for our purposes is termed the "utilitarian concept of value:" The value of a project is a measure of degree to which that project enables the organization to achieve its objectives. This view of value was first articulated in the fourth century BC by Aristotle —the value of something is not an intrinsic property of that thing, but rather is determined by its usefulness to those that want it.

Aristotle

Organizations conduct projects because they believe that the consequences or outcomes of the projects will be useful. Thus, we define the value of a project to be the worth, to the organization, of obtaining the consequences of the project. The project consequences that are desired depend, of course, on the organization's fundamental objectives. Suppose, for example, that an organization's fundamental objective is to create value for its shareholders (more discussion of shareholder value is provided below). Suppose further that the organization is considering a hypothetical project that provides only one consequence: an immediate, one-time cash flow to the company of one million dollars (to be more precise, suppose that the magnitude of the cash flow from the project is such that, after all tax and accounting considerations are addressed, increases the net worth of the company by $1 million). Under the specified assumptions, the value of that project would logically be $1 million.

Note that the value of a project does not depend on its cost. (The decision of whether to conduct a project, does, of course, depend on cost, where a measure such as net value—value minus cost—would be relevant.) An exception to the rule that project value does not depend on project cost would be a case where paying more for a project impacts the organization's ability to benefit from the project. An extreme example would be a project anticipated to produce great project outcomes, but whose cost would bankrupt the company. In such cases, the value of the project would logically be the value of the project consequences taking into account any effects of having to pay for the project. Since we are concerned with organizations that conduct numerous projects, each of which consumes only a portion of the budget, we can typically ignore such effects.

To be precise, I define the value of a project to be the maximum amount the organization's leaders would rationally be willing to spend in order to obtain the consequences of doing the project, including consideration of risks. (In other words, if project consequences are uncertain, project value is the maximum amount leaders would pay for the uncertain project consequences.) In the literature on valuation, this value is termed a "buying indifference price"—it is a buying price because it is an amount the organization would pay to buy the project consequences, and it is an indifference price because it is the price point such the organization is indifferent between declining the project versus paying the price and obtaining the project consequences.

Project value

Project value is the maximum price the organization is willing to pay to obtain the project consequences

Our definition of value has a critically important property—namely, it exactly maps to organizational project preferences. Faced with two projects requiring the same resources, an organization will prefer Project A to Project B if and only if our measure of project value is higher for Project A than it is for Project B. Many common approaches to project prioritization don't use a project evaluation measure with this essential property (see, for example, strategic alignment). Unless project value is defined in a way that maps to the true preferences of the organization, that measure cannot correctly prioritize projects. Our definition of project value is a true measure of the relative attractiveness to the organization of its project alternatives, and that measure can be compared with project costs to determine the projects that are worth doing and their priorities.

How do we determine how much an organization should be willing to pay to obtain the anticipated consequences of conducting a project? Before addressing that question we need to consider more carefully why organizations conduct projects.

Projects Determine the Evolution of the Business

The business of an organization is always evolving, and the projects that the organization chooses affect that evolution. For example, a new technology might become available that would allow the organization to reduce its costs. If a project is conducted to install the technology, the organization would incur lower operating costs than it would have had if it chosen not to do the project.

It is also true that the projects that an organization chooses not to do affect the evolution of the business. For most organizations, standing still means falling behind. There are many reasons for this, including increasing competition, changing customer preferences, and the aging of organizational assets. Thus, to take one example, if the organization chooses not to do projects that maintain or replace aging assets, the service provided by those assets will decline.

Figure 18 illustrates a useful way to think about project value. At the point in time when an organization is considering a new project, it is really facing a choice between two alternative future states. If the project is conducted, that project will, presumably, transform the business to some more desirable state. If the project is not conducted, some other, presumably less-desirable, state will result.


Projects determine business evolution.

Figure 18:   Project choices determine the future state of the business.



The perspective of Figure 18 provides a basis for computing project value. The value of a project is the difference between the values of the two potential future states of the business:

Project value  =  Value with the project  -  Value without the project

(Equation 1)

Some Observations Regarding Project Value

Based on Equation 1, we can conclude some important things about project value.

  • To estimate project value, it is necessary to consider what would happen if the project is not conducted as well as what would happen if the project is conducted. Many project scoring systems ignore the implications of not doing projects, which causes some types of projects to be grossly undervalued.
  • The same project can have different values to different organizations. For the purposes of estimating project value, therefore, it is not sufficient to consider just the characteristics of the project itself. It is also necessary to consider characteristics of the business that determine how useful that project will be to that business.
  • The value of a project can change depending on what other projects are conducted. Because of synergies and economies of scale, for example, the costs and benefits of a project can change based on the other projects that are also conducted. If such dependencies exist, optimizing the project portfolio requires estimating the value of conducting different groupings of the interdependent projects.

For the purpose of identifying metrics for measuring project value, the most important implication of Equation 1 is this: Project value is the contribution that the project makes to the total value achieved by the organization. Because project value is the difference between the value that the organization attains in two alternative states, the methods for estimating project value are essentially the same as the methods for estimating organizational value. This is good news, since management scientists have devoted considerable effort to determining how best to measure the value created by organizations. We can use the concepts and methods that they have devised to enable us to quantify project value.

Inadequacy of Financial Metrics

So, how can you measure the value created by an organization? Interestingly, most businesses get it wrong.

Most businesses use financial metrics computed from cash flows to measure value, for example, return on investment (ROI), internal rate of return (IRR), net present value (NPV), payback period. Using these metrics to evaluate candidate projects requires forecasting the cash flows that would be produced by the project. Some impacts on cash flow may be relatively easy to forecast, in particular, the costs to conduct the project and any direct impacts the project will have on the firm's future costs and revenues. However, it is difficult to translate many project consequences into impacts on cash flow. For example, how would a project designed to collect better information about customer preferences impact future cash flows? From a practical standpoint, cash flow analysis will not capture many project impacts.

Thus, at best, financial metrics provide only a partial representation of what is important to a business. According to a study by Research Technology Management, companies that rely mostly on financial metrics obtain "unbalanced portfolios" that are not well matched to the strategy of the firm [2].

The limitations of financial metrics are even more obvious when it comes to evaluating projects in the public sector. Public-sector organizations have social missions and may not even sell goods and services that generate cash flows. Even it they do, earning a financial return may not be a primary objective. For example, a water utility has a mission that includes serving community water needs. A public school has a mission that includes educating students. Cash flow analysis will not do a good job of indicating how well these organizations are accomplishing their missions. Financial metrics fail to measure the value of projects intended to achieve non-financial objectives.


footer
Lee Merkhofer Consulting. All rights reserved © 2002-2012.