Lee Merkhofer Consulting Priority Systems
Implementing project portfolio management

"The metrics for evaluating projects must support the main goal of PPM; namely, maximizing the value of the project portfolio."

Part 3:  Lack of the Right Metrics


Metrics imply incentives

The metrics that an organization uses to evaluate projects have a big impact not only on the projects that get chosen but also on the projects that get proposed. "Tell me how you will measure me, and I will tell you how I will behave" [1]. Even if the metrics aren't intended to create incentives, managers interpret them as indicating what the organization regards as important.

Establishing and using metrics that accurately reflect the organization's true objectives can have a significant impact on organizational success. Conversely, lacking metrics or using the wrong metrics puts the organization at a serious disadvantage. Lack of the right metrics is the third reason organizations choose the wrong projects.

Defining Project Value

The metrics for evaluating projects must support the main goal of project portfolio management (PPM); namely, maximizing the value of the project portfolio. Thus, metrics are needed for measuring project and portfolio value. How do you measure project value? More fundamentally, how do you define value? These questions are more perplexing than you might at first think.

The Utilitarian View of Value

People have been arguing about the definition of value for centuries. The relevant concept of value for project selection is the "utilitarian concept of value:" This view of value was first articulated in the fourth century BC by Aristotle —the value of something is not an intrinsic property of that thing, but rather is determined by its usefulness to those who want it [2]. An item is perceived useful to the extent that it advances the achievement of one's objectives.

The utilitarian view explains organizational interest in projects. Organizations conduct projects because they believe that the outcomes or consequences of the projects will be useful. The more useful the project consequences, the more valuable the project will be. The project consequences that are desired depend, of course, on the organization's objectives.

A Calculation of Project Value

Aristotle

Suppose, for example, that a company's fundamental objective is to create value for its shareholders (more discussion of shareholder value is provided later). Suppose further that the company is considering a hypothetical project that provides only one consequence: an immediate, one-time cash infusion to the company of one million dollars (to be more precise, suppose that the magnitude of the cash flow from the project is such that, after all tax and accounting considerations are addressed, increases the net worth of the company by $1 million). Under these assumptions, it would seem sensible to conclude that the value of that project is $1 million.

Willingness to Pay

A generalization of the above reasoning is to argue that the value of a project is what the organization would be just willing to pay to obtain the consequences of the project [3]. Stated more precisely, project value might be defined as the maximum amount of money that the organization's most senior executives would be willing to pay for the project's consequences, without having to pay the project's costs and including consideration of risk. (In other words, if project consequences are uncertain, project value could be defined as the maximum amount of money executives would be willing to give up in exchange for owning the uncertainty—the "lottery"—over the project's consequences.) In the portfolio context, the value of a project could be the amount of cash that would make executives indifferent between (a) the portfolio without the project and (b) the portfolio with the project's consequences but with a debt equal to that amount of cash.

Indifference Prices

In the literature on valuation, the above definition of value is termed the buying indifference price (also called the breakeven buying price) [4]. This term applies because at this price the organization would be indifferent between not getting the project consequences and getting the consequences but having to pay the price (the breakeven price at which the company believes it would neither gain nor lose, but simply break even). A close relative is the selling indifference price (or breakeven selling price), which argues that the value of a project is the amount of cash that would make executives indifferent between (a) the portfolio with the project consequences and (b) the portfolio without the project but with that amount of additional cash.

Complexities for Determining Indifference Prices

Though indifference pricing may seem non-controversial, at least in theory, the concept is less straightforward when you think about the impact that paying a buying indifference price or obtaining a selling indifference price has on other decisions. Suppose I ask you to tell me the most that you would be willing to pay to buy the consequences of project A. If you consider (as you should) that buying project A's consequences means that you won't have the resources needed to buy some other project B, the true cost of buying A is not getting the most valuable B that you could have bought. Losing the opportunity to buy project B is the opportunity cost of buying project A [5]. With opportunity cost reasoning, the most an organization should be willing to pay to obtain a project should be less if doing the project means foregoing other, very good investments than when it would mean foregoing only marginal investments.

The situation gets more complex still in the portfolio context. If project buying prices are determined sequentially, the remaining resources available for buying projects declines as projects are added to the portfolio. Furthermore, the value of obtaining project outcomes might change as you commit to doing other projects and, therefore, anticipate experiencing the consequences of these other projects. Thus, a troublesome implication of indifference prices, when opportunity costs are considered, is that a project's indifference price logically depends on the order in which projects are added to the portfolio. If the indifference price for a project is determined based on it being the first project in the portfolio (or if its value is determined in isolation to other projects), logic says we should get a different value than if we assume it is being added to other portfolio projects. Also, project indifference prices ignoring other projects will not sum to the indifference price for the project portfolio (however, it can be shown that if you sum sequential project indifference prices the result will equal the indifference price for the portfolio as a whole) [6].

Determining Indifference Prices "On the Margin"

In practice, in my opinion at least, the above complexities need not be of significant practical concern. Organizations obtain capital from a variety of sources and spend that capital on numerous different investments. Typically, each individual project in a project portfolio consumes a relatively small portion of the organization's total capital. My experience is that executives asked to estimate a project's indifference price give little, if any, thought to the specific opportunities that would need to be foregone if that amount was actually spent. From a practical standpoint, it is impossible to know either the true opportunity cost of paying a buying indifference price or the opportunity value generated from receiving a selling indifference price. Most projects can be evaluated "on the margin," with the assumption that the value the project will provide can be assessed without worrying too much about what the impact will be on other activities.

Make Sure Projects are Properly Defined

Before saying much more about the definition of project value and the metrics for measuring value, it must be acknowledged that projects need to be defined in such a way that they can be valued. If the usefulness of a project depends to a significant degree on the other projects that are in the project portfolio, then it will not be possible to assign a unique value to it. Fortunately, I've found that projects to be prioritized can usually be specified in a way that allows them to be valued.

Redefine Projects to be Independent of One Another Too many projects

The exception to the assumption that projects can be valued individually and on the margin occurs when projects are interdependent, which is typically the case when a collection of related projects are designed to be conducted together. For example, if one project is defined to be the purchase of a desired asset and another project is to use that asset to achieve some desired end, then it would not make sense to try to assess the value of the projects in isolation of one another. In such cases, the obvious solution is to combine the interdependent projects into one larger project such that the larger project is, at least approximately, independent of other projects. My advice to organizations implementing PPM is to establish a policy of defining projects in such a way that each project encompasses all the work needed to achieve the ends that justify the work.

Define Multiple Versions of Large Projects

A potential problem with combining interdependent projects into larger independent projects is that the approach can lead to "all-or-nothing" choices for some important, and large, undertakings. The cost of a large project composed of many interrelated tasks might be so great that it doesn't compete well with other projects based on its ratio of value to cost. To avoid all-or-nothing choices, I often recommend that project proposers be asked to submit one or more lower cost "versions" whenever a large, combination project exceeds a specified dollar amount. Even if the organization lacks a means for quantifying project value, project proponents understand the concept that a reduced scope version of what they would like to accomplish may have a higher value-to-cost ratio. As you would expect, most project proponents are eager to define lower-cost versions of their projects to avoid the risk that none of the elements of what they want to do will be funded. Whenever multiple versions of a project are defined, then the version that leads to the highest total portfolio value is the one that should be recommended for funding. Usually, this will be the highest cost version of the project that makes the budget funding cut off.

This same approach works in the case where the smaller project components of a large project can stand on their own merit, but where there are synergies if they are conducted together. For example, project A and project B might be interdependent, but project A and project B have some merit on their own. Then various collections of the interdependent projects can be considered, for example, project A by itself, project B by itself, and project A and project B together. Similarly, if the value of a project depends to a degree on many other projects that might be funded, its value may be assessed based on some most likely scenario regarding those other projects. If, at the point in time when a go versus no-go decision for the project is needed, the earlier assumption about which other projects will be conducted proves wrong, the project whose value depends on those other projects may be reassessed using a more accurate assumption about the other projects to be included in the project portfolio.

A Working Definition for Project Value

Let's return to the original question of how to define project value. I've found that most people find it intuitive that the value of something is the amount of money viewed equally desirable to that thing. Accordingly, my working definition for project value is:


Project value

=

The amount of money the organization's executives would prefer equally, at present, to the uncertain future consequences of conducting the project

Eq.1


If the equivalent monetary amount changes significantly depending on other choices that the organization makes, then, as stated above, contingent values may be specified that depend on those other choices.

My definition of project value is a variation of the selling indifference price—it is a selling price because it is an amount the organization would need to be compensated to voluntarily give up ownership of the project's consequences. It is an indifference price because it is the price point such that the organization is indifferent between obtaining the price in return for giving up the project consequences.

Projects Determine the Evolution of the Business

OK, assuming that you agree that the worth to the organization of the project consequences is an appropriate definition of project value, how do we determine how much cash an organization would deem of equal value to the anticipated consequences of a project? Before addressing that question we need to consider more carefully why organizations conduct projects.

The business of an organization is always evolving, and the projects that the organization chooses affect that evolution. For example, a new technology might become available that would allow the organization to reduce its costs. If a project is conducted to install the technology, the organization would incur lower operating costs than it would have if it had chosen not to do the project. Achieving a significant step change in operating costs could, obviously, dramatically change the course of the business.

It is also true that the projects that an organization chooses not to conduct affect the evolution of the business. For most organizations, standing still means falling behind. There are many reasons for this, including increasing competition, changing customer preferences, and aging of the organization's assets. Thus, to take one example, if the organization chooses not to do projects that maintain or replace aging assets, the service provided by those assets will decline.

Figure 1 illustrates a useful way to think about project value. At the point in time when an organization is considering a new project, it is really facing a choice between two alternative futures. If the project is conducted, that project will, presumably, transform the business to some more desirable state. If the project is not conducted, some other, presumably less-desirable, state will result.


Projects determine business evolution.

Figure 1:   Project choices determine the future state of the business.



The perspective of Figure 1 provides another basis for computing project value. The value of a project is the difference between the values of two potential future states of the organization:


Project value

 =

  Value with the project  

-

  Value without the project

Eq.2


(Actually, one could argue that Equation 2 gives the net value of the project, since in the without-project case the cost of the project could be spent elsewhere. However, as I argued above, people rarely consider opportunity costs when judging the impacts of projects.)

Equation 2 provides an alternative and often useful way to think about the value of projects, but it is especially helpful for projects that produce changes in the way the organization conducts its business. Take, for example, a proposed project to create an on-site cafeteria for employees. An effective way to identify and estimate of the benefits of such a project is to imagine and contrast futures with and without the cafeteria. Doing so, it might be possible to develop, for example, credible estimates for the time saved by staff who would otherwise travel off site for lunch. Author John Goodpasture provides a similar example involving an HR payroll project [7]. To value the project, two sets of estimates were made corresponding to with-project and without-project scenarios. The estimates included annual overtime costs required to be paid to HR staff, transaction error rates, workforce satisfaction expressed on a 1-to-5 constructed scale, and the absenteeism rate in the HR department.

The next page provides additional insights regarding project value based on my definition.

References

  1. E. Goldratt, The Haystack Syndrome, North River Press, 1991.
  2. G. Barry & J. Lewis(1964). "Aristotle and the Development of Value Theory". Quarterly Journal of Economics 78 (1): 115-128. 1964
  3. H. R. Varian, Microeconomic Analysis, Vol. 3. New York: W.W. Norton 1992.
  4. R. Carmona, Indifference Pricing: Theory and Applications Princeton University Press 2009.
  5. D. R. Henderson, "Opportunity Cost" Concise Encyclopedia of Economics (2nd ed.), Indianapolis: Library of Economics and Liberty 2008.
  6. J. E. Smith & R. F. Nau. "Valuing Risky Projects: Option Pricing Theory and Decision Analysis," Management Science, 41 (5) 795-816, 1995.
  7. J. C. Goodpasture, Maximizing Project Value: A Project Manager's Guide, Management Concepts Press, 2013.