"The metrics for measuring value are the inputs to a project value model." |
The primary function of project portfolio management (ppm) is to manage the project portfolio so as to generate maximum value [27, 28]. The organization's senior executives determine for whom value should be created. Consistent with the organization's mission, some organizations focus on generating value for shareholders; others seek value for customers, citizens, employees and/or other organizational stakeholders. Regardless, in order to maximize the value generated by the project portfolio, organizations need the ability to measure the value of projects. Project ValueThe value of a project is not an intrinsic property of the project. Rather, project value is a reflection of the organization that is considering doing it. In addition to the specifications of what the project will do, project value depends on the organization's strategy, the needs and opportunities that it is facing, the resources that it has available to it, and the preferences of its senior decision makers, including their willingness to accept risks. Value Means WorthDespite the fact that so many organization-specific factors determine project value, a single, dollar value can be ascribed to every project at every organization. Project value, as I've defined it, is what obtaining the project consequences is worth to the organization—it is, the amount of money, at present, that is perceived by the organization's senior decision makers as being of equal worth to the prospect of obtaining the project's, possibly uncertain, future consequences. |
|||||||||||
Organizations Seek to Maximize Project ValueAlthough every organization understands the concept of project value, most lack the ability to assign dollar values to projects. However, even without a formal model for quantifying project value, organizational decision makers, those individuals who make the final choices about which projects to conduct, often do so based on an intuitive feeling that some projects are worth more than others [29]. If the value estimated to be contributed by a candidate project is perceived to be sufficiently large, the project is selected. If not, it is rejected. Unfortunately, due to the complexity of projects, the difficulty of forecasting project consequences, and the limits of human intuitive reasoning, it is easy for good projects to be overlooked and poor projects to be chosen. Clearly, it would be useful to organizations to have the ability to quantify the value of projects. Many organizations are, in fact, experimenting with formal methods for evaluating and prioritizing projects. So, how can an organization measure the value of its projects? Or, to state the question in the way it is more typically posed: "What are the metrics for measuring project value?" Bottom Up vs. Top Down ModelingMost organizations have difficulty answering this question. The natural tendency is to measure what's easy to measure, not necessarily what's most useful or important. Organizations typically follow a bottom-up approach. Managers start by defining what appear to be useful metrics, but then can't come up with equations for combining those metrics into a proper measure of project value. They end up using scoring models ungrounded in any theory of value and/or vague and dubious concepts such as "balance" and "strategic alignment." The resulting numbers assigned to projects have little if anything to do with the actual value of those projects. Unless there is a logical way to combine selected metrics into a measure of project value, those metrics will not be of much help for identifying value-maximizing project portfolios. So, how, then, do you determine the metrics and associated equations for computing the value of projects? Value ModelsThe answer is: "Reverse the process, use a top-down approach and create a project value model" [30]. Consistent with my definition of project value, I define a project value model to be a model for computing the value of a project. A value model embodies the values and preferences that the organization wishes to apply for the purpose of comparing, prioritizing and selecting projects, data that is appropriately provided by the organization's most senior executives. Like other kinds of mathematical models, a value model has inputs and outputs, consists of variables and equations, and can be implemented in software. The inputs to a project value model describe the characteristics and estimated impacts of candidate projects. The output of the model is an estimate of what doing the project is worth to the organization. An Example Value ModelThere are many different types of value models. Net present value (NPV), for example, is a value model that accounts for one type of value, financial value. The inputs to NPV are the cash flows resulting from an investment, incremental cash that would be received and/or paid out over time as consequences of a decision to conduct the project. The preference judgment needed for the model is the discount rate. The output of the NPV value model is the present value of those cash flows, an amount estimated to be of equivalent worth to the project. NPV is a start, but we need value models capable of accounting for other types of project value in addition to financial value. That's because financial return isn't the only objective relevant to project decisions. Multiple objectives complicate decision making because the objectives often conflict with one another, requiring the decision maker to accept tradeoffs. For example, when choosing to purchase a product, you might care about cost and quality. Higher quality often means a higher price. So, how do you create a multi-criteria model for measuring project value? Theory for Measuring ValueThere is, in fact, a theory for measuring value. The theory is called utility theory, but it also goes by the more presumptuous title, The Theory of Rational Decision Making. The concepts were first outlined more than a half century ago by mathematicians Oscar Morgenstern and John von Neumann in their book, "Theory of Games and Economic Behavior" [31]. Morgenstern and von Neumann were interested in how one ought to choose among gambles, referred to as lotteries. They began by specifying a set of axioms (assumptions) intended to define what it means to choose rationally. Although this initial work dealt with choosing among lotteries with well-defined probabilities, Leonard Savage and others subsequently expanded the theory so as to apply to any uncertainties a decision maker might face, including those for which lack of data means probabilities must be assigned subjectively [32]. Axioms of RationalityThe axioms proposed for defining rational choice can be expressed in several different ways [33]. For example, one axiom, called transitivity, states: If you prefer item A to item B, and you prefer item B to item C, then you must prefer item A to item C. An axiom called monotonicity states: If you are offered two gambles involving prizes A and B and you prefer prize A to prize B, then you must prefer the gamble where the probability of winning A is higher. An axiom called continuity states: If you prefer item A to item B and item B to item C, then there must be some probability p that would make you indifferent between item B and a gamble that provides item A with probability p and item C with probability 1-p. A full list of axioms for rational choice is provided in my glossary. Most people find the axioms for rational decision making reasonable and easy to accept [34]. Even so, the axioms underlying decision theory aren't meant to describe how people actually make decisions, they describe a proposed ideal for how people ought to make decisions. In practice, and for complex decisions especially, studies show people routinely violate the axioms, in part due to the common biases and errors described in Part 1 of this paper. Expected Utility TheoremRemarkably, Morgenstern and von Neumann, and then Savage, were able to prove that, if a decision maker accepts the axioms of rationality, a function called a utility function exists that assigns numbers (utilities expressed in units called "utiles" or "utils") to outcomes such that the outcomes with higher utility numbers (more utiles) are always more preferred. Furthermore, if outcomes to a choice are uncertain, and the decision maker's utilities and probabilities are assigned to each of the possibilities, then the most preferred choice is always the one with the highest expected value of utilities (possible numbers of utiles multiplied by their probabilities and then summed) [35]. If the outcomes are undesired (rather than desired) the computed utility numbers may be negative, in which case, the highest utility will be the one with the least negative number. The utility function concept would not be of much use if it were not for the fact that decision analysts have devised methods for deriving utility functions, including using an interviewing process whereby an individual's personal utility function is assessed by asking a series of questions [36,37]. (The next page of this paper describes some of the methods for assessing utility functions.) Methods have also been developed for probability encoding; that is, eliciting an individual's subjective probability distribution for an uncertain quantity [38]. Following Savage, the probabilities that are assigned to an uncertain outcome must reflect the decision maker's beliefs about how likely the possible outcomes are. However, if objective data are available for deriving probabilities, those probabilities may be used, provided, of course, that computed results reflect the decision maker's beliefs. Building Value Models Based on Utility FunctionsUtility functions and the methods for assessing them provide the means for creating value models. However, in contrast to the bottom up approach often used to build other types of models, value models based on utility functions are created from the top down, starting with specifying and structuring the decision maker's objectives. A performance measure is then defined for each objective. For example, as noted above, cost and quality might be objectives for purchase decisions. Product price in dollars might be chosen as the performance measure for cost, and the number of years the product will last might be the performance measure for quality. If the utility function is contructed with performance measures for more than one objective, it is referred to as a multi-attribute utility function. The performance measures are the attributes for a multi-attribute function. Deterministic vs. Probabilistic Decision ModelsWhen building a value model, the analyst can decide whether to ignore uncertainties and make the model deterministic, or, alternatively, to make the model probabilistic by assigning probabilities to one or more uncertain variables. In either case, a decision maker's preferences can be modeled with utility functions. However, by convention, if the model is deterministic, the utility function is termed a value function [39,40]. A value function is simply a special case of the utility function applicable where it is assumed that there is no uncertainty over decision outcomes. The value function measures preference under conditions of certainty. The utility function measures preference under conditions of uncertainty. The general mathematical forms for utility functions, plus two of the most useful, special forms are presented below. Utility Function MathematicsMath wise, a utility function is usually denoted U(x), where x is the outcome to be evaluated by the decision maker. So, according to the theory, if the decision maker is choosing among alternatives that lead to different x, then the best (most preferred) choice will be the one producing the outcome x that maximizes U(x). If x is uncertain and the decision maker assigns a probability distribution, denoted p(x), describing his or her beliefs about how likely the possible values of x are, the best alternative will be the one that maximizes the expected value of U, denoted E[U(x)]. If the utility function is multi-attribute and based on N objectives, designated O_{i}, for i = 1 to N, there will be N corresponding attributes denoted X_{i}, for i = 1 to N, where each attribute X_{i} measure the performance of the project with respect to its corresponding objective O_{i}. If, for example, the decision maker is a manager at a construction company, worker safety might be one objective. The number of injuries occurring annually might be chosen as an attribute for measuring the level of safety achieved. As a result of the alternatives that the decision maker selects, each X_{i} attribute will take on some particular level. For example, the number of injuries that occur to workers might be "3." The outcomes for each attribute X_{i} are denoted x_{i} (in other words, upper case is used to represent the variable and lower case is used to represent a particular outcome for that variable). Figure 4: Building a value model from the top down. A collection of outcomes for the attributes, denoted (x_{1},x_{2}...x_{N}) is sometimes referred to as an outcome "bundle." The utility achieved given a specific outcome bundle is denoted U(x_{1},x_{2}...x_{N}). A utility function is a mapping of a multi-dimensional attribute space into a single dimensioned preference space. The standard notation for expressing value functions is analogous to that used for the more general utility functions. If there are N objectives designated O_{i}, for i = 1 to N, the corresponding attributes for measuring performance against those objectives are designated X_{i}, for i = 1 to N. A possible outcome bundle for the attributes is denoted (x_{1},x_{2}...x_{N}) where each x_{i} is a specified level of the i'th attribute, X_{i}. Once a set of fundamental objectives and associated attributes are defined, a value model may be constructed by associating with each outcome bundle (x_{1},x_{2}...x_{N}) a number indicating the decision maker's preference for that outcome bundle. Perhaps the most important practical result in the theory of value functions is this: If the decision-maker's preferences for the x_{i} satisfy a set of conditions known as additive independence, an additive value function may be constructed [41]:
In this equation, V is overall value, the V_{i} are single attribute value functions that express preferences for consequences that differ in terms of only a single attribute, and the w_{i} are weights. The single attribute value functions are often termed scaling functions. The notation and simplifications useful for utility functions are analogous to those for value functions. As in the case with value functions, there is a similar additive form for utility functions. An additive utility function computes total utility U by weighting and summing single-attribute utility functions U_{i}(x_{i}) for each of the individual attributes x_{i}.
Another special form for the utility function is an exponential function. As we've seen, utility functions and value functions are such that higher is better. Any constantly increasing scaling applied to the functions (i.e., any monotonic transformation) will preserve the order of the utility numbers. Thus, a possible mathematical form for the utility function is an exponential function wherein an additive value function appears in the exponent of the function. More specifically, a possible (in fact, fairly common) mathematical form that may be selected for a utility function is:
Here U is the overall utility function quantifying preferences, V(x_{1},x_{2}...x_{N}) is an additive value function, e denotes the Euler's number (~2.78128), and R is a parameter called risk tolerance that can be set to reflect different decision-making attitudes toward risk (see Part 4 of this paper for more discussion of risk tolerance). The exponential form for the utility function is attractive because it allows an additive value function to be converted to a utility function applicable to a probabilistic decision model. In other words, if the condition of additive independence is satisfied for the value function, the specification of the above form for the utility function requires simply determining the V_{i}, w_{i} and the risk tolerance R. The utility function in this case is sometimes called the relative risk aversion function, since it is the risk aversion expressed relative to the value function [42]. Additive Value Function ExampleThe following example shows how the assumption of an additive form simplifies the construction of a value model. Suppose, a young woman is choosing an apartment. For simplicity, assume there are just two objectives, (1) minimize the commute distance from her apartment to her job and (2) minimize monthly rent. Further, assume there are just three alternatives: (1) an apartment in the building where she works, (2) a nearby, downtown apartment, and (3) an apartment in the country. The objectives are to be "minimized" because commuting and paying rent are both undesirable activities. The example reflects this by using negative utilities. Natural scales are selected for the two attributes: commute distance in miles and rent in dollars. Assuming the additive value function of the form shown in Equation 1 applies, assume, for convenience, that the single attribute value functions go from zero (best level) to minus one (worst level). For the rent scale, we assume that the woman feels the "undesirability" of rent in proportion to the dollars to be paid. A value of zero is assigned if the rent is zero (like living in her parent's house) and a value of minus one is assigned for a rent of $2000 per month (the highest rent that she can afford). (If, as in this example, a scaling function is linear in the attribute, it can be removed from the equation since the weight may be used to convert the attribute level to the value of that attribute level.) With regard to commute distance, as shown in the figure below, we suppose the woman assigns a value of zero to any distance between zero and one mile (a distance such that she could walk to work). Above one mile she feels the level of undesirability is linear in miles up to 60 miles (picked as the limit because it is the longest distance she is willing to commute). The maximum commute distance of 60 miles is assigned a value of minus one. Figure 5: Assumed scaling function for commute distance. Weights of 0.4 and 0.6 are selected to represent the relative value the women would experience (how much she would enjoy) moving from minus one to zero on each scale (these are assumed to be assessed using the swing weight method). As a result of these assumptions the utilities of possible alternatives must lie between zero and minus one utile. Since one of the objectives is measured in dollars, and utility is assumed to be linear in dollars, the ratio of dollars to utiles for rent can be computed and used to express the value of commute distances in dollar equivalents. Thus, we compute the total (negative) value of each alternative as shown below. The alternative with the least negative equivalent cost is, assuming the assumptions are correct, the most desirable choice. Figure 6: Example additive value function analysis. Utility Theory Provides the Mathematical Foundation for Creating Value ModelsCompare the utility function concept to my definition of project value as summarized above and described more fully on the first page of this Part 3 of my paper. Since value maps to preferences exactly and utility maps to preferences exactly, utility and value must rank projects in exactly the same order. This means that the equation for the utility function and the equation for computing project value must be the same, save only for a possible monotonic transformation. If we have a way to obtain a utility function for projects from an organization's senior decision maker, then we will have a way to rank projects for that organization [43]. Furthermore, if the utility function has the additive form and money is one of the criteria (as it almost always is) then, as illustrated by the example above, we can reverse the relationship between money and utility and express project value in monetary units. In summary, utility theory provides a solid basis for constructing value models which, because of the expected utility theorem, will provide an equivalent monetary value for projects, even when the consequences of the projects are uncertain. Metrics for Measuring ValueNow that you've been introduced to value models, the appropriate metrics for measuring project value should be obvious. The desired metrics are the inputs to a multi-criteria value model; namely, the measures used by the model to express performance with respect to each of the model's objectives. The performance measures are estimates of the degree to which a selected project alternative impacts (positively or negatively) each of the organizations objectives. Thus, based on a top-down modeling approach, the appropriate metrics for measuring project value are measures for estimating the degree to which the organization is achieving its objectives. |