Effective project portfolio management (ppm) requires the ability to identify the best projects to include in the project portfolio. The projects that can be included within the portfolio are limited, of course, by the constraints that exist on the organization's resources. Subject to these constraints, the best project portfolio, as I've argued on the previous two pages, consists of the projects that collectively generate the most value. The organization's senior decision makers decide for whom value should be generated (e.g, for shareholders or stakeholders). Regardless, the value of a project is the worth, as judged by the organization, of the project's consequences. In order to identify the best projects, the organization must be able to measure project value. What, then, are the appropriate metrics for measuring project value? |
|||||||||||||||||||||
Organizations typically have difficulty answering this question. The natural tendency is to measure what's easy to measure, not necessarily what's most important. A bottom-up approach is used. Managers start by defining what appear to be useful metrics, but then can't come up with equations for combining those metrics into a measure of project value. They end up using simplistic aggregation equations, such as indefensible scoring models, or vague and dubious concepts such as "balance" and "strategic alignment." The resulting numbers assigned to projects have little if anything to do with the actual value of those projects. Unless there is a logical way to combine the selected metrics into a measure of project value, those metrics will not be of much help for identifying value-maximizing project portfolios. So, how, then, do you determine the metrics and associated equations for computing the value of projects? The answer is—Reverse the process, use a top-down approach and create a project value model [24]. Value ModelsA value model is a model for measuring the worth, expressed in monetary terms, of an alternative, outcome, or uncertainty. Like other kinds of mathematical models, a value model has inputs and outputs, consists of variables and equations, and can be implemented in software. There are many different types of value models. Net present value (NPV), for example, is a value model that accounts for one type of value, financial value. We need a value model capable of accounting for other types of project value as well. So, how do you construct a model for measuring project value? Decision Making MethodsA promising place to look for advice on how to create a value model is the literature on formal decision-making methods. Because decision making is important and often difficult, especially when there are multiple objectives, researchers in various sub-fields of operations research and management science have proposed numerous methods for identifying optimal choices [25]. The table of acronyms below identifies some of these decision making methods. The entries are examples, with a little research you'll find many more.
Each of the above methods has its own characteristics, but nearly all involve 4 major steps:
The equations by which the various methods combine weights and performance estimates to provide a ranking metric differ. For example, WSM, which stands for the weighted sum model, multiplies the estimates of performance against each criterion by a criterion weight and adds the results. WPM stands for the weighted product model, and as the name suggests it uses an aggregation equation that involves multiplication rather than summation. With most of the others, including the analytic hierarchy process (AHP), TOPSIS, and ELECTRE, the mathematics for obtaining the ranking metrics are more complicated. Computer programs are available for applying all of these methods. Although few of the decision making methods claim that their computed ranking metric measures the value of alternatives, the methods are of interest for our purpose because their developers have provided step-by-step instructions for their application, including how to generate the algorithm for computing the method's ranking metric. Thus, the so-called decision making methods potentially provide instructions for creating value models. Multi Objective Decision AnalysisFrom available decision-making methods, the one that, in my opinion, provides the best basis for creating a project value model is multi-objective decision analysis (MODA), also known as multi-attribute utility analysis (MUA) [26, 27]. The reasons that I believe MODA to be the best choice for project prioritization are:
The Theory of Rational Decision MakingA theory of rational decision making was first outlined more than a half century ago by Oscar Morgenstern and John von Neumann in their book, "Theory of Games and Economic Behavior" [29]. The mathematicians were interested in how one should choose among gambles, referred to as "lotteries." They began by specifying a set of axioms (assumptions) intended to define what it means to choose rationally. Although this initial work dealt with choosing among lotteries with well-defined probabilities, Leonard Savage and others subsequently expanded the theory so as to apply to any uncertainties a decision maker might face, including those for which lack of data means probabilities must be assigned subjectively [30]. Axioms of RationalityThe axioms that have been proposed for defining rational choice can be expressed in several different ways [31]. For example, one axiom, called transitivity, states: If you prefer item A to item B, and you prefer item B to item C, then you must prefer item A to item C. An axiom called monotonicity states: If you are offered two gambles involving prizes A and B and you prefer prize A to prize B, then you must prefer the gamble where the probability of winning A is higher. An axiom called continuity states: If you prefer item A to item B and item B to item C, then there must be some probability p that would make you indifferent between item B and a gamble that provides item A with probability p and item C with probability 1-p. A full list of axioms for rational choice is provided in my glossary. Most people find the axioms for rational decision making reasonable and easy to accept [32]. Even so, the axioms underlying decision theory aren't meant to describe how people actually make decisions, they describe a proposed ideal for how people ought to make decisions. In practice, and for complex decisions especially, studies show people routinely violate the axioms, in part due to the common biases and errors described in Part 1 of this paper. Expected Utility TheoremRemarkably, Morgenstern and von Neumann, and then Savage, were able to prove that, if a decision maker accepts the axioms of rationality, a function called a utility function exists that assigns numbers (utilities expressed in units called "utiles" or "utils") to outcomes such that the outcomes with higher utility numbers (more utiles) are always more preferred. Furthermore, if outcomes to a choice are uncertain, and utilities and probabilities are assigned to each of the possibilities, then the most preferred choice is always the one with the highest expected value of utilities (possible numbers of utiles multiplied by their probabilities and then summed) [33]. If the outcomes are undesired (rather than desired) the computed utility numbers may be negative, in which case, the highest utility will be the one with the least negative number. The utility function concept would not be of much use to us if it were not for the fact that decision analysts have devised methods for deriving utility functions, including using an interviewing process whereby an individual's personal utility function is assessed by asking a series of questions [34,35]. (The next page provides a description of the methods for assessing utility functions.) Methods have also been developed for probability encoding; that is, eliciting a subject's subjective probability distribution for an uncertain quantity [36]. Following Savage, the probabilities that are assigned to an uncertain outcome must reflect the decision maker's beliefs about how likely the possible outcomes are. However, if objective data are available for deriving probabilities, those probabilities may be used, provided, of course, that computed results reflect the decision maker's true beliefs. Thus, the utility function together with available assessment techniques provide the foundation for mathematically identifying optimal, value-maximizing choices. Top-Down Model ConstructionIn contrast to the bottom up approach often used to build other types of models, MODA models are created from the top down, starting with specifying and structuring the decision maker's objectives. A performance measure is then defined for each objective. For example, product price in dollars might be chosen as the performance measure for cost, and the number of years the product will last might be the performance measure for quality. The performance measures are the attributes for the multi-attribute utility function. Deterministic vs. Probabilistic Decision ModelsWhen building a model, the analyst can decide whether to ignore uncertainties and make the model deterministic, or, alternatively, to make the model probabilistic by assigning probabilities to one or more uncertain variables. In either case, a decision maker's preferences can be modeled with MODA. However, by convention, if the model is deterministic, the utility function is termed a value function [37,38]. A value function is simply a special case of the utility function applicable when it is assumed that there is no uncertainty over decision outcomes. The value function measures preference under conditions of certainty. The utility function measures preference under conditions of uncertainty. Metrics for Measuring ValueNow that you've been introduced to the mathematics of value models, the appropriate metrics for measuring project value should be obvious. The appropriate metrics are the inputs to a multi-objective value model, which will be either a utility function or a value function specified to capture the values of your organization. The model will assign numbers (utiles) to projects such that the higher the number is, the more desirable the project is. As shown in the above example, provided that the value or utility function is additive and linear in money, the preference measure can be expressed in equivalent monetary units, the most convenient unit for project prioritization. One way to obtain a value/utility function for your MODA value model is to elicit it directly from a decision maker. There are other ways as well, but there are times when eliciting the function from the decision maker is most convenient. The next page of this paper describes the interviewing process for eliciting a value or utility function from a decision maker. |