Lee Merkhofer Consulting Priority Systems
Implementing project portfolio management

Project Value Models

Bottom up or top down

Effective project portfolio management (ppm) requires the ability to identify the best projects to include in the project portfolio. The projects that can be included within the portfolio are limited, of course, by the constraints that exist on the organization's resources. Subject to these constraints, the best project portfolio, as I've argued on the previous two pages, consists of the projects that collectively generate the most value.

The organization's senior decision makers decide for whom value should be generated (e.g, for shareholders or stakeholders). Regardless, the value of a project is the worth, as judged by the organization, of the project's consequences. In order to identify the best projects, the organization must be able to measure project value.

What, then, are the appropriate metrics for measuring project value?

Organizations typically have difficulty answering this question. The natural tendency is to measure what's easy to measure, not necessarily what's most important. A bottom-up approach is used. Managers start by defining what appear to be useful metrics, but then can't come up with equations for combining those metrics into a measure of project value. They end up using simplistic aggregation equations, such as indefensible scoring models, or vague and dubious concepts such as "balance" and "strategic alignment." The resulting numbers assigned to projects have little if anything to do with the actual value of those projects. Unless there is a logical way to combine the selected metrics into a measure of project value, those metrics will not be of much help for identifying value-maximizing project portfolios.

Choose one..

So, how, then, do you determine the metrics and associated equations for computing the value of projects?

The answer is—Reverse the process, use a top-down approach and create a project value model [24].

Value Models

A value model is a model for measuring the worth, expressed in monetary terms, of an alternative, outcome, or uncertainty. Like other kinds of mathematical models, a value model has inputs and outputs, consists of variables and equations, and can be implemented in software.

There are many different types of value models. Net present value (NPV), for example, is a value model that accounts for one type of value, financial value. We need a value model capable of accounting for other types of project value as well.

So, how do you construct a model for measuring project value?

Decision Making Methods

A promising place to look for advice on how to create a value model is the literature on formal decision-making methods. Because decision making is important and often difficult, especially when there are multiple objectives, researchers in various sub-fields of operations research and management science have proposed numerous methods for identifying optimal choices [25]. The table of acronyms below identifies some of these decision making methods. The entries are examples, with a little research you'll find many more.


Acronyms for Some Formal Decision-Making Methods
AIRM ELECTRE SMAA
AHP ER SMART
ANP GP TOPSIS
COMET MACBETH WPM
DEX PROMETHEE WSM

Each of the above methods has its own characteristics, but nearly all involve 4 major steps:

  1. Specifying criteria for evaluating alternatives
  2. Assigning weights to indicate the importance of the criteria
  3. Estimating the performance of each alternative with respect to each criterion
  4. Applying a mathematical algorithm for combining the weights and performance judgments to obtain a metric for ranking the alternatives

The equations by which the various methods combine weights and performance estimates to provide a ranking metric differ. For example, WSM, which stands for the weighted sum model, multiplies the estimates of performance against each criterion by a criterion weight and adds the results. WPM stands for the weighted product model, and as the name suggests it uses an aggregation equation that involves multiplication rather than summation. With most of the others, including the analytic hierarchy process (AHP), TOPSIS, and ELECTRE, the mathematics for obtaining the ranking metrics are more complicated. Computer programs are available for applying all of these methods.

Although few of the decision making methods claim that their computed ranking metric measures the value of alternatives, the methods are of interest for our purpose because their developers have provided step-by-step instructions for their application, including how to generate the algorithm for computing the method's ranking metric. Thus, the so-called decision making methods potentially provide instructions for creating value models.

Multi Objective Decision Analysis

From available decision-making methods, the one that, in my opinion, provides the best basis for creating a project value model is multi-objective decision analysis (MODA), also known as multi-attribute utility analysis (MUA) [26, 27]. The reasons that I believe MODA to be the best choice for project prioritization are:

  1. MODA is able to measure the value of projects in monetary units (though, it doesn't require this).
  2. MODA can be applied to all types of project decisions, including dynamic project choices where subsequent project decisions depend on the outcomes to initial project choices.
  3. MODA provides a formal logic for specifying the equation for combining metrics and weights. It shows that the correct form of the aggregation equation (e.g., additive, multiplicative, etc.) depends on the degree to which objectives and their performance measures satisfy specified independence conditions.
  4. MODA applies to organizations that have hierarchies of objectives and sub objectives. It also works if the organization subscribes to only a single objective. In fact, if the organization's only objective is profit (with sub objectives of maximizing income in each year), it can be shown to produce NPV as the proper ranking metric [28].
  5. MODA accounts for uncertainty and risk. MODA is also able to account for organizational risk tolerance, the degree to which the organization is averse to taking risks.
  6. A MODA value model is constructed using a top-down approach starting with identifying and structuring the organization's objectives. The method includes tools and techniques for facilitating the model-development process, including methods for obtaining weights, quantifying judgmental uncertainties, designing scales for obtaining numerical and categorical inputs for the model, and incorporating judgments from subject matter experts.
  7. Components of value
  8. A MODA value model identifies the different types of project value and shows the contribution of each to the combined project value. The models are well-suited for sensitivity analysis, including showing the impact of alternative tradeoffs among objectives on the prioritization.
  9. MODA is regarded as among the most defensible of decision making approaches, and has been identified as best practice by numerous experts and has been labeled so by panels from government agencies in the U.S., Canada, and the United Kingdom.
  10. Perhaps most significantly, MODA derives, literally, from a theory of how to make rational decisions

The Theory of Rational Decision Making

Morgenstern and von Neumann

A theory of rational decision making was first outlined more than a half century ago by Oscar Morgenstern and John von Neumann in their book, "Theory of Games and Economic Behavior" [29]. The mathematicians were interested in how one should choose among gambles, referred to as "lotteries." They began by specifying a set of axioms (assumptions) intended to define what it means to choose rationally. Although this initial work dealt with choosing among lotteries with well-defined probabilities, Leonard Savage and others subsequently expanded the theory so as to apply to any uncertainties a decision maker might face, including those for which lack of data means probabilities must be assigned subjectively [30].

Axioms of Rationality

The axioms that have been proposed for defining rational choice can be expressed in several different ways [31]. For example, one axiom, called transitivity, states: If you prefer item A to item B, and you prefer item B to item C, then you must prefer item A to item C. An axiom called monotonicity states: If you are offered two gambles involving prizes A and B and you prefer prize A to prize B, then you must prefer the gamble where the probability of winning A is higher. An axiom called continuity states: If you prefer item A to item B and item B to item C, then there must be some probability p that would make you indifferent between item B and a gamble that provides item A with probability p and item C with probability 1-p. A full list of axioms for rational choice is provided in my glossary.

Most people find the axioms for rational decision making reasonable and easy to accept [32]. Even so, the axioms underlying decision theory aren't meant to describe how people actually make decisions, they describe a proposed ideal for how people ought to make decisions. In practice, and for complex decisions especially, studies show people routinely violate the axioms, in part due to the common biases and errors described in Part 1 of this paper.

Expected Utility Theorem

Remarkably, Morgenstern and von Neumann, and then Savage, were able to prove that, if a decision maker accepts the axioms of rationality, a function called a utility function exists that assigns numbers (utilities expressed in units called "utiles" or "utils") to outcomes such that the outcomes with higher utility numbers (more utiles) are always more preferred. Furthermore, if outcomes to a choice are uncertain, and utilities and probabilities are assigned to each of the possibilities, then the most preferred choice is always the one with the highest expected value of utilities (possible numbers of utiles multiplied by their probabilities and then summed) [33]. If the outcomes are undesired (rather than desired) the computed utility numbers may be negative, in which case, the highest utility will be the one with the least negative number.

The utility function concept would not be of much use to us if it were not for the fact that decision analysts have devised methods for deriving utility functions, including using an interviewing process whereby an individual's personal utility function is assessed by asking a series of questions [34,35]. (The next page provides a description of the methods for assessing utility functions.) Methods have also been developed for probability encoding; that is, eliciting a subject's subjective probability distribution for an uncertain quantity [36]. Following Savage, the probabilities that are assigned to an uncertain outcome must reflect the decision maker's beliefs about how likely the possible outcomes are. However, if objective data are available for deriving probabilities, those probabilities may be used, provided, of course, that computed results reflect the decision maker's true beliefs. Thus, the utility function together with available assessment techniques provide the foundation for mathematically identifying optimal, value-maximizing choices.

Top-Down Model Construction

In contrast to the bottom up approach often used to build other types of models, MODA models are created from the top down, starting with specifying and structuring the decision maker's objectives. A performance measure is then defined for each objective. For example, product price in dollars might be chosen as the performance measure for cost, and the number of years the product will last might be the performance measure for quality. The performance measures are the attributes for the multi-attribute utility function.

Deterministic vs. Probabilistic Decision Models

When building a model, the analyst can decide whether to ignore uncertainties and make the model deterministic, or, alternatively, to make the model probabilistic by assigning probabilities to one or more uncertain variables. In either case, a decision maker's preferences can be modeled with MODA. However, by convention, if the model is deterministic, the utility function is termed a value function [37,38]. A value function is simply a special case of the utility function applicable when it is assumed that there is no uncertainty over decision outcomes. The value function measures preference under conditions of certainty. The utility function measures preference under conditions of uncertainty.

Metrics for Measuring Value

Metrics

Now that you've been introduced to the mathematics of value models, the appropriate metrics for measuring project value should be obvious. The appropriate metrics are the inputs to a multi-objective value model, which will be either a utility function or a value function specified to capture the values of your organization. The model will assign numbers (utiles) to projects such that the higher the number is, the more desirable the project is. As shown in the above example, provided that the value or utility function is additive and linear in money, the preference measure can be expressed in equivalent monetary units, the most convenient unit for project prioritization.

One way to obtain a value/utility function for your MODA value model is to elicit it directly from a decision maker. There are other ways as well, but there are times when eliciting the function from the decision maker is most convenient. The next page of this paper describes the interviewing process for eliciting a value or utility function from a decision maker.