Home

More Papers

Project-Selection Decision Models

Printable version

So, if the goal is to choose projects based on value generated, what kinds of metrics reflect impacts on value? Many organizations have trouble answering this question. Organizations tend to measure what's easy to measure, not necessarily what's important or useful. Most organizations use a bottom-up approach. They define interesting metrics, but then can't come up with the equations or algorithms for computing project value based on those metrics. They end up using arbitrary aggregation equations, such as weighted summations, or vague and unjustified concepts, such as "balance" or "strategic alignment." Unless there is a logical way to combine metrics to determine the value added by projects, the metrics will not be of much help in identifying go/no-go decision rules or in choosing value-maximizing project portfolios. How do you determine the metrics and associated equations for computing the value added by projects?

Create a Decision Model

The answer is—You need to reverse the process, use a top-down approach and create a project-selection decision model [3]. A decision model is an analytic representation, typically implemented in software, that captures the key considerations for decision making and derives recommended choices from them (Figure 20). The 3 decision components of a decision are: (1) the alternatives that are available to you, (2) what you want, and (3) what you know and believe about how the choices that you make affect the outcomes that you will receive.

Figure 20:   A project-selection decision model values projects in two steps.

The decision model is structured using logic and incorporates principles for clear and unbiased thinking. From a mechanical standpoint, the model that you create takes as input characteristics of the project, the project's anticipated impacts, and other factors, and produces as output a dollar measure of the value added by the project (Figure 21). The model then seeks those choices that maximize value added.

Figure 21:   A project-selection decision model values projects in two steps.

As indicated by the by Figure 21, a decision model values projects in two steps. First, the model estimates project consequences relevant to the achievement of the organization's fundamental objectives (e.g., creating shareholder or stakeholder value). Project consequences are estimated based on cause-effect reasoning and inference (in the figure, this step is labeled "(simulation"). Then, those consequences are translated into the worth to the organization of obtaining those consequences (the "valuation" step). The consequences are valued based on a formal theory of valuation.

Defining Objectives

Project selection decision models are constructed "from the top down" because the first step is structuring objectives. Structuring objectives means creating a an objectives hierarchy by first identifying the organization's fundamental objective, identifying sub-objectives that specify or clarify how that fundamental objective may be achieved, and continuing to define additional sub-objectives as needed or to the extent that it is helpful.

For example, suppose the organization's fundamental objective is creating stakeholder value. As shown previously (Figure 19), shareholder value may be regarded as the NPV of the company's projected cash flows plus the company's option value. Thus, a shareholder value objective could be split into two sub-objectives: (1) maximizing the NPV of future cash flows and (2) maximizing option value. Relevant sub-objectives for increasing NPV would include (1) increasing revenue and (2) decreasing costs. Sub-objectives for maximizing option value depend on the organization and the nature of it's business, but generally such sub-objectives involve mechanisms for increasing the ability of the organization to identify and take advantage of future opportunities and avoid future risks, as well as other sub-objectives that, if achieved, would enhance investor expectations and, therefore, increase the market value of the company.

If, on the other hand, the fundamental objective of the organization is maximizing stakeholder value, then the objectives hierarchy may be established by identifying who the stakeholders are and then identifying and structuring their objectives. An example of an objectives hierarchy for a case wherein the organization's fundamental objective is to create value stakeholders is provided in the next sub-section.

Much has been written about how to define, characterize, and structure decision objectives. Basically, the goal is to specify objectives that are (1) fundamental (a common error is to specify as an objective something that is only one means for achieving what is really desired), (2) well defined (such that people readily agree on the meaning of each objective), (3) measurable (it is possible to know and measure how well each objective is being achieved), (4) comprehensive (the set of objectives is sufficiently complete to capture all of the strengths and weaknesses of available alternatives), and (5) non-redundant (at any specified level of the hierarchy, there is no overlap or double counting among the objectives at that level).

Characterizing Distinctions

Once objectives have been structured, the next step for constructing a decision model is specifying measures for characterizing the distinctions that determine how well the objectives are being achieved. The distinctions that we seek to identify are the attributes or characteristics that we care about. For example, suppose the context is decision making by a utility that distributes electric power. Customers (and therefore the utility) care about power outages, so one distinction that matters when thinking about electricity delivery is whether or not power is available to the customer. Any measure we define must have at least two possible states, for example, "yes power" or "no power," but we can define measures to have many more states if it helps us to differentiate alternatives and measure how well we are achieving our objectives. For example, the measure for electric power to the customer could alternatively be defined as the voltage available from the customer's power outlet, which could be zero, 110 volts, 109 volts, 108 volts, etc.

Oftentimes, multiple measures are required to characterize with adequate precision the achievement of a decision objective. For example, another distinction that would matter in the context of customer outages would be how long it takes to get the customer back on line. Thus, outage duration could be characterized by the number of minutes that the customer is without power. In general, measures should be defined whenever (1) the degree to which the objective is achieved depends significantly on performance against that measure, (2) performance against the measure doesn't track or exactly correlate with performance against some other measure that has already been defined (i.e., you need the additional measure to distinguish possibilities), and (3) available understanding is sufficient to conclude that the level of performance against the measure will differ depending on the choice that is made (i.e., the measure differentiates among the alternatives).

Measures must be defined for each decision objective. For example, customers care about the accuracy of their bills. Thus, the percentage of customer invoices with errors might be a relevant measure for an organization concerned about improving customer billing. As an example relevant to measuring option value, customer brand loyalty might affect investor expectations. A possible measure for impact on brand loyalty might be the percent of customers making repeat purchases or the company's ranking in customer satisfaction surveys.

The selection of measures is one of the more creative aspects of model building. In addition to choosing measures that quantify the achievement of the objectives, measure selection should be based in part on the types of data the organization does or can collect to assess performance. Influence diagrams (illustrated in the next sub-section), are often used to identify project performance measures and other useful metrics.

Describing Possibilities and Likelihoods

Measures that characterize the distinctions that matter provide the framework needed to capture project consequences. To illustrate the concept, imagine that we combine the measures that we've defined into a tree. The tree starts with a set of branches indicating the possible states for the first measure, which are connected to branches for the second measure, and so forth. The paths through the tree define all of the possibilities, and that set of possibilities is the same regardless of the order in which the tree has been drawn. Some of the possible scenarios will, of course, be more desirable than others, and it will be the job of the valuation model to sort this out (see below). The job of the consequence simulation model is to capture how the choice of project alternatives affects the likelihood of the various scenarios (paths through the tree).

Sometimes, a project choice that we make will ensure that we end up on a specific branch (reflecting a specific outcome or level of performance) for a measure. For example, if the electric utility from the example above conducts a project to add a distribution line that will reach 100 new homes, and the number of residential customers receiving service is a measure of performance, then, assuming the project is successfully completed, the number of customers will be incremented by 100. If there is uncertainty about project consequences, a choice merely affects the likelihood of the various branches for that uncertain measure (e.g., the number of new customers that will be added is somewhere between 0 and 100, depending on how many of the new homes in the housing development that the line will serve will be built and sold to homeowners).

The above illustrates that one way to capture the relevant beliefs about the consequences of choices is to assign probabilities to the possible performance outcomes that correspond to the paths through the tree of possibilities. We could do this by assigning probabilities to the branches for the first measure, conditional probabilities to the branches for the second measure (that indicate how likely each level of performance is for the second measure given each possible level of performance for the first measure), and so forth. If we then multiply all of the probabilities along any path, we will obtain the probability of that scenario. Since the goal of the consequence model is to express our beliefs about how choices affect our ability to achieve the distinctions we care about, a modeling approach based on assigning probabilities to the branches in a tree of possibilities would provide a fully adequate consequence model.

Creating a Consequence Model

The purpose of above description is not to suggest that developing a tree structure and assigning probabilities is always the best way to construct a project consequence model. Rather, it is to demonstrate the basic problem that must be solved and to illustrate that there is at least one solution approach that will always work. Consequence models built on tree structures are, in fact, quite common. For example, for evaluating pharmaceutical projects, tree models are typically used to lay out the likelihood of success and timing and to simulate the outcomes of the various tests and regulatory approvals that are required.

There are, however, many other model forms that may be used to infer or simulate project consequences. The simplest approach is direct estimation (What do you think will happen if the project is conducted?). A common approach for the case where data are available on the outcomes of similar projects is to use statistical analyses to establish correlations and suggest cause-effect relationships between project characteristics and project consequences.

Importantly, there are many existing models for estimating various types of business consequences. For example, perhaps the most common sub-model used in project evaluation is the financial, or "business case model". The typical project financial consequence model computes the NPV of cash flows from the project (free cash flows), accounting for the timing of incremental costs and revenues, taxes, the time value of money, and other considerations that affect the equivalent, current financial worth of the cash flows to the business. Thus, project consequence models typically include a cash-flow NPV model as the sub-model for measuring the project's direct financial benefits.

Other models have been developed for capturing understanding regarding many different processes, activities, and impacts relevant to evaluating project impacts on business objectives, and these models can likewise be utilized. For example, there are well-established models for estimating R&D success, the impact of project characteristics on customer choice, the rate at which new products penetrate exiting markets, the effectiveness of projects at reducing pollution, the productivity and reliability of business assets, and so on and so forth. Selecting, refining, and piecing together various sub-models that have worked well in other applications is the most common approach for creating project selection decision models.

Valuing Consequences

Project consequence must be valued in a common unit of measurement; otherwise they can't be combined to obtain a total project value. Since financial gain is an expected consequence for many projects, dollars (or another unit of currency) is the obvious choice for valuing all project consequences, including non-financial benefits. Also, unless we express project value in terms of dollars, we won't be able to determine whether project benefits justify project costs. Money is the clear choice for the unit for measuring value.

One approach for determining the monetary value of potential project consequences would be to ask the organization's leaders to provide the numbers. However, the task would be difficult and it would be hard to obtain consistent answers. Thus, methods based on formal analysis are used. Various theories and associated models exist for determining how much decision makers should be willing to pay for something, assuming certain assumptions apply. For example, the discounted cash flow model computes a current value for a project that generates a time stream of future payments based on discounting. The discounted cash flow model can be derived from two assumptions: (1) that investors wish to maximize their wealth and (2) that a market exists that will provide a return from investing cash.

Other valuation methods exist for valuing projects that produce non-financial as well as financial project consequences. The three best known methods for assessing the equivalent dollar value of decision outcomes are (1) cost benefit analysis, which derives the dollar value of outcomes based on contingent valuation (people's willingness to pay) and the hedonic price method (analyzing market prices to determine how factors impact market prices), (2) multi-attribute utility analysis (and related multi-criteria methods, such as AHP), which assign values based on modeling people's preferences and their willingness to make tradeoffs, and (3) real options analysis, which includes in the valuation the flexibility made available or destroyed by project decisions. These methods, which have long been used to quantify shareholder and stakeholder value, are described in detail in many articles and books. Each includes capability to include considerations of risk as part of the valuation (see Part 4 of this paper).

For potentially controversial dollar conversions, such as for health and environmental impacts, rather than assign values directly, most organizations find it convenient and less controversial to leverage results from other sources, including academic research, government recommended values, and values adopted by other organizations. Sensitivity analysis, that is, varying assumed values across a range to see whether it changes decision recommendations, is commonly used to demonstrate that most project decisions are not overly sensitive to assumed dollar conversions.

Defining Value

How Difficult Is It?