Home

More Papers

Project-Selection Decision Models

Printable version

So, if the goal is to choose projects based on value generated, what kinds of metrics reflect impacts on value? Many organizations have trouble answering this question. Organizations tend to measure what's easy to measure, not necessarily what's important or useful. Most organizations use a bottom-up approach. They define interesting metrics, but then can't come up with the equations or algorithms for computing project value based on those metrics. They end up using arbitrary aggregation equations, such as weighted summations, or vague and unjustified concepts, such as "balance" or "strategic alignment." Unless there is a logical way to combine metrics to determine the value added by projects, the metrics will not be of much help for obtaining value-maximizing project portfolios through making optimal project decisions. How do you determine the metrics and associated equations for computing the value added by projects?

Create a Decision Model

The answer is—You need to reverse the process, use a top-down approach and create a project-selection decision model [3]. A decision model is an analytic representation, typically implemented in software, that captures the key considerations for decision making and derives recommended choices from them. As illustrated on Figure 20, the three basic components of a decision model are: (1) the alternatives that are available (what you can do), (2) objectives and preferences (what you want), and (3) information and beliefs about what will happen depending on the alternative that is selected (what you know).

Figure 20:   A project-selection decision model represents the 3 components to any decision.

A decision model is constructed by systematically applying logic and clear and unbiased thinking. Basically, the modeling process involves expressing understanding about (1) what might likely happen depending on the alternatives that are selected, and (2) how desirable or undesirable those outcomes would be. The model is expressed in terms of variables and mathematical relationships. If the model is a project-selection decision model, it will take as input characteristics of the project, the project's anticipated impacts, and other factors, and produce as output the value that would be added by doing the project. As indicated in Figure 21, the model will typically consist of two sub-models. The first (labeled "simulation") generates the relevant consequences of doing the project. The second (labeled "valuation") computes the value of those consequences. Since the value of a project is what the organization would logically pay to obtain the project consequences, value is expressed in dollar (or other currency) units. The model may be used to identify those project choices that maximize value added.

Figure 21:   A project-selection decision model values projects in two steps.

Creating a decision model is as much art as science, however, an effective step-by-step model-building process based on multi-attribute utility analysis (MUA) (also called multi-attribute decision analysis or MADA) has been developed. The approach constructs a decision model "from the top down," beginning with the definition and structuring of objectives.

Structuring Objectives

Structuring objectives means appropriately identifying, characterizing, and organizing the objectives for conducting projects into an objectives hierarchy. The objectives in the hierarchy define the ways in which projects might create value, since a project that contributes to the achievement of any objective produces some amount of value depending on how much that particular objective is advanced and the importance of advancing that objective (compared to the advancement of other objectives). The key is to structure the objectives hierarchy in such a way that the mathematic equation for correctly aggregating the various value components (the aggregation equation) can be derived from the hierarchy. Procedurally, the process for constructing an objectives hierarchy begins with identifying the organization's highest-level, most fundamental objective. Then sub-objectives that specify or clarify how that fundamental objective may be achieved are identified. Typically, objectives are stated in terms of maximizing some desired, or minimizing some undesired, object of interest. The process of identifying and adding sub-objectives continues to the extent necessary to produce a comprehensive and sufficiently detailed mapping of how projects can create value.

For example, suppose the organization's fundamental objective is maximizing shareholder value. As shown previously (Figure 19), shareholder value may be regarded as the net present value (NPV) of the company's projected cash flows plus option value. Thus, a shareholder value objective could be split into two sub-objectives: (1) maximizing the NPV of future cash flows and (2) maximizing option value. Relevant sub-objectives for increasing NPV would include (1) increasing revenue and (2) decreasing costs. Sub-objectives for maximizing option value depend on the organization and the nature of its business, but generally such sub-objectives involve mechanisms for increasing the ability of the organization to identify and take advantage of future opportunities and avoid future risks, as well as other sub-objectives that, if achieved, would enhance investor expectations and, therefore, increase the market value of the company.

If, on the other hand, the fundamental objective of the organization is maximizing stakeholder value, then the objectives hierarchy may be established by identifying who the stakeholders are and then identifying and structuring their objectives. An example of an objectives hierarchy for a case wherein the organization's fundamental objective is to maximize value for stakeholders is provided in the next sub-section.

Much has been written about how to define, characterize, and structure decision objectives so that the mathematical form of aggregation equation for combining value components can be derived. Basically, the goal is to specify objectives that are (1) fundamental (a common error is to specify as an objective something that is only one means for achieving what is really desired), (2) well defined (such that people readily agree on the meaning of each objective), (3) measurable (it is possible to know and measure how well each objective is being achieved), (4) comprehensive (the set of objectives is sufficiently complete to capture all of the strengths and weaknesses of available alternatives), and (5) non-redundant (at any specified level of the hierarchy, there is no overlap or double counting among the objectives at that level). A serious, and all-too-common error made by those designing project selection and prioritization models is to assume that value components can be weighted and added when an analysis of the associated objectives hierarchy would prove that this is not the case.

Characterizing Distinctions

Once objectives have been structured, the next step for constructing a decision model is specifying measures for characterizing the distinctions that determine how well the objectives are being achieved. The distinctions that we seek to identify are the attributes or characteristics that we care about. For example, suppose the context is decision making by a utility that distributes electric power. Customers (and therefore the utility) care about power outages, so one distinction that matters when thinking about electricity delivery is whether or not power is available to the customer. Any measure we define must have at least two possible states, for example, "yes power" or "no power," but we can define measures to have many more states if it helps us to differentiate alternatives and measure how well we are achieving our objectives. For example, the measure for electric power to the customer could alternatively be defined as the voltage available from the customer's power outlet, which could be zero, 110 volts, 109 volts, 108 volts, etc.

Oftentimes, multiple measures are required to characterize with adequate precision the achievement of a decision objective. For example, another distinction that would matter in the context of customer outages would be how long it takes to get the customer back on line. Thus, outage duration could be characterized by the number of minutes that the customer is without power. In general, measures should be defined whenever (1) the degree to which the objective is achieved depends significantly on performance against that measure, (2) performance against the measure doesn't track or exactly correlate with performance against some other measure that has already been defined (i.e., you need the additional measure to distinguish possibilities), and (3) available understanding is sufficient to conclude that the level of performance against the measure will differ depending on the project choices that are made (i.e., the measure differentiates among the alternatives).

Measures must be defined for each decision objective. For example, customers care about the accuracy of their bills. Thus, the percentage of customer invoices with errors might be a relevant measure for an organization considering projects for improving customer billing. As an example relevant to measuring option value, customer brand loyalty might affect investor expectations. A possible measure for impact on brand loyalty might be the percent of customers making repeat purchases or the company's ranking in customer satisfaction surveys.

The selection of measures is one of the more creative aspects of model building. In addition to choosing measures that quantify the achievement of the objectives, measure selection should be based in part on the types of data the organization does or can collect to assess performance. Influence diagrams (illustrated in the next sub-section), can be very effective for identifying useful performance measures.

Describing Possibilities and Likelihoods

Measures that characterize the distinctions that matter provide the framework needed to capture project consequences. To illustrate the concept, imagine that we combine the measures that we've defined into a tree structure. The tree starts with a set of branches indicating the possible states for the first measure, which are connected to branches for the second measure, and so forth. The paths through the tree define all of the possibilities, and that set of possibilities is the same regardless of the order in which the tree has been drawn. Some of the possible combinations will, of course, be more desirable than others, and it will be the job of the valuation model to sort this out (see below). The job of the consequence simulation model is to capture how the choice of project alternatives affects the likelihood of the various performance combinations (paths through the tree).

Sometimes, a project choice that we make will ensure that we end up on a specific branch for some measure (reflecting a specific outcome or level of performance for that measure). For example, if the electric utility from the example above conducts a project to add a distribution line that will reach 100 new homes, and the number of residential customers receiving service is a measure of performance, then, assuming the project is successfully completed, the number of customers will be incremented by 100. If there is uncertainty about project consequences, a choice merely affects the likelihood of the various branches for that uncertain measure (e.g., the number of new customers that will be added is somewhere between 0 and 100, depending on how many of the new homes in the housing development that the line will serve will be built and sold to homeowners).

The above illustrates that one way to capture the relevant beliefs about the consequences of choices is to assign probabilities to the possible performance outcomes that correspond to the paths through the tree of possibilities. We could do this by assigning probabilities to the branches for the first measure, conditional probabilities to the branches for the second measure (that indicate how likely each level of performance is for the second measure given each possible level of performance for the first measure), and so forth. If we then multiply all of the probabilities along any path, we will obtain the probability of that scenario. Since the goal of the consequence model is to express our beliefs about how choices affect our ability to achieve the distinctions we care about, a modeling approach based on assigning probabilities to the branches in a tree of possibilities would provide a fully adequate consequence model.

Creating a Consequence Model

The purpose of above description is not to suggest that developing a tree structure and assigning probabilities is always the best way to construct a project consequence model. Rather, it is to demonstrate the basic problem that must be solved and to illustrate that there is at least one solution approach that will always work. Consequence models built on tree structures (e.g., decision trees and event trees) are, in fact, quite common. For example, for evaluating pharmaceutical projects, tree models are typically used to lay out the likelihood of success and timing and to simulate the outcomes of the various tests and regulatory approvals that are required.

There are, however, many other model forms that may be used to infer or simulate project consequences. The simplest approach is direct estimation (What do you think will happen if the project is conducted?). A common approach for the case where data are available on the outcomes of similar projects is to use statistical analyses to establish correlations and suggest cause-effect relationships between project characteristics and project consequences.

Importantly, there are many existing models for measuring performance against numerous types of business and organizational objectives. For example, perhaps the most common sub-model used in project evaluation is the financial, or "business case model". The typical project financial consequence model computes the NPV of cash flows from the project (free cash flows), accounting for the timing of incremental costs and revenues, taxes, the time value of money, and other considerations that affect the equivalent, current financial worth of the cash flows to the business. Thus, project consequence models typically include a cash-flow NPV model as the sub-model for measuring the project's direct financial benefits.

Other models have been developed for capturing understanding regarding many different processes, activities, and impacts relevant to evaluating project impacts on business objectives, and these models can likewise be utilized. For example, there are well-established models for estimating R&D success, the impact of project characteristics on customer choice, the rate at which new products penetrate exiting markets, the effectiveness of projects at reducing pollution, the productivity and reliability of business assets, and so on and so forth. Selecting, refining, and piecing together various sub-models that have worked well in other applications is the most common approach for creating project selection decision models.

Valuing Consequences

Project consequences must be valued in a common unit of measurement; otherwise they can't be combined to obtain a total project value. Since financial gain is an expected consequence for many projects, dollars (or another unit of currency) is the obvious choice for valuing all project consequences, including non-financial benefits. Because people regularly use money to make transactions, it is easy to interpret worth in monetary units. Also, unless we express project value in terms of dollars, we won't be able to determine whether project benefits justify project costs. Money is the clear choice for the unit for measuring project value.

One approach for determining the monetary value of potential project consequences would be to ask the organization's leaders to provide the numbers directly. However, the task would be difficult and it would be hard to obtain consistent answers. Thus, methods based on formal analysis are used instead. The worth of each component of value is separately assessed. Then, as noted above, provided that the value components are based on a well-constructed objectives hierarchy, an appropriate aggregation equation can be derived for combining the individual value components into a total.

Various theories and associated models exist for determining the worth of many types of outcomes that result from doing projects. Oftentimes, the methods are derived directly from a few simple assumptions and are well-established. For example, the discounted cash flow model computes an equivalent, current monetary value for a time stream of future payments based on discounting. The discounted cash flow model can be derived from two assumptions: (1) that investors wish to maximize their wealth and (2) that a market exists that will provide a return from investing cash.

Valuation methods likewise exist for many non-financial project consequences. The three best known methods for assessing the equivalent dollar value of non-financial outcomes are (1) cost benefit analysis, which typically derives the dollar value of outcomes based on market prices, contingent valuation (people's willingness to pay), and the hedonic price method (analyzing market prices to determine how factors impact market prices), (2) multi-attribute utility analysis (and related multi-criteria methods, such as AHP), which assign values based on modeling people's preferences and their willingness to make tradeoffs, and (3) real options analysis, which includes in the valuation the flexibility made available or destroyed by project decisions. These methods, which have long been used to quantify shareholder and stakeholder value, are described in detail in many articles and books. Each includes capability to include considerations of risk as part of the valuation (see Part 4 of this paper).

For potentially controversial dollar conversions, such as for health and environmental impacts, rather than assign values directly, most organizations find it convenient and less controversial to leverage results from other sources, including academic research, government recommended values, and values adopted by other organizations. Sensitivity analysis, that is, varying assumed values across a range to see whether it changes decision recommendations, is commonly used to demonstrate that most project decisions are not overly sensitive to the dollar conversions assumed for specific project outcomes.

Two Views on Value

How Difficult Is It?