Home

More Papers

Project-Selection Decision Models

Printable version

So, what kinds of metrics reflect impacts on value? Many organizations have trouble answering this question. Organizations tend to measure what's easy to measure, not necessarily what's important. Most organizations use a bottom-up approach. They define interesting metrics, but then can't come up with the algorithms for computing value added based on those metrics. They end up using arbitrary aggregation equations, such as weighted summations, or vague and unjustified concepts, such as "balance" or "alignment." Unless there is a logical way to combine metrics to determine the value added by projects, the metrics will not be of much help in identifying go/no-go decision rules or in choosing value-maximizing project portfolios. How can you determine the value added by projects?

Create a Decision Model

The answer is—You need to reverse the process, use a top-down approach and create a project-selection decision model [3]. A decision model is an analytical model that takes as input characteristics of the project, the project's anticipated impacts, and other factors, and produces as output a dollar measure of the value added by they project (Figure 21).

Figure 21:   A project-selection decision model values projects in two steps.

As indicated by the figure, a decision model values projects in two steps. First, project consequences relevant to the achievement of the organization's fundamental objectives (shareholder or stakeholder value) are estimated based on cause-effect reasoning (simulation). Then, those consequences are translated into a measure of the value that the organization will derive from them. The consequences are valued based on a formal theory of valuation.

Decision models are constructed "from the top down" because the first step is clarifying and structuring objectives. For example, suppose the organization's fundamental objective is shareholder value. As shown in the last section, shareholder value may be regarded as the sum of the NPV of projected cash flows plus option value. Thus, relevant sub-objectives (lower-level objectives that explain how the fundamental objective may be achieved) include the financial objectives of increasing revenue and decreasing costs, plus other objectives that, depending on the context, affect investor perceptions. (The sub-objectives that affect option value depend on the company and the nature of its business.) For example, a relevant non-financial objective affecting option value might be customer brand loyalty. If brand loyalty is a relevant objective, then project metrics would be chosen that measure brand loyalty, for example, percent of customers making repeat purchases or company rankings in customer surveys. Like the underlying objectives, the relevant metrics depend on the company and also on the data potentially available to it.

If the fundamental objective is maximizing stakeholder value, then the first step for constructing the decision model is identifying stakeholders and understanding their objectives. Relevant metrics measure the ways that stakeholders may be impacted by the organization's projects. For example, a supplier may care about the accuracy of bills, so percent of invoices with errors might be a relevant metric for that stakeholder. The decision model values impacts based on assumptions about whether those impacts are desired or undesired by each type of stakeholder and by how much. As with decision models for shareholder value, a model for stakeholder value quantifies impacts on stakeholders in units of dollars.

A decision model can incorporate various methods for simulating project consequences. The simplest approach is direct estimation (What do you think will happen if the project is conducted?). If data are available, statistical analyses can be conducted to establish relationships between measures and the achievement of some objectives. More sophisticated methods are available in the form of the well-established models that are available for simulating certain types of project consequences. For example, the likelihood of success and timing for a project to create a new drug are often estimated based on a model that simulates the outcomes of the various tests and regulatory approvals that are required.

Logic can be used to create simple simulation models. For example, if you believe IT projects improve the organization's capability to launch new products and services, you might want to favor projects that have the broadest impact, provide the most potential, and add value the soonest. Thus, you might want metrics that indicate scope of impact (e.g., fraction of the company's existing and potential markets where new products might be enabled), the magnitude of the opportunities potentially created (e.g., Could sales in an impacted market increase by 10%,? By 50%?), and the timing when such opportunities might be available (Within 1 year? Within 5 years?).

A very common simulation model is the financial model— The financial model estimates the net cash flows (free cash flows) from the project, accounting for the timing of incremental costs and revenues, taxes, and other considerations that affect the actual financial benefits that the company would obtain from the project.

With regard to consequence valuation, well-established methods are available for determining the dollar value of project consequences. Real options and multi-attribute utility analysis may be used to quantify impacts on shareholder value. Multi-attribute utility analysis and variations of cost-benefit analysis are often used to quantify stakeholder value.

Is it Hard to Create a Project Decision Model?

The prospect of creating a model that quantifies the dollar value of projects no-doubt sounds hard. Building a decision model requires developing and documenting understanding of what your organization does, how it does it, and how the choices that are made determine the value that is created. This understanding is critical to knowing what to do to create organizational success. One could argue that using decision modeling to identify project evaluation metrics merely forces the organization to do what it should be doing anyway.

Actually, building a decision model is not as difficult as it may sound. A sophisticated decision model can be structured in a 3-4 day framing workshop (using techniques based on value modeling, influence diagramming, and cause effect reasoning). The resulting qualitative model can then be quantified fairly quickly using what are often well-defined relationships for estimating various types of benefits. The model captures the understanding of the organization's experts in relevant areas such as R&D, engineering, manufacturing, marketing, sales, customer relations, legal counsel, regulatory affairs, etc. The decision model establishes an explicit connection between the characteristics of the business that may be impacted by proposed projects and the value ultimately derived.

An Example

Figure 22 provides an example. The figure is a graphical representation of a portion of a project-selection decision model that I helped a team from an electric power delivery company develop. (The figure provides detail only for that portion of the model for measuring project value attributed to improving the satisfaction of existing customers by improving service reliability.)

Figure 22:   Graphic representation of a portion of a decision model for an electric power delivery company.

The upper part of the figure is a hierarchy of objectives for selecting projects. As indicated, the utility adopted stakeholder value as its overall objective. Key stakeholders were identified to be shareholders, customers, workers, citizens, and others (e.g., business partners, elected officials, some specific state and federal agencies, etc.). The sub-objectives for stakeholder value represent the fundamental concerns of the various stakeholders: (1) the utility's financial performance; (2) health, safety and the environment; (3) satisfying customers (both existing customers who want high-quality service and anticipated new customers (e.g., people who might live in future housing developments) who will require new service; (4) satisfying other stakeholders (e.g., responding to regulator concerns and maintaining a good image with the citizens of local communities), and (5) building a platform for future success (providing learning, improved capability, flexibility, ability to respond quickly, etc.).

The lower part of the figure is the influence diagram constructed for one of the objectives. It identifies the factors and relationships believed to determine the level of customer satisfaction based on the reliability of service provided. As indicated by the nodes and arrows in the diagram, the utility believes customer satisfaction depends primarily on the frequency and duration of outages and on the number of voltage sags ("brown outs") that customers experience.

As represented by this decision model, when the utility prioritizes projects to upgrade its electric distribution network, the utility considers not just financial benefits (saving in maintenance spending and incremental revenue), it also considers improved customer satisfaction as a component of project value. To evaluate the latter, the model takes as input project-specific estimates of the number and types of customers whose service quality will be improved and the nature of the expected improvements. For example, is the project expected to reduce the frequency or decrease the duration of outages experienced by a specific group of customers? If so, then the estimated improvements in service reliability are quantified and then weighted based the number and type of impacted customers (Are they residential customers or large commercial customers?). To obtain a dollar value of the estimated improvements in reliability, the utility uses data on the costs of outages to customers and surveys indicating what customers are willing to pay for improved power reliability.

As further illustration, Figure 23 shows how another portion of the model is quantified. As indicated by the sub-objectives shown in Figure 22, serving customers' need for energy is a component of value created by projects. (The value of electric power to customers is greater than the price they pay. Thus, providing power to customers creates value for those customers. This principle is termed "consumer surplus" by economists.). To estimate the value of satisfying otherwise unmet energy needs, the following logic is used. First, the expected shortfall in energy service is estimated based on forecasts of growing energy demands relative to the capacity constraints of the existing electric distribution network. In the utility industry, the shortfall is referred to as expected unserved energy (EUE), and it is measured in kilowatt hours.

Figure 23:   Sub-model for valuing the delivery of new energy.

The black curve in Figure 23 shows the results of such a forecast of EUE (the "jumps" in the curve in the early years correspond to the completion of known residential and commercial development projects; the curve increases smoothly in later years because in this time frame the forecast is based only on forecast average annual growth rates).

The second step is to forecast the EUE that would exist if the project is conducted. The EUE with the project (the red curve shows a sample forecast) is obtained based on the schedule by which the project would bring new capacity on line ( this determines the shape of the curve in the initial years), the total amount of new capacity provided by the project, and the point in time when the new capacity would be fully utilized (this determines the gap between the two curves in the later years). The difference between the two curves represents the EUE avoided (in kilowatt hours). To convert EUE avoided to a dollar value, the utility again uses data from willingness-to-pay surveys.

Too Complicated?

If this sounds a bit complicated, consider that the utility using the above-described decision model spends hundreds of millions of dollars each year on new projects. Yet, limitations on resources means that many project proposals must be delayed or killed. If the improved decisions that result from the formal decision model and associated priority system only increase the value derived from projects by a few percentage points, the required effort is easy justified by the value gained in just the first year of use. Furthermore, a decision model can be a very effective way of explaining decisions and justifying the need for project resources. The utility in the above example has presented its decision model to its regulators and used model results to help explain and defend proposed rate increases to its state utility commission.

Less complicated decision models are typical for organizations with smaller project budgets, less complicated decisions, and less need to defend choices to outsiders. However, the basic principles for building the decision model are the same.

Although organizations may be initially uncertain about many of the relationships that must be specified to define their project decision models, understanding these relationships is key to success. Organizations that create decision models document their current best-understanding. As understanding improves, they revise their models and thereby further improve their ability to optimize their project portfolios.

Decision Model Uses

Creating a decision model takes work, but it is worth it. Having a decision model is critical to making intelligent choices. Knowing project value allows you to determine whether the project should be done at all, and whether, after it has been started, it should be continued. Knowing the value of your various project portfolios tells you whether you are allocating too much or tool little to each, and enables you to determine the right allocation of resources across your various organizational units and business functions.

A decision model has other uses as well. For example, a decision model provides a way to estimate the value of a day of additional effort, the value of a project feature, or the value created by expending a dollar more of project cost. The project team or portfolio manager can use the decision model to illustrate how a marginal change in resources, say plus or minus 10%, might affect the overall value to be generated. A decision model is a means for explaining and justifying the resources required for doing projects.

Finally, the decision model sends important signals to those who propose and manage projects. The decision model tells engineers and others what project characteristics and attributes are valued in the funding process. It tells managers who execute funded projects what performance is expected if the project is to create the value that motivated its funding.

Defining Value

The Scientific Method