Lee Merkhofer Consulting Priority Systems
Implementing project portfolio management

Is It Hard to Create a Project Decision Model?

The prospect of creating a model that quantifies the dollar value of projects no-doubt sounds hard. Building a decision model requires developing and documenting understanding of what your organization does, how it does it, and how the choices that are made determine the value that is created. This understanding is critical to knowing what to do to create organizational success. Thus, one could argue that using decision modeling to identify project evaluation metrics merely forces the organization to do what it should be doing anyway.

Actually, building a decision model is not as difficult as it may sound. In my experience, the basic design for even a sophisticated decision model can be structured in a 3-day framing workshop (using techniques based on value modeling, influence diagramming, and cause effect reasoning). The resulting qualitative model can then be quantified fairly quickly by refining and piecing together pre-existing sub-models and using well-established relationships for estimating various types of benefits. As noted previously, in addition to project financial value, standardized sub-models are available for quantifying health, safety, and environmental value; customer value derived from new products and services, brand image value; learning value; and so forth. The combined model captures business understanding in relevant areas such as R&D, engineering, manufacturing, marketing, sales, IT, customer relations, legal counsel, regulatory affairs, etc. The result establishes an explicit connection between the project, the impacts the project would create, and the value of those impacts.

An Example

Figure 25 provides an example. The figure is a graphical representation of a portion of a project-selection decision model developed by a team at an electric utility. (The figure provides detail only for that portion of the model for measuring project value attributed to improving electric service reliability.)


Example project-selection decision model

Figure 25:   Portion of a project selection decision model for an electric power delivery company.



The upper part of the figure is a hierarchy of objectives for selecting projects. As indicated, the utility adopted stakeholder value as its overall objective. Key stakeholders were identified to be shareholders, customers, workers, citizens, and others (e.g., business partners, elected officials, some state and federal agencies, etc.). The sub-objectives for stakeholder value represent the fundamental concerns of the various stakeholders: (1) the utility's financial performance; (2) health, safety and the environment; (3) satisfying customers (both existing customers who want high-quality service and anticipated new customers, e.g., people who might live in future housing developments) who will desire electric service; (4) satisfying other stakeholders (e.g., responding to regulator concerns and maintaining a good image with the citizens of local communities), and (5) building a platform for future success (providing learning, improved capability, flexibility, ability to respond quickly, etc.).

The middle part of the figure is the influence diagram constructed for the service reliability objective. It identifies the factors, relationships and metrics assumed to determine the level of satisfaction derived by customers based on the reliability of the electric service provided. As indicated by the nodes and arrows in the diagram, the utility believes that a customer's level of satisfaction with service reliability depends primarily on the frequency and duration of the outages that the customer experiences—the fewer outages experienced and the shorter the duration of those outages, the happier the customer will be.

The bottom portion of the figure indicates how the reliability component of the decision model was quantified. As indicated, the service reliability sub-model takes as input the metrics identified in the influence diagram and produces as output its measure of reliability performance: "total weighted annual customer minutes out." This performance measure assumes that the level of customer dissatisfaction is roughly proportional to the total amount of time that the customer is without power (some utilities use a scaling function to indicate that particularly long outages, for example, outages that would cause food in refrigerators to spoil or companies to send workers home is proportionally more costly). Finally, through weights assigned to different customer types, the model reflects the assumption that outages are more harmful or costly for some types of customers than for others. This utility chose the weights for residential, commercial and industrial customers based on industry studies estimating the actual dollar costs of outages to these various types of customers. In addition, the utility defined "critical customer" to serve as a special category with a very high weight for customers, such as a hospital, where reliability of service is judged to be especially important.

The sub-model for measuring reliability benefit works as follows. If a proposed project will impact service reliability, the project proponent estimates the scope of impact (How many residential, commercial, industrial, and critical customers will be impacted?). These numbers can be derived based on the configuration of the utility's distribution network and the specific components of the network that the project will impact. In addition, forecasts are provided for the number and duration of outages experienced by these customers (1) if the project is not conducted and (2) if the project is conducted. These forecasts are made by extrapolating data on historical outages within the project scope and based on estimates for how the proposed project will affect system reliability. The measure of project reliability benefit is total weighted customer outage minutes avoided, and the value is expressed in dollar terms based on the weights assigned to the customer types

To illustrate another aspect of this decision model, Figure 26 shows how the value of serving unmet energy needs was quantified. The assumption is that providing power to new customers creates value for those customers because the value derived is greater than the price charged. (This principle is termed "consumer surplus" by economists.) To estimate the value of projects that provide this benefit, the following logic is used. First, the expected shortfall in energy service is estimated based on forecasts of growing energy demands relative to the capacity constraints of the existing electric distribution network. . If increments to capacity are not put in place, new homes will not be built and businesses will not expand because the required electric service cannot be assured. In the utility industry, the shortfall in delivery relative to forecast demand is referred to as expected unserved energy (EUE), and it is measured in kilowatt hours.


A sub-model for project value

Figure 26:   The utility's sub-model for new energy delivery.



The black curve in Figure 23 shows the results of such a forecast of EUE (the "jumps" in the curve in the early years correspond to the completion of known residential and commercial development projects; the curve increases smoothly in later years because in this time frame the forecast is based only on estimated average annual growth rates).

The second step is to forecast the EUE that would exist if the project is conducted. The EUE with the project (the red curve shows a sample forecast) is obtained based on the schedule by which the project would bring new capacity on line ( this determines the shape of the curve in the initial years), the total amount of new capacity provided by the project, and the point in time when the new capacity would be fully utilized (this determines the gap between the two curves in the later years). The difference between the two curves represents the EUE avoided (in kilowatt hours). To convert EUE avoided to a dollar value, the utility again uses data from willingness-to-pay surveys.

Too Complicated?

If this sounds a bit complicated, consider that the utility using the above-described decision model spends hundreds of millions of dollars each year on new projects. Yet, limitations on resources means that many project proposals must be delayed or killed. If the improved decisions that result from the formal decision model and associated priority system only increase the value derived from projects by a few percentage points, the required effort is easily justified by the value gained in just the first year of model use. Furthermore, a quality decision model can be a very effective way of explaining decisions. Managers increasingly are put in the position of having to explain, "Why we did what we did." Finally, the system enables managers to demonstrate exactly why they need the resources they are requesting, the benefits that they expect to deliver, and the consequences of trying to perform with too few resources. The utility in the above example has presented its decision model to its regulators and used model results to help explain and defend its project choices as well as proposed rate increases that would enable it to conduct additional projects.

Less complicated decision models are typical for organizations with smaller project budgets, less complicated decisions, and less need to defend choices to outsiders. However, the basic principles for building the decision model are the same.

Although organizations may be initially uncertain about many of the relationships that must be specified to define their project decision models, understanding these relationships is key to success. Again, the management science and industry science literature contains numerous, often business-specific sub-models for estimating and measuring performance. Thus, as noted previously, the process of defining the relevant computations typically involves selecting and combining elements of previously developed and accepted approaches. Organizations that create explicit project-selection decision models document current best-understanding. As understanding improves, they revise their models and thereby further improve their ability to optimize their project portfolios.

Decision Model Uses

Creating a decision model takes work, but it is worth it. Having a decision model is critical to ensuring consistent, intelligent choices. Knowing project value allows you to determine whether the project should be done at all, and whether, after it has been started, it should be continued. Knowing the value of your various project portfolios tells you whether you are allocating too much or tool little to each, and enables you to determine the right allocation of resources across your various organizational units and business functions.

A decision model has other uses as well. For example, a decision model provides a way to estimate the value of a day of additional effort, the value of a product feature, or the value created by expending a dollar more of project cost. The project team or portfolio manager can use the decision model to illustrate how a marginal change in resources, say plus or minus 10%, might affect the overall value to be generated. A decision model is a means for explaining and justifying the resources required for doing projects.

Finally, the decision model sends important signals to those who propose and manage projects. The decision model tells engineers and others what project characteristics and attributes are valued in the funding process. It tells managers who execute funded projects what performance is expected if the project is to create the value that motivated its funding.