Lee Merkhofer Consulting Priority Systems
Tools for Project Portfolio Management

Project Portfolio Management Tools:

Which Approach is Best?

Part 5: Decision Models

Part 1 identified available project portfolio management tools, Part 2 described key differences, Part 3 summarized costs and risks, and Part 4 identified inability to optimize the project portfolio as the weak link for most tools. This part describes the components of PPM tools, including the internal model a tool needs to evaluate and recommend projects.

Before purchasing or developing a project portfolio management (PPM) tool, it is helpful to understand tool components. A tool for PPM should include three basic components:


  1. A data management component for collecting, storing, and retrieving the information that serves as the foundation for decision making.
  2. A decision model that manipulates the data and translates the information into intelligence by providing insights into the consequences of making alternative choices and the value of those consequences.
  3. A reporting component that displays the results graphically and otherwise communicates them to the relevant parties.

Components 1 and 3 are within the domain of computer science, and software providers typically do a good job in these areas. Component 2 is the subject of decision analysis, a topic less familiar to most tool providers. Lack of a quality decision model is a critical weakness of many tools. Bubble diagrams, charts, and other graphic displays are great for displaying information, but they are not by themselves decision models.

Buying a tool that provides great data management and reporting capability won't necessarily improve decisions. It may just promote information overload. To provide an analogy, if you are captaining a boat, you've got instruments that can tell you the barometric pressure, wind direction, and temperature, but what you may really want is a weather forecast. You need a model to give you that, so to get it, you go to a weather forecaster (who obtains the forecast from a weather model). Likewise, to obtain a forecast of the value to be produced if a project is accepted you need a model, not just a bunch of data about that project.

A good tool is one that is sufficiently good in all three components: data management, decision model, and reporting. As the old saying asks, "If you needed surgery, would you rather be operated on by a surgeon with a butcher knife, or by a butcher with a scalpel?" Using a data intensive tool with impressive graphics is risky if the tool is not based on a quality decision model.

Modeling

An analytic model is a mathematical construct, typically implemented as a computer program, that describes the behavior of some system of interest. In the case of a decision model, the system of interest is the problem of choosing projects that will create maximum value. An appropriate decision model for PPM provides predictions about how effective the various project alternatives will be in creating value. If the model is a good one, an alternative that the model says will create value is likely to do so in the real world.

The key in constructing any model is to abstract from the real-world situation the basic elements and relationships needed to describe the behavior of the whole without losing or obscuring important effects. Advances in the art and science of modeling — mathematical, symbolic, graphic, etc. — provide the means for exploring the structure, dynamics, and interactions that make up the decision problem that we wish to understand. The model-builder represents these interactions in a model. The model captures and makes explicit essential beliefs about "how things work."

Why Models Work

Models are useful because they address fundamental limitations of human problem solving. Research [1] shows that humans have limited information processing skills, can be biased, and are often inconsistent when making choices. People are good at creative tasks like generating alternatives. They are also good at recognizing structure and at making the sort of "small", well-defined judgments that are required in order to provide inputs to models.

Models work because they break a complex problem into pieces, allowing people to do what they do best while enabling computers to do the calculations.

If you have understanding, you can create an analytic model. You can't manage what you can't understand. Therefore, you can create a model for any situation you can hope to be able to manage better. Creating the model is the feasible and necessary step that allows for much more effective project portfolio management.

Project Selection Decisions May Require Sophisticated Models

Because a successful decision model must capture every critical aspect of the decision, more complex decisions typically require more sophisticated models. “There is a simple solution to every complex problem; unfortunately, it is wrong” [2]. This reality creates a challenge for tool designers. Project decisions are often high-stakes, dynamic decisions with complex technical issues—precisely the kinds of decisions that are most difficult to model:

  • Project selection decisions are high-stakes because of their strategic implications. The projects a company chooses can define the products it supplies, the work it does, and the direction it takes in the marketplace. Thus, project decisions can impact every business stakeholder, including customers, employees, partners, regulators, and shareholders. A sophisticated model may be needed to capture strategic implications.
  • Project decisions are dynamic because a project may be conducted over several budgeting cycles, with repeated opportunities to slow, accelerate, re-scale, or terminate the project. Also, a successful project may produce new assets or products that create time-varying financial returns and other impacts lasting many years. A more sophisticated model is needed to address dynamics.
  • Project decisions typically produce many different types of impacts on the organization. For example, a project might increase revenue or reduce future costs. A project might impact how customers or investors perceive the organization; that is, whether or not a project is conducted might affect corporate image. A project might provide new capability or learning important to future success. Making good choices requires not just estimating the financial return on investment; it requires capturing all of the ways that projects add value. A larger and more sophisticated model is needed to account for all of the different types of potential impacts that project selection decisions can create.
  • Project decisions often entail risk and uncertainty. The significance of project risk and project deferral risk depends on the nature of that risk and on the other risks that the organization is taking. A more sophisticated model is needed to correctly deal with risk and uncertainty.

Though project selection decision models are necessarily sophisticated, they need not (and shouldn't be) be complex. What matters is that the critical considerations for choice be captured and properly incorporated into the decision making logic. Though a great many factors may be relevant, it is nearly always the case that only a few factors produce the biggest impacts on choice. So long as the decision model correctly identifies the "decision drivers" and incorporates the proper mathematics, then a relative small, compact model can provide project recommendations and other outputs with surprising accuracy.

A Two-Stage Decision Model

Decision models come in various forms. For project prioritization, a very useful form, illustrated below, is a decision model composed of two distinct but integrated components (sub models).

Decision Model

The first component is a simulation model that provides predictions of the consequences of conducting projects, based on, among other things, the characteristics of those projects, the needs they address, and the effectiveness of the projects in addressing those needs. For example, a simulation model might estimate the improvements to products or customer service that might result from a project, as well as the anticipated increases in revenue or reductions in cost, and other changes relevant to objectives that are important to the organization. If project and portfolio risk are important, it may be desirable to use a probabilistic model that provides a description of the uncertainty over project consequences.

The second component of the decision model is a value model. The value model translates the estimated consequences of conducting projects, produced by the simulation model, into measures of the value of those consequences to the organization. The value model needs to account for the relative importance of the various financial and non-financial objectives impacted by projects, the organization's willingness to accept risk, and the organization's time preference (the natural desire to postpone undesired outcomes and speed up desired outcomes).

Advantages of Separating the Two Stages

Structuring the decision model as a simulation model linked to a value model provides two important advantages. First, it enables individuals to contribute to the development of the decision model appropriately according to their knowledge, roles, and responsibilities. For example, technical experts can ensure that the simulation model captures best-understanding about how projects impact business outcomes—the organization's financial experts can verify that logic for generating financial estimates is correct, marketing experts can ensure that logic for estimating impacts on sales is reasonable, safety experts can ensure that the logic for estimating impacts on public or worker safety make sense, etc. Conversely, the organization's senior executives, those responsible for setting policy and direction, can provide the judgments needed to specify the value model, including the weights assigned to specify the relative worth of the various types of consequences to the organization.

Second, the two-part structure means that the evaluation of projects is explicitly based on forecasts of what the consequences of doing those projects will be. This ensures that predictions are grounded in reality. Over time, forecasts originally produced or provided for projects that are funded can be compared with the actual outcomes that occur. Based on results, the organization has the opportunity to learn, address errors, and improve the decision model over time.

Evaluating Tools Requires Evaluating Decision Models

I hope I've convinced you that the quality of a PPM tool depends critically on the quality of the tool's decision model. A tool with an inadequate decision model can mislead decision makers, potentially producing poorer and less defensible decisions than would be made without it. Part 6 provides criteria for evaluating tools and their underlying decision models.

Notes


  1. See, for example, R. Hogarth, Judgment and Choice, 2nd ed. New York Wiley, 1987.
  2. Various sources have been cited for variations of this popular quotation, including H. L. Mencken.