Part 1 identified available project portfolio management tools, Part 2 described key differences, Part 3 summarized costs and risks, and Part 4 identified inability to optimize the project portfolio as the weak link for most tools. This part describes the components of PPM tools, including the internal model a tool needs to evaluate and recommend projects.
Before purchasing or developing a project portfolio management (PPM) tool, it is helpful to understand tool components. A tool for PPM should include three basic components:
Most Tool Evaluation Guides Pay Scant Attention to Models
Despite the importance of the model used to evaluate and prioritize projects, most guides for selecting project portfolio management tools focus mainly on data management and reporting capabilities. For example, one guide advises potential buyers to consider whether the tool "allows users to easily update forms," "annotate assumptions," "drill down," "collaborate with other team members," "retain historic information for trend analysis," "change the axis, bubble size, and color of displays," etc.
With regard to the decision model, the guide suggests only that tools should "allow the user to specify their own company-specific formulas for computing priority...[based on] ROI, EVA, NPV or RONA." The guide fails to indicate, however, that using any of these to prioritize projects would be a serious mistake, since none of the mentioned metrics provides a correct basis for prioritizing projects. Thus, a tool providing only capability to apply such metrics won't enable you to accomplish the fundamental PPM goal of maximizing the value of the project portfolio. (See the Glossary and paper on Mathematical Theory).
Components 1 and 3 are within the domain of computer science, and software providers typically do a good job in these areas. Component 2 is the subject of decision analysis, a topic less familiar to most tool providers. Lack of a quality decision model is a critical weakness of many tools. Bubble diagrams, charts, and other graphic displays are great for displaying information, but they are not by themselves decision models.
Buying a tool that provides great data management and reporting capability won't necessarily improve decisions. It may just promote information overload. To provide an analogy, if you are captaining a boat, you've got instruments that can tell you the barometric pressure, wind direction, and temperature, but what you may really want is a weather forecast. You need a model to give you that, so to get it, you go to a weather forecaster (who obtains the forecast from a weather model). Likewise, to obtain a forecast of the value to be produced if a project is accepted you need a model, not just a bunch of data about that project.
A good tool is one that is sufficiently good in all three components: data management, decision model, and reporting. As the old saying asks, "If you needed surgery, would you rather be operated on by a surgeon with a butcher knife, or by a butcher with a scalpel?" Using a data intensive tool with impressive graphics is risky if the tool is not based on a quality decision model.
Types of Decision Models
Decision models can be categorized in various ways. One taxonomy separates deterministic models from probabilistic (also called stochastic) models and static models from dynamic models. In a deterministic model, all relevant data are assumed to be known with certainty. Probabilistic models incorporate uncertainty via probabilities. Static models ignore time, while dynamic models represent the sequence by which changes occur.
Deterministic models can more easily handle situations where there are many decisions that must be made simultaneously and where there are many constraints on what options can be chosen. Examples of deterministic models include difference equations, typical cost-benefit analysis and linear programming. Examples of probabilistic models include event trees and Markov models. Often times, a deterministic model can be made probabilistic via Monte Carlo analysis.
An analytic model is a mathematical construct, typically implemented as a computer program, that describes the behavior of some system of interest. In the case of a decision model, the system of interest is the problem of choosing projects that will create maximum value. An appropriate decision model for PPM provides predictions about how effective the various project alternatives will be in creating value. If the model is a good one, an alternative that the model says will create value is likely to do so in the real world.
The key in constructing any model is to abstract from the real-world situation the basic elements and relationships needed to describe the behavior of the whole without losing or obscuring important effects. Advances in the art and science of modeling — mathematical, symbolic, graphic, etc. — provide the means for exploring the structure, dynamics, and interactions that make up the decision problem that we wish to understand. The model-builder represents these interactions in a model. The model captures and makes explicit essential beliefs about "how things work."
Why Models Work
Models are useful because they address fundamental limitations of human problem solving. Research  shows that humans have limited information processing skills, can be biased, and are often inconsistent when making choices. People are good at creative tasks like generating alternatives. They are also good at recognizing structure and at making the sort of "small", well-defined judgments that are required in order to provide inputs to models.
Models work because they break a complex problem into pieces, allowing people to do what they do best while enabling computers to do the calculations.
If you have understanding, you can create an analytic model. You can't manage what you can't understand. Therefore, you can create a model for any situation you can hope to be able to manage better. Creating the model is the feasible and necessary step that allows for much more effective project portfolio management.
Project Selection Decisions May Require Sophisticated Models
Because a successful decision model must capture every critical aspect of the decision, more complex decisions typically require more sophisticated models. “There is a simple solution to every complex problem; unfortunately, it is wrong” . This reality creates a challenge for tool designers. Project decisions are often high-stakes, dynamic decisions with complex technical issues—precisely the kinds of decisions that are most difficult to model:
Though project selection decision models are necessarily sophisticated, they need not (and shouldn't be) be complex. What matters is that the critical considerations for choice be captured and properly incorporated into the decision making logic. Though a great many factors may be relevant, it is nearly always the case that only a few factors produce the biggest impacts on choice. So long as the decision model correctly identifies the "decision drivers" and incorporates the proper mathematics, then a relative small, compact model can provide project recommendations and other outputs with surprising accuracy.
A Two-Stage Decision Model
Benefits of Decision Models
A well-designed model can provide three general types of benefits:
First, a model can produce better choices. One way it does this is by reducing errors, biases, and inconsistencies in human judgments. If the model measures the value derived from projects, it can help the organization to make value-maximizing choices. A model can also create better choices by providing insights that suggest new alternatives and by controlling the role of politics.
Second, a model can improve the decision process. A model levels the playing field for the competition for resources by providing consistent rules for evaluating proposals. A model clarifies what information is needed and shows how it can be incorporated into decisions. A model can be used to involve stakeholders, for example, by allowing them to contribute to the design of the model or provide selected model inputs. A model tends to promote consensus, since it is often easier to get people to agree on the rules for making a decision than on the decision itself. A decision model can also serve as a catalyst for action by providing a way to redirect and end unproductive and unfocused debate.
Finally, a model can increase the defensibility of decisions. It documents the assumptions underlying decisions and provides a transparent logic for choices. A model also allows "what if" analysis wherein inputs are varied across a range of values to see if recommendations change. This is often useful for demonstrating that a decision (e.g., "Should we conduct the project?") is not sensitive to a specific unresolved issue or disagreement.
Decision models come in various forms. For project prioritization, a very useful form, illustrated below, is a decision model composed of two distinct but integrated components (sub models).
The first component is a simulation model that provides predictions of the consequences of conducting projects, based on, among other things, the characteristics of those projects, the needs they address, and the effectiveness of the projects in addressing those needs. For example, a simulation model might estimate the improvements to products or customer service that might result from a project, as well as the anticipated increases in revenue or reductions in cost, and other changes relevant to objectives that are important to the organization. If project and portfolio risk are important, it may be desirable to use a probabilistic model that provides a description of the uncertainty over project consequences.
The second component of the decision model is a value model. The value model translates the estimated consequences of conducting projects, produced by the simulation model, into measures of the value of those consequences to the organization. The value model needs to account for the relative importance of the various financial and non-financial objectives impacted by projects, the organization's willingness to accept risk, and the organization's time preference (the natural desire to postpone undesired outcomes and speed up desired outcomes).
Advantages of Separating the Two Stages
Structuring the decision model as a simulation model linked to a value model provides two important advantages. First, it enables individuals to contribute to the development of the decision model appropriately according to their knowledge, roles, and responsibilities. For example, technical experts can ensure that the simulation model captures best-understanding about how projects impact business outcomes—the organization's financial experts can verify that logic for generating financial estimates is correct, marketing experts can ensure that logic for estimating impacts on sales is reasonable, safety experts can ensure that the logic for estimating impacts on public or worker safety make sense, etc. Conversely, the organization's senior executives, those responsible for setting policy and direction, can provide the judgments needed to specify the value model, including the weights assigned to specify the relative worth of the various types of consequences to the organization.
Second, the two-part structure means that the evaluation of projects is explicitly based on forecasts of what the consequences of doing those projects will be. This ensures that predictions are grounded in reality. Over time, forecasts originally produced or provided for projects that are funded can be compared with the actual outcomes that occur. Based on results, the organization has the opportunity to learn, address errors, and improve the decision model over time.
Evaluating Tools Requires Evaluating Decision Models
I hope I've convinced you that the quality of a PPM tool depends critically on the quality of the tool's decision model. A tool with an inadequate decision model can mislead decision makers, potentially producing poorer and less defensible decisions than would be made without it. Part 6 provides criteria for evaluating tools and their underlying decision models.