|
|
|
|
Part 1 of this paper characterized the tools available for prioritizing projects and managing project portfolios, and Part 2 identified inability to optimize the project portfolio as the week link for most tools. This part describes the components of PPM tools, including the internal model a tool needs to evaluate and recommend projects. Before purchasing or developing a PPM tool, it is helpful to understand tool components. A tool for project portfolio management should include three basic components:
|
||
Most Tool Evaluation Guides Pay Scant Attention to the Models Used Prioritize ProjectsMost guides for selecting project portfolio management tools focus mainly on data management and reporting capabilities. For example, one guide advises potential buyers to consider whether the tool "allows users to easily update forms," "annotate assumptions," "drill down," "collaborate with other team members," "retain historic information for trend analysis," "change the axis, bubble size, and color of displays," etc. With regard to the decision model, the guide suggests only that tools should "allow the user to specify their own company-specific formulas for computing priority...[based on] ROI, EVA, NPV or RONA." The guide fails to indicate, however, that using any of these would be a serious mistake, since none of mentioned metrics provides a correct basis for computing project value. Thus, such metrics won't enable you to accomplish the fundamental PPM goal of maximizing the value of the project portfolio. (See the Glossary and paper on Mathematical Theory). Components 1 and 3 are within the domain of computer science, and software providers typically do a good job in these areas. Component 2 is the subject of decision analysis, a topic less familiar to most tool providers. Lack of a quality decision model is a critical weakness of many tools. Bubble diagrams, charts, and other graphic displays are great for displaying information, but they are not by themselves decision models. Buying a tool that provides great data management and reporting capability won't necessarily improve decisions. It may just promote information overload. To provide an analogy, if you are captaining a boat, you've got instruments that can tell you the barometric pressure, wind direction, and temperature, but what you really want is the weather forecast. You need a model to give you that. Likewise, you need a model to be able to estimate and value the consequences of your available project choices. A good tool is good in all three components. As the old saying asks, "If you needed surgery, would you rather be operated on by a surgeon with a butcher knife, or by a butcher with a scalpel?" Using a data intensive tool with impressive graphics is risky if the tool is not based on a quality decision model. ModelingTypes of Decision ModelsDecision models can be categorized in various ways. One taxonomy separates deterministic models from probabilistic models and static models from dynamic models. In a deterministic model, all relevant data are assumed to be known with certainty. Probabilistic models incorporate uncertainty via probabilities. Static models ignore time, while dynamic models represent the time sequence with which changes occur. Deterministic models can more easily handle situations where there are many decisions that must be made simultaneously and where there are many constraints on what options can be chosen. Examples of deterministic models include difference equations, typical cost-benefit analysis and linear programming. Examples of probabilistic models include decision trees and Markov models. Often times, a deterministic model can be made probabilistic via Monte Carlo analysis. An analytic model is a mathematical construct, typically implemented as a computer program, that describes the behavior of some system of interest. In the case of a decision model, the system of interest is the problem of choosing projects that will create maximum value. A good decision model will provide predictions about how effective the various project alternatives will be in creating value. If the model is a good one, an alternative that the model says will perform well is likely to perform well in the real world. The key in constructing a decision model is to abstract from the real-world situation the basic elements and relationships needed to describe the behavior of the whole without losing or obscuring important effects. Advances in the art and science of modeling — mathematical, symbolic, graphic, etc. — provide the means for exploring the structure, dynamics, and interactions that make up the decision problem that we wish to understand. The model-builder represents these interaction, as they are identified, in a model. The model captures and makes explicit essential beliefs about "how things work." Why Models WorkModels are useful because they address fundamental limitations of human problem solving. Research [1] shows that humans have limited information processing skills, can be biased, and are often inconsistent when making choices. People are good at creative tasks like generating alternatives. They are also good at recognizing structure and at making the sort of "small", well-defined judgments that are required in order to provide inputs to models. Models work because they break a complex problem into pieces, allowing people to do what they do best while enabling computers to do the calculations. If you have understanding, you can create a model. You can't manage what you can't understand. Therefore, you can create an analytic model for any situation you can hope to be able to manage better. Project Selection Decisions Require Sophisticated ModelsBecause a successful model must capture every critical aspect of the decision, more complex decisions typically require more sophisticated models. “There is a simple solution to every complex problem; unfortunately, it is wrong” [2]. This reality creates a major challenge for tool designers. Project decisions are often high-stakes, dynamic decisions with complex technical issues—precisely the kinds of decisions that are most difficult to model:
A Two-Stage Decision ModelBenefits of Decision ModelsA well-designed model can provide three general types of benefits: First, a model can produce better choices. One way it does this is by reducing errors, biases, and inconsistencies in human judgments. If the model measures the value derived from projects, it can help the organization to make value-maximizing choices. A model can also create better choices by providing insights that suggest new alternatives and by controlling the role of politics. Second, a model can improve the decision process. A model levels the playing field for the competition for resources by providing consistent rules for evaluating proposals. A model clarifies what information is needed and shows how it can be incorporated into decisions. A model can be used to involve stakeholders, for example, by allowing them to provide selected model inputs. A model tends to promote consensus, since it is often easier to get people to agree on the rules for making a decision than on the decision itself. A decision model can also serve as a catalyst for action by providing a way to redirect and end unproductive and unfocused debate. Finally, a model can increase the defensibility of decisions. It documents the assumptions underlying decisions and provides a transparent logic for choices. A model also allows "what if" analysis wherein inputs are varied across a range of values to see if recommendations change. This is often useful for demonstrating that a decision is not sensitive to a specific unresolved issue or disagreement. Decision models come in various forms. For project prioritization, a very useful form, illustrated below, is a decision model composed of two distinct but integrated components (sub models).  The first component is a simulation model that predicts the consequences of conducting projects, based on, among other things, the characteristics of those projects and the needs they address. For example, a simulation model might estimate the improvements to products or customer service that might result from a project, as well as the anticipated increases in revenue or reductions in cost, and other changes relevant to objectives that are important to the organization. If project and portfolio risk are important, it may be desirable to use a probabilistic model that provides a description of the uncertainty over project consequences. The second component of the decision model is a value model. The value model translates the estimated consequences of conducting projects, produced by the simulation model, into measures of the value of those consequences to the organization. The value model needs to account for the relative importance of the various financial and non-financial objectives impacted by projects, the organization's willingness to accept risk, and the organization's time preference (the natural desire to postpone undesired outcomes and speed up desired outcomes). Advantages of Separating the Two StagesStructuring the decision model as a simulation model linked to a value model provides two important advantages. First, it enables individuals to contribute to the development of the decision model appropriately according to their knowledge, roles, and responsibilities. For example, technical experts can ensure that the consequence model captures best-understanding about how projects impact business outcomes—the organization's financial experts can verify that logic for generating financial estimates is correct, marketing experts can ensure that logic for estimating impacts on sales is reasonable, safety experts can ensure that the logic for estimating impacts on public safety make sense, etc. Conversely, the organization's senior executives, those responsible for setting policy and direction, can provide the judgments needed to specify the value model. Second, the two-part structure means that the evaluation of projects is explicitly based on forecasts of what the consequences of doing those projects will be. This ensures that predictions are grounded in reality. Over time, forecasts originally produced or provided for projects that are then funded can be compared with the actual outcomes that occur. Thus, the organization has the opportunity to learn, address errors, and improve the decision model over time. Evaluating Tools Requires Evaluating Decision ModelsThe quality of the tool depends critically on the quality of the decision model. A tool with an inadequate decision model can mislead decision makers, potentially producing poorer and less defensible decisions than would be made without it. Part 4 of this paper provides criteria for evaluating tools and their underlying decision models. Notes
|
|||
Printable
version
Part 2: The Weak Link