Home

More Papers

Project Portfolio Management Tools: Which Approach is Best? Part 2: The Weak Link for Most Tools

Printable version

A significant difference among available tools is the logic used to make project recommendations (see the side box below). Quantitative methods exist for optimizing project portfolios, but few tools use the best of the available techniques. For most PPM tools the logic used to recommend projects is the weak link.

The conclusion that the logic used by PPM tools to recommend projects is the weak link may seem puzzling, given that, as described in the previous section, most PPM tools allow users to define and enter numbers for any measures they want for evaluating projects. However, the capability to score projects based on user-defined measures is not as helpful as it might at first seem. What you would really like to have is the ability to use the tool to compute the relevant values for appropriate project evaluation measures, not just allow you to manually enter them based on external sources. For example, the net present value (NPV) of the incremental, after tax, net cash flow is often used as a measure for evaluating a project from a financial standpoint. Many tools will allow you to compute NPV externally and then assign the value to a project as an input to the tool. However, this approach makes it harder to enforce consistency across projects in the way that such measures are calculated and impossible to conduct sensitivity analysis to see the impact on recommendations of changing the assumptions used when computing the evaluation measures (e.g., Would it change project rankings if we used a lower discount rate?).

Likewise, most tools will allow users to rank projects based on weighting and adding the measures that they have input for projects. However, weight-and-add operations are typically insufficient for computing aggregate or "overall" measures for ranking projects. NPV, for example, involves mathematical operations other than weight and add. Recognizing the importance of financial measures, many tools include financial models for computing project impacts on discounted cash flows. However, there are many other natural computations that can't be accomplished using weight and add. For example, the probability that a project will be successful may be an important consideration for some types of projects. It is almost never appropriate, when computing an overall measure of project attractiveness, to weight and add a probability measure to other types of project metrics. Rather than provide just the ability to define, enter, weight, and add user-defined project measures, what you need is the ability to create models (which involve operations beyond weight and add) that provide as output the overall measure you need for evaluating and comparing projects.

Valuing Projects

The overall measure that you need for evaluating projects depends on the goal. The goal for PPM is to select the project portfolio that, subject to applicable resource constraints, creates the greatest possible value. Thus you need a tool with the capability to compute project value. Although many tools are advertised as having the capability to compute measures of project value, few actually define value in a way that makes sense for project portfolio management. In effect, most tools confuse the issue of value maximization with less appropriate (but easier-to-implement) concepts such balance, strategic alignment, goal achievement, maximizing points, etc.

Webster's Dictionary lists several definitions for the word "value." Near the top are "the monetary worth of something" and "marketable price." These are consistent with the definitions used in financial portfolio management and are reasonable definitions to use for PPM. Webster's alternative definitions for the word value include "a numerical quantity assigned or computed" and a "degree of excellence." Although easier to implement, these definitions of value are inconsistent with the principles of financial portfolio management and do not reflect the fundamental objectives of organizations. Such concepts of project value cannot be used as a sound basis for finding optimum project portfolios.

Project Value Must be Expressed in Dollars

To be most useful for project portfolio management, project value must be measured in dollars. Unless all project benefits are expressed in common dollar units, you can't determine whether project benefits justify costs. Expressing project value in dollars allows you to make apples-to-apples comparisons, and you can combine financial and non-financial project benefits. Furthermore, if you can determine the relationship between portfolio value and total portfolio cost, you can determine how best to allocate resources across organizational units responsible for different project portfolios.

Unlike the imprecise definitions of project value (i.e., that value is "points" or "degree of excellence"), defining value as monetary worth ensures that there is an objective, comparative basis for the numbers that might be computed or assigned using the tool. If a tool computes that a project is worth X dollars, then the organization should find the project potentially attractive if the cost is less than X dollars (provided that there are no better project opportunities for using the constrained resources), but not if it is greater than X dollars. Likewise, if a tool computes the market price of project deliverables to be X dollars, then, if those deliverables were offered for sale in the free market, then the selling price should, in fact, turn out to be X dollars. Although you probably won't be able to assign project values directly using these techniques, expressing project value in dollar units ensures that there is a "sanity check" for validating the project values computed by tools that measure value in this way.

Optimizing the Project Portfolio

Most tools that recommend project portfolios use a simple optimization method—they rank projects (based on, as indicated above, individual criteria or on weighted sums of those criteria, possibly dividing by project cost). At best, ranking is an approximate technique that can only work if neither the costs nor benefits of projects depend on the other projects that are conducted (see the paper on mathematical theory ). Thus, to accurately optimize the project portfolio, it is necessary to recognize and account for any project interdependencies. Note that tools that claim to handle dependencies typically only allow the user to record dependencies among projects. Only rarely will the tools properly account for the dependencies when recommending project portfolios.

Also, be aware that some tools can't easily handle different versions of the same project (e.g., a lower cost, reduced scope approach to accomplishing the same needs). Instead, these tools assume the only decision allowed for each project is go versus no-go. A no-go may be unacceptable in some applications, for example, prioritizing maintenance projects where the option of eliminating maintenance altogether may be not exist.

Another common limitation relates to the types of constraints that can be specified. Most tools only allow for one type of constraint, a constraint on total funding. It is rarer to find tools that allow for different types of dollar constraints (e.g., choose the optimal set of projects with out-year costs less than or equal to X) or multiple constraints on different types of resources (e.g., workers with different skill sets).

False Claims

Unfortunately, as competition has heightened, some tool providers have tended to put more effort into making better-sounding marketing claims than into improving the way that their tools recommend projects. If you visit the websites of the major providers, you'll often see the following self-conflicting claims:

1. That their tools are based on "unique" approaches that address risk, uncertainty, project interdependencies, timing, and virtually every other consideration that can be imagined.
2. That their tools provide "flexibility" for defining and using whatever prioritizing criteria that the client organization wants.
3. That their tools correctly identify value-maximizing project portfolios.

In truth, most portfolio management tools fail to employ even the most basic portfolio optimization techniques. Portfolio optimization is mathematically complicated, especially when projects produce non-financial as well as financial benefits, when risks are important, or when project costs or benefits depend on when the project is initiated or on what other projects are conducted. Not only do most tools fail to properly address such issues, they are often structured in such a way that it is impossible to tailor them to roughly approximate a mathematically correct solution.

Thus, although many tools are strong on project execution, most are very weak on project selection. While it is true that a well-designed PPM tool can provide a consistent, logical way to evaluate and compare project proposals, a poorly designed tool or one that doesn't fit the need can distort decisions, increase costs, and create considerable frustration. I believe this is the reason that many organizations are disappointed with their first experiences using PPM tools.

So, how can an organization interested in project prioritization really be sure that the tool acquired makes sound project recommendations? Understanding decision models and how they work is the first step. Part 3 of this paper describes in more detail how tools use models to evaluate projects and recommend project selection decisions.

Part 1: Options

Part 3: Decision Models