Home

More Papers

Project Portfolio Management Tools: Which Approach is Best? Part 4: The Weak Link for Most Tools

Printable version

Part 1 identified available tools for project portfolio management, Part 2 described key differences, and Part 3 summarized costs and risks. This part identifies the weak link in the design of most tools.

A significant difference among available tools is the logic used to make project recommendations (see the side box below). Quantitative methods exist for optimizing project portfolios, but few tools use the best of the available techniques. For most PPM tools the logic used to recommend projects is the weak link.

As described previously, nearly all PPM Tools allow users to define and enter their own measures for ranking projects. In addition, PPM tools typically allow users to rank projects based on any criterion defined by weighting and adding measures that have been assigned to projects. However, the ability to define and create weighted combinations of measures is not as helpful as it might at first seem.

Weighting Project Measures Doesn't Enable Prioritization

Projects create different kinds of benefits. Prioritizing projects requires measuring the various types of benefits that would be provided if the project is conducted. For example, you might want to use net present value (NPV) as a measure of a project's financial benefit. You could certainly compute each project's NPV externally and enter it into the tool. However, this approach makes it hard to ensure consistency in the way that NPV's are calculated, and impossible to conduct sensitivity analysis to see the impact on rankings of changing assumptions (e.g., What if we assumed a lower risk-free discount rate?).

What you would like is to have a tool compute measures of project benefit. However, you cannot use the tool's weight and add capability to do this because benefit is rarely a weighted, additive combination of what you know or could readily estimate about a project (e.g., the formula for computing NPV from cash flows and discount rate is not weight and add).

Recognizing the importance of measuring financial benefits, nearly all PPM tools include at least a simple financial model for computing project NPV and other common financial measures. However, what about models for quantifying other project benefits (benefits other than financial) that cannot be constructed using weight and add? More useful would be a tool that contains all of the models needed to compute the types of benefits that accrue from the types of projects that your organization conducts. However, the types of benefits differ from industry to industry, as does the project information available for use as inputs for the models. This is why it is nearly impossible for general purpose tools to provide the analytics to properly value and prioritize projects.

Valuing Projects

The goal for selecting projects is to obtain the project portfolio that, subject to applicable resource constraints, creates the greatest possible (risk-adjusted) value for the organization. This, of course, requires the capability to compute project value. Although many tools are advertised as having the capability to compute measures of project value, few actually define value in a way that makes sense for PPM. In effect, most tools confuse the issue of value maximization with less appropriate (but easier-to-implement) concepts such balance, strategic alignment, maximizing points, etc.

Webster's Dictionary lists several definitions for the word "value." Near the top are, "the monetary worth of something" and "marketable price." These are reasonable definitions to use as a basis for prioritizing projects. However, quantifying the equivalent monetary worth of a project that includes non-financial as well as financial benefits requires sophisticated analysis (using techniques such as multi-attribute utility analysis and real options analysis, see Part 4). Many tools are incapable of applying these types of analyses. Webster's alternative definitions for "value" include "a numerical quantity assigned or computed" and a "degree of excellence." Although easier to implement, these definitions do not reflect the fundamental objectives of organizations (to create value) and do not provide a sound basis for finding optimum project portfolios.

Project Value Must be Expressed in Dollars

To be most useful for PPM, value must be measured in dollars. Unless all project benefits are expressed in common dollar units, you can't correctly combine financial and non-financial project benefits. Also, you can't determine whether benefits justify costs. Furthermore, without a common unit for measuring benefit, you can't determine how best to allocate a budget across organizational units responsible for different project portfolios.

Unlike the imprecise definitions of project value, defining value as monetary worth ensures that there is an objective basis for validating the valuations computed or assigned using the tool. If someone claims that something is worth X dollars, then he or she should be willing to buy it if it costs less than X dollars, but not if it costs more. Likewise, if the market price of something is claimed to be X dollars, then, if that item is offered for sale in the free market, its selling price should, in fact, turn out to be X dollars.

Optimizing the Project Portfolio

Most tools that recommend project portfolios don't use optimization methods—they rank projects (based on, as indicated above, individual criteria or on weighted sums of those criteria, possibly dividing by project cost). At best, ranking is an approximate technique that can only work if neither the costs nor benefits of projects depend on the other projects that are conducted (see mathematical theory ). Most tools do not consider dependencies when ranking projects. Note that tools that claim to handle dependencies usually only allow the user to record dependencies among projects (and account for bundling requirements, e.g., "You can't do Project A unless you also do Project B."

Also, be aware that some tools can't easily handle different versions of the same project (e.g., a lower cost, reduced scope approach to accomplishing the same needs). Instead, these tools assume the only decision allowed for each project is go versus no-go. A no-go may be unacceptable in some applications, for example, prioritizing maintenance projects where the option of eliminating maintenance altogether may be not exist.

Another common limitation relates to the types of constraints that can be specified. Most tools only allow for one type of constraint; namely, a constraint on total funding. It is rarer to find tools that allow for different types of dollar constraints (e.g., choose the optimal set of projects with out-year costs less than or equal to Y) or multiple constraints on different types of resources (e.g., choose a portfolio that can be conducted given our current workers and skill sets).

False Claims

Unfortunately, as competition has heightened, some tool providers have tended to put more effort into making better-sounding marketing claims than into improving the way that their tools recommend projects. If you visit the websites of the major providers, you'll often see the following self-conflicting claims:

1. That their tools are based on "unique" approaches that address risk, uncertainty, project interdependencies, timing, and virtually every other consideration that can be imagined.
2. That their tools provide "flexibility" for defining and using whatever prioritizing criteria that the client organization wants.
3. That their tools correctly identify value-maximizing project portfolios.

In truth, most portfolio management tools fail to employ even the most basic portfolio optimization techniques. Portfolio optimization is mathematically complex, especially when projects produce non-financial as well as financial benefits, when risks are important, and when project costs or benefits depend on when the project is initiated or on what other projects are conducted. Not only do most tools fail to properly address such issues, they are often structured in such a way that it is impossible to tailor them to roughly approximate a mathematically correct solution.

Thus, although many tools are strong on project execution, most are very weak on project selection. While it is true that a well-designed PPM tool can provide a consistent, logical way to evaluate and compare project proposals, a poorly designed tool or one that doesn't fit the need can distort decisions, increase costs, and create considerable frustration. I believe this is the reason that many organizations are disappointed with their first experiences using PPM tools.

So, how can an organization interested in project prioritization really be sure that the tool acquired makes sound recommendations? Understanding decision models and how they work is the first step. Part 5 describes in more detail how tools use models to evaluate projects and recommend project selection decisions.

Notes

1. Wikipedia, The Free Encyclopedia, "Project Portfolio Management," March 2008.

Part 3: Tool Cost & Risk

Part 5: Decision Models