How to Evaluate and Compare Project Portfolio Management (PPM) Software Tools

Purchasing a project portfolio management (PPM) software tool is not the first step to implementing PPM. Among other things, you must secure executive and stakeholder support, define the changes you need to make to organizational structure and processes, and design the framework that you will use to value and prioritize your projects (see Seven Keys to Implementing Project Portfolio Management). However, when you are ready, selecting a tool that fits your needs is key to reaping the rewards of PPM.

This resource is intended to help you evaluate PPM tools. It provides a structured evaluation process, criteria, and a simple aid (Excel® spreadsheet program) that you can use to compare candidate tools. The aid can help you document your considerations and reasoning, useful, for example, if you are conducting a formal, tool-selection process. Importantly, the spreadsheet includes criteria for evaluating tool cability to prioritize projects and optimize project portfolios, considerations that are often poorly addressed by PPM tool reviews and evaluation aids.

What Project Portfolio Management Is

PPM is a tool-supported process for making project decisions so as to generate the greatest possible value (return) from the organization's portfolio of projects. Portfolio optimization methods have proven very successful in the world of financial investing. The idea behind PPM is to apply similar methods to project investments.

Project Portfolio Management Tools

More than 60 software providers offer PPM tools (see Project Portfolio Management Tools - Which Approach is Best). The available tools are targeted at different industries and types of projects, and they have very different strengths and weaknesses. Choosing the right tool will enable your organization to derive greater value from projects while reducing costs and risks. Choosing the wrong tool wastes time, money, and effort. Also, a low-quality tool can produce inaccurate recommendations that may mislead decision makers into making poorer choices than would be made without the tool.

What PPM Tools Do

What all PPM tools do is allow you to:

Create, manage, and view data from a database of proposed, planned, and ongoing projects.

What PPM tools do to varying degrees is help you to:

Prioritize projects and optimize the project portfolio.
Plan projects and manage the execution of approved projects.
Manage the supply and demand for project resources.

The Evaluation Aid

The aid consists of spreadsheets containing 128 tool evaluation criteria organized according to the above categories (32 criteria per category).¹ You can assign importance weights to the criteria and score candidate tools against the criteria. The program computes the total points assigned to each tool by multiplying weights and scores, and adding.

Using the Aid

There are 3 steps:

1. Decide What's Important and Assign Weights.

No tool excels on every criterion. However, not all criteria (or even all criteria categories) are likely to be relevant given your needs. As you go through the criteria, determine how important each is. The following scale is provided for assigning weights:

Importance of Criterion	Weight
Not needed, not important, or already provided by your existing tools	0
Nice, but of minor importance; not having would result in minimal loss	1
Useful; not having would result in some loss and/or extra work	2
Important; not having would result in signicant loss and/or extra work	3
Extremely important; not having would result in major loss and/or much extra work	4
Essential; not having disqualifies the tool	E

2. Screen Tools and Obtain Information

Screen tools to select candidates for evaluation. You can screen out tools aimed at industries different from yours or types of projects different from those within your portfolios. Also, you may be able to screen tools depending on delivery method, for example, desktop software, web-based software, or on-demand services (where the data is stored on a supplier's server and the tool is available for an on-going fee). Identify the candidates by naming them above the columns in the spreadsheet. Contact providers to obtain the necessary information for scoring the candidate tools.

3. Score Candidate Tools

The following scale may be used to score candidate tools:

Estimated Performance Against Criterion	Score
Criterion not achieved or supported	0
Criterion partly achieved or supported	1
Criterion mostly achieved or supported	2
Criterion fully achieved or supported	3

As illustrated in Figure 1, any tool that fails to at least partially achieve a criterion rated "essential" is rejected.²

Tools that fail to achieve essential criteria are rejected

Figure 1: Example—A tool that fails to at least partially achieve an essential criterion is rejected.

Warning: Results Will Be Imprecise

Be aware that the aggregate scores and rankings produced by simple rate-and-weight schemes such as this are notoriously imprecise (especially if there are a great many criteria with some overlap or double counting in the way criteria are defined). Therefore, don't make your choice based solely on total points or rankings. Instead look closely at the short list of all highly ranked candidates. Then, make your decision using the same principles that govern choosing projects for PPM: Select the tool that you believe will generate the greatest value, considering total life-cycle costs, resource constraints, and risks.³

Insist on Hands-On Testing

Although guidance such as this can be helpful, there is no substitute for first hand experience. Be sure to "test-drive" final candidate tools before committing to a purchase. Insist that tool providers allow you to thoroughly test their products with the tool properly configured or customized to your application and using your real project data.

Notes

You can add criteria to the spreadsheets if there are things important to you that have been left out.
If an otherwise attractive tool doesn't achieve an essential or highly-weighted criterion, before eliminating that tool, you might want to inquire if the provider may be willing to incorporate the desired capability, and, if so, how long it would take and what the added costs would be.
The decision to acquire a PPM tool is itself a project portfolio management decision. I've not heard of a PPM tool provider offering the use of its tool to evaluate PPM tools, but the concept is intriguing!

Appendix: The Criteria