Term
|
Explanation
|
|
analytic hierarchy process
|
A theory-based mathematical decision-making technique developed by Thomas Saaty in the early 1970s. Some project portfolio management tools
use AHP to prioritize projects. Because there are many variations to how it is applied, it is impossible to provide general
conclusions about AHP's effectiveness for prioritizing projects.
As originally defined by Saaty, AHP involves asking decision makers to express their preferences over pairs of alternatives based on various objectives. For example, "With
regard to improving financial performance, do you prefer project A or project B, and by how much?" Oftentimes, a nine-point scale is recommended for specifying strength of preference
(the points on the scale are defined qualitatively, for example, equal preference, moderate preference, strong preference, etc.). Based on the expressed preferences, a mathematical
procedure is used to derive a model for ranking alternatives. Saaty provided an axiomatic theory to support this approach to applying AHP. The theory has been
criticized based on the observation that AHP can produce a result wherein the addition of a new alternative, say a new project, can change the ranking of existing projects, even though
the new project does not influence the costs or benefits of the existing projects. However, this undesirable result rarely occurs in practice.
Applications to project portfolio management generally use a variation of AHP in which the preference comparisons are expressed for objectives, not for the projects. For example,
"Compare the relative importance of the objectives 'improve time to market' and 'improve financial performance.'" The answers are used to derive a set of weights interpreted to represent
the relative importance of the objectives. Projects are then scored to indicate their contributions to each objective (e.g., no contribution = 0, slight contribution = 0.1, ..., excellent contribution = 1). The scores
are weighted and added to obtain an overall metric for ranking projects.
Like decision analysis, this version of AHP produces a model for evaluating projects. Since the model is derived by encoding the fundamental preferences of the organization's
decision makers, the underlying philosophy is that projects should be ranked based on how much they are preferred, not according to goals such as
"balance" or "strategic alignment." The decision model is simpler than that produced by decision analysis in that it does not
require the intermediate step of estimating the consequences of projects. Also, AHP does not bother with probabilities for uncertainties. Like traditional net present value
analysis, all evaluations are conducted assuming a most likely scenario.
Unfortunately, this standard approach to applying AHP does not accurately prioritize projects (it also violates the axioms of Saaty's theory). It is not possible to obtain meaningful
preference comparisons between objectives unless the amounts of improvement are specified (e.g., the amount by which I prefer the objective "improve financial performance" depends on how
much financial performance is improved). Also, it is not correct to weight and add performance scores unless the value of achieving a given level of performance on one measure does not
depend on the level of performance achieved on any other measure.
An alternative approach to applying AHP overcomes the above problems by using the step-by-step process prescribed by multi-attribute utility analysis
to ensure that weights refer to specified improvements against objectives and that performance measures satisfy the requirements necessary to permit them to be weighted
and added. In effect, the resulting approach to developing the model for valuing projects is multi-attribute utility analysis, however, AHP's pairwise comparison technique is
used to determine the weights.
|
|
balanced scorecard
|
A process developed in the early 1990's by Robert Kaplan and David Norton for translating an organization's mission and strategy statements into a comprehensive system for measuring
organizational performance. Balanced scorecards collect diverse information intended to "balance" the traditional, but narrow, financial view of performance. The balanced scorecard is an
excellent tool for helping managers to understand how the organization is performing and helps translate strategy into action. However, balanced scorecards are not very useful for
prioritizing and choosing projects, and they are often misused in this regard.
According to the balanced-scorecard approach, performance measures should be defined in four areas: (1) finance, (2) customer satisfaction, (3) internal processes, and (4) innovation and learning
for employees. The selected measures are specific to the organization and are chosen to reflect the drivers believed to most important to understanding success.
As examples, measures of organizational financial performance might include return on investment (ROI), rate of revenue growth, amount of debt, etc. Customer satisfaction measures might
include number of customer complaints, results of customer surveys, average time to process phone calls, etc. Internal process measures might include fraction of projects delivered on schedule,
number of units requiring rework, process yield rates, etc. Learning measures might include number of employee hours spent in training, numbers of employee suggestions submitted, etc.
Measures can be backward-looking, to monitor how the organization is doing, or forward-looking, to assess the future impacts of alternative courses of actions. Assessments against the measures are
arrayed on pages or displays referred to as "scorecards." Target levels of performance may be assigned to the measures. Balanced
scorecards are being used in a broad range of activities, from product
planning to incentive compensation, and by federal, state, and local governments.
The major weakness of balanced scorecards is that the approach does not provide a basis for trading off performance on different measures. In other words, if an organization
improves performance on one measure without degrading performance on any other measure, that is a good thing. However, if making a change intended to boost performance in a given
area (e.g., customer satisfaction) threatens performance in some other area (e.g., finance), a traditional balanced scorecard cannot indicate whether that change should be made.
In an attempt to address the above weakness, balanced scorecards have sometimes been expanded to include a means for aggregating individual performance measures into a quantity
meant to represent the overall performance of the organization. For example, most commercially available tools for project portfolio management allow users to define equations for combining performance
measures—A scorecard is defined for assessing projects, and the various measures on the scorecard are mathematically aggregated to provide an indicator intended to represent
the relative desirability of the project. Typically, the form of the equation is weight-and-add (sometimes, the performance measures are merely averaged, which, presumably, implies
a desire to weight the measures equally).
Choosing projects so as to maximize a weighted sum of scorecard measures is nearly always incorrect. Yet, some try to justify this approach by arguing that organizations should
strive for balance in performance across various areas and measures. However, maximizing a weighted average does not necessarily lead to balance (if balance means including projects
that address all measures). In any case, the goal of project selection is to choose projects that create the most value, not balance (whatever balance means).
A project that has a high weighted-average performance score may or may not be a high-value project. How well the weighted-average score relates to value
depends on, among other things, how the measures are defined, the number of measures within each area and the degree to which they overlap or "double
count," the organization's current level of performance, and the basic objectives of the organization. It is possible to define performance measures that can be aggregated into an
overall measure of project value, but this requires a different process for defining performance measures than that used in the balanced scorecard approach
(see multi-attribute utility analysis (MUA).
|
|
decision analysis (DA)
|
A body of knowledge and related analytical techniques for implementing decision theory. DA is relevant to project
portfolio management because it provides a framework for analyzing project selection decisions. Included within the framework are tools for quantifying project value and for
addressing project risk.
DA was developed in the 1960's and 1970's at Harvard, Stanford, MIT, Michigan, and other major universities. The term "decision analysis" was coined in 1964 by Ron Howard, a professor
at Stanford University. DA is generally considered a branch of the field of operations research, but also has links to management science, economics, systems analysis
and psychology. DA is an area of consulting specialty, and there are journals and a professional society devoted to the topic.
According to DA, a good decision is one that (1) considers the full range of alternatives that are available to the decision maker, (2) accounts for what the decision maker believes will be the consequences
of choosing each alternative, and (3) is consistent with the decision-makers preferences for the various possible decision consequences. In other words, making good decisions requires knowing
what you can do, what you believe, and what you want.
DA employs various procedures and tools for understanding how the actions taken in a decision determine the consequences that may result, as well as the significance
of those consequences relative to the decision-maker's objectives. Analytic models are constructed that represent these two components (a consequence model that simulates decision outcomes and
a value model for measuring the decision-makers preferences for those consequences). Statistical and probabilistic reasoning is used to quantify risk and determine whether additional information should be collected before committing to a course of
action. The models allow sensitivity analysis, a process that identifies the issues that make the most difference and helps decision makers avoid "paralysis by analysis."
DA is generally focused on two types of decisions: (1) one-time decisions where alternatives must achieve multiple and possibly competing objectives and (2)
sequential decisions where uncertainties and learning play an important role. In both cases, a major task for the decision analyst is constructing the value model that allows the
overall desirability of alternatives to be computed based on how they perform on a set of evaluation measures, or "attributes." Multi-attribute utility analysis
(MUA) is often used to construct the value model. To represent decision timing and uncertainty, sequential decisions are analyzed using decision trees and models known as
influence diagrams. Decision analyses of project decisions often include calculations of project expected net present value (ENPV) and may include
valuations based on real options analysis.
|
|
decision theory
|
A theory of how individuals should make decisions, related to the concept of "rationality" used in economics. Also called subjective expected utility theory, or simply utility theory, the theory is derived from a set of easily-accepted axioms (hypotheses)
defining how rational people behave. For example, one such axiom (transitivity) states that if a person prefers outcome A to outcome B and outcome B to outcome C, that person should
prefer outcome A to outcome C. Another axiom (substitution) states that if a person is participating in a lottery where the prize is A, and if that person is completely indifferent
between receiving prize A and some alternative prize C, then that person should not care if the lottery is modified by substituting prize C for the equally desirable prize A.
Decision theory shows that if these and a few other axioms are accepted, then it can be proven that there is a mathematical function called a "utility function," denoted U, that
aggregates all of the different considerations that must be taken into account when deciding among alternatives. Furthermore, the best alternative (the one that is most preferred)
will be the one that maximizes the value of U (or, if there are uncertainties, the expected value of U).
Thus, the major focus of decision theory is estimating the unknown function U. Multi-attribute utility analysis is a set of techniques for
estimating U for the common situation where there are multiple characteristics or "attributes" relevant to determining the desirability of alternatives.
|
|
decision unit
|
The choice that is to be made in the context of a decision problem, including the alternatives under consideration. Decision units should be defined as
part of the framing process.
Decision units are important because they determine the granularity for analysis, including spatial, temporal, and intensity assumptions. When shopping for
project portfolio management tools, it is very important to understand the restrictions that are placed on the definition of decision units. For example,
if you need a tool to help you prioritize capital projects, it might be reasonable to consider one that evaluates "fund" versus "don't fund" options for each project. However,
such a tool might be useless for evaluating maintenance projects if the appropriate decision unit is the choice among alternative 5-year spending plans for programs consisting of
groups of similar assets.
|
|
