Technical Policy Briefing Notes - 6
Multi-Criteria Analysis
Description of the Method
Policy Briefs
Multi-Criteria Analysis
Description of the Method
MCA is an approach that allows consideration
of both quantitative and qualitative data in the ranking of alternative
options.
The approach provides a systematic method for assessing and scoring options against a range of decision criteria, some of which are expressed in physical or monetary units, and some which are qualitative. The various criteria can then be weighted to provide an overall ranking of options. These steps are undertaken using stakeholder consultation and/or expert input.
The approach identifies alternative options, selects criteria and scores options against these, then assigns weights to each criterion to provide a weighted sum that is used to rank options (Janssen and Van Herwijnen, 2006). The process allows the weights (for each criterion) to reflect the preferences of the decision-makers and the weighted sum of the different criteria is used to rank the options.
MCA has been widely applied for ranking various alternatives, especially in the environmental domain. It is often included in guidance as one of a number of potential tools for option appraisal (e.g. as for [regulatory] impact assessment, EC, 2005). It can be used for strategy level analysis or for individual projects or investment decisions. It is also sometimes used as a complementary tool to support cost-benefit analysis, to consider the performance of options against criteria that may be difficult to value or involve qualitative aspects. Such applications include supporting decision analysis for transport appraisal (Dodgson et al, 2000).
A simplified example is included in the box below.
In the implementation of MCA it is important to reflect very carefully on how to score the alternatives and which range of scores should be applied. It is also essential to make sure that the weighted scores can be added, i.e. all criteria should be formulated in positive terms, or all criteria should be expressed in negative terms.
Usually scores are standardized, so that the high and low levels of the scores represent the judgement about the performance of the alternatives as precisely as possible. The weights then need to be made explicit based on the assessment of the decision makers, or for instance starting from equal weights and then according to a set of logical and plausible weights that express the values of various categories of stakeholders.
This ensures that the impacts of the various sets of weights on the ranking can be assessed transparently. The process allows decision makers to learn about the characteristics of the alternatives and the ranking of the alternatives for various sets of scores and weights.
There are many methods to establish the ranking of the alternatives. The most commonly applied is the method of weighted summation. However, alternative methods include pair wise comparison; Analytic Hierarchy Process (AHP) or more complicated mathematical methods. Detailed descriptions of these methods are available from the sources in the further reading list on Multi Criteria Analysis (e.g. Belton and Stewart, 2002).
Details of the Analytic Hierarchy Process approach are provided in a separate Policy Briefing Note (No 7).
The approach provides a systematic method for assessing and scoring options against a range of decision criteria, some of which are expressed in physical or monetary units, and some which are qualitative. The various criteria can then be weighted to provide an overall ranking of options. These steps are undertaken using stakeholder consultation and/or expert input.
The approach identifies alternative options, selects criteria and scores options against these, then assigns weights to each criterion to provide a weighted sum that is used to rank options (Janssen and Van Herwijnen, 2006). The process allows the weights (for each criterion) to reflect the preferences of the decision-makers and the weighted sum of the different criteria is used to rank the options.
MCA has been widely applied for ranking various alternatives, especially in the environmental domain. It is often included in guidance as one of a number of potential tools for option appraisal (e.g. as for [regulatory] impact assessment, EC, 2005). It can be used for strategy level analysis or for individual projects or investment decisions. It is also sometimes used as a complementary tool to support cost-benefit analysis, to consider the performance of options against criteria that may be difficult to value or involve qualitative aspects. Such applications include supporting decision analysis for transport appraisal (Dodgson et al, 2000).
A simplified example is included in the box below.
- The approach involves a number of common steps.
- To identify the objectives and important decision criteria. To identify potential options. Note that stakeholder consultation is often used to identify the most important options.
- Identify relevant criteria to assess the options against. The number of criteria can range from a few key criteria to twenty or more criteria, though a higher the number of criteria increases the resources and analysis needed. For each criteria, a scale is needed, either as a quantitative metric (e.g. costs), or for qualitative metrics, a range (e.g. 1 to 5, 1 to 10).
- All options are scored against all the criteria. This process can be undertaken through analysis, stakeholder engagement (and workshops) or through expect elicitation.
- Assess the weighting of alternative criteria. This provides the relative importance of each of the individual criteria in the overall decision. While all criteria can be given equal weighting, it is normal in MCA to give different weightings to different criteria, reflecting their important in the objectives. These weights can also be set through stakeholder engagement or expect elicitation.
- The weighted scores for each option are calculated. This then provides a prioritised ranking of options, though alternative approaches are possible (see later).
Box
1: Multi-Criteria Analysis: Example
A simple example of MCA is illustrated below. This aims to rank three alternative investment projects A, B, and C, based on the criteria (i) profitability, (ii) risk and (iii) whether it is a core activity. The first step is to provide scores for each of the criteria related to these alternatives, as the example below.
A simple example of MCA is illustrated below. This aims to rank three alternative investment projects A, B, and C, based on the criteria (i) profitability, (ii) risk and (iii) whether it is a core activity. The first step is to provide scores for each of the criteria related to these alternatives, as the example below.
Table 1. Scores per
criteria per alternative: a hypothetical example
Note: 5=very high 4=high 3=average
2= low 1=very low
We then formulate the weights that we attach to each of the criteria, for instance equal weights of all
criteria, i.e. thus 0.333 for each, though different weights are often assigned. This enables us to calculate
the weighted scores for each of the alternatives. The weighted scores are then as follows:
– for Alternative A: 0.33*5+0.333*2+0.333*3=3.33
– for Alternative B: 0.33*3+0.333*4+0.333*1=2.664
– for Alternative C: 0.33*2+0.333*3+0.333*4=2.997
Table 2. Table with the weights per criteria, the weighted scores per criteria per alternative, and the total
weighted score per alternative based on weighted summation: a hypothetical example
Note: 5=very high 4=high 3=average 2= low 1=very low
Table 2 shows that alternative A would be preferred, because it has the highest total score.
We then formulate the weights that we attach to each of the criteria, for instance equal weights of all
criteria, i.e. thus 0.333 for each, though different weights are often assigned. This enables us to calculate
the weighted scores for each of the alternatives. The weighted scores are then as follows:
– for Alternative A: 0.33*5+0.333*2+0.333*3=3.33
– for Alternative B: 0.33*3+0.333*4+0.333*1=2.664
– for Alternative C: 0.33*2+0.333*3+0.333*4=2.997
Table 2. Table with the weights per criteria, the weighted scores per criteria per alternative, and the total
weighted score per alternative based on weighted summation: a hypothetical example
Note: 5=very high 4=high 3=average 2= low 1=very low
Table 2 shows that alternative A would be preferred, because it has the highest total score.
In the implementation of MCA it is important to reflect very carefully on how to score the alternatives and which range of scores should be applied. It is also essential to make sure that the weighted scores can be added, i.e. all criteria should be formulated in positive terms, or all criteria should be expressed in negative terms.
Usually scores are standardized, so that the high and low levels of the scores represent the judgement about the performance of the alternatives as precisely as possible. The weights then need to be made explicit based on the assessment of the decision makers, or for instance starting from equal weights and then according to a set of logical and plausible weights that express the values of various categories of stakeholders.
This ensures that the impacts of the various sets of weights on the ranking can be assessed transparently. The process allows decision makers to learn about the characteristics of the alternatives and the ranking of the alternatives for various sets of scores and weights.
There are many methods to establish the ranking of the alternatives. The most commonly applied is the method of weighted summation. However, alternative methods include pair wise comparison; Analytic Hierarchy Process (AHP) or more complicated mathematical methods. Detailed descriptions of these methods are available from the sources in the further reading list on Multi Criteria Analysis (e.g. Belton and Stewart, 2002).
Details of the Analytic Hierarchy Process approach are provided in a separate Policy Briefing Note (No 7).
download this briefing note