Technical Policy Briefing Notes - 1
Summary of Methods and Case Study Examples from the MEDIATION Project
Decision Making Under Uncertainty
Policy Briefs
Summary Methods and Case Study Examples
Decision Making Under Uncertainty
While the three techniques above all have
potential applications for adaptation, they have relatively low
potential for considering uncertainty. However, there is a growing
recognition of the need to consider this issue in designing adaptation
(e.g. Adger et al. 2006; Dessai and van der Sluijs 2007; Downing,
2012).
At the same time, there has been a shift in the mainstream adaptation literature away from a scenario-based impact assessment framework, where adaptation is assessed on the basis of a predict-then-optimise framework, to a greater focus on adaptation assessment and decision making under uncertainty (Füssel and Klein, 2006; UNFCCC, 2009; IPCC SREX, 2012).
As a result there is a growing interest in decision support techniques that can address different elements of uncertainty. A brief description of some of the emerging approaches is presented below.
Real Option Analysis (ROA) is an economic decision support tool that quantifies the investment risk associated with uncertain future outcomes. The approach derives from the financial markets, where it has been used to assess the valuation of financial options and risk transfer. The same insights are also useful when there is risk or uncertainty involved with investment in physical assets, hence ‘real’ options.
The approach can be used to consider the value of flexibility. This includes the flexibility over the timing of a capital investment, but also the flexibility to adjust the investment as it progresses over time, i.e. allowing a project to adapt, expand or scale-back in response to unfolding events or from new information (learning). The approach can therefore assess whether it is better to invest now or to wait – or whether it is better to invest in options that offer greater flexibility in the future. ROA can also be used to support initial enabling steps to help secure projects for future development. The most common application of the approach uses decision trees, defining possible outcomes, and assigning probabilities to these, then using expected values to compare alternatives.
Real Option Analysis has been widely cited as a possible decision tool for adaptation, as the concepts of the approach align closely with iterative decision making and adaptive management. It is a powerful decision-support tool with a number of strengths, most notably that it provides information in quantitative and economic terms. However, it is a technical complex and resource intensive, and requires probabilistic (or probabilistic-like) information on outcomes. In practice, the approach is most relevant for particular types of decisions, primarily those that involve large up-front irreversible investments (e.g. coastal protection, large water storage projects), where there is flexibility in the timing, and the opportunity for new information to emerge. The framework is most likely to be supportive of projects that have some combination of substantial near-term benefits, and the ability to scale-up or down in line with learning regarding potential upside benefits or downside risks.
Robust Decision Making (RDM) is a decision support tool that is used in situations of deep uncertainty, i.e. in the absence of probabilistic information on scenarios and outcomes. The key aim of RDM is to seek strategies that are robust over many future outcomes, i.e. that are 'good enough' and minimize regret. It therefore offers an alternative to a conventional cost-benefit analysis and the identification of optimal options on the basis of economic efficiency.
The formal application of the approach (Lempert et al, 2003: Groves and Lempert, 2007) uses quantitative models, or scenario generators, with data mining algorithms, to evaluate how different strategies perform under large ensembles of scenarios reflecting different plausible future conditions (using hundreds to thousands to millions of input combinations). The aim is to help policymakers minimise the negative impacts of possible future outcomes. Iterative and interactive techniques are then applied to “stress test” different strategies to identify potentia
vulnerabilities or weaknesses of proposed approaches.
RDM has many attributes that align with the concept of adaptive management (Lempert and Groves, 2010) and the approach has been widely recommended for adaptation. The approach can assess robustness using various metrics, including physical effectiveness or economic efficiency. It can also be used in an iterative framework (Lempert, 2010) which aligns it more closely to the iterative adaptation management concepts of monitoring, research, evaluation and learning.
It is a particularly useful tool when future uncertainties are poorly characterised and probabilistic information is limited or not available – a key strength for long-term climate change related decisions – though the formal (computer modelling based) application of the approach has a high demand for quantitative information and computing power. As a result, the broad concepts of the approach have also been applied to the climate domain looking at climate uncertainty, to help in selecting options that are robust across a wide-range of plausible (climate) futures.
Portfolio Analysis (PA) is a decision support tool that helps in developing portfolios of options, rather than single options. It originated in the context of financial markets to explore the potential for portfolios of financial assets to maximise the financial return on investments, subject to a given level of risk. PA helps in the design of such portfolios. It aims to spread investments over a range of asset types to spread risks at the same time, thereby reducing the dependence on a single asset.
The approach highlights the trade-off between the returns on an investment and the riskiness of that investment, measuring risk by estimating the variance (standard deviation) of the portfolio return: thus a portfolio with a relatively high (low) variance is judged to have a higher (lower) risk (Aerts et. al. 2008). The information on returns and risks is used to identify a portfolio that most closely matches (risk) preferences.
The principles of diversification and the use of portfolios have high relevance for climate change adaptation (including iterative adaptive management, see IPCC SREX, 2012), and the technique can quantify the effectiveness of portfolios of options against climate change uncertainty. In the climate change context, the trade-off is then between the possibility of a high degree of effectiveness in reducing climate risks, and the risk that the adaptation options will fail to be effective over a certain range of climate change. PA allows the selection a set of options that, together, are effective over the range of possible projected future climates, rather than one option that is best suited to one possible future climate.
The main strength of the approach is that it provides a structured way to assess adaptation portfolios in a way that other decision tools do not allow. The approach can be undertaken using various metrics, including physical effectiveness, cost effectiveness, or economic efficiency and this provides flexibility for application to a wide range of applications.
However, the formal application is technically complex, and relies on the availability of data on effectiveness and co-variance, which requires probability, and it can be difficult to apply for some risks, notably where climate attribution is less straightforward, and/or the effectiveness of adaptation is not easily measured.
Adaptive Management (Iterative Risk Management) is a long established approach that uses a monitoring, research, evaluation and learning process to improve future management strategies.
The potential for an iterative approach for addressing climate change has been recognised for some time (Tompkins and Adger, 2004) and aligns with the IPCC AR4 Act-Learn-Then act again approach (Klein et al. 2007). The approach has been widely recommended for adaptation, including in the latest IPCC Special Report on Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation (SREX) (IPCC, 2012).
The approach is less formalised than many of the tools above, but the focus is on the management of uncertainty, allowing adaptation to work within a process of learning and iteration. Recent applications have also used the term ‘adaptation pathways’ to reflect a shift from predict-andoptimise based approaches to a dynamic pathway that incorporates uncertainty (Downing, 2012).
The most recent applications identify possible risk or impact thresholds (and accompanying indicators) and assess options (or portfolios of options) that can respond to these threshold levels. These are accompanied by monitoring plans that track key indicators, and through a cycle of evaluation and learning, allows the adjustment of plans over time. The appraisal of options within these pathways can be undertaken using some of the tools above, using qualitative or semi-quantitative decision support tools, such as MCA, or more formalised economic appraisal with CBA (though it would also be possible to use the alternative tools above in such a framework). The results of these iterative assessments are often presented as adaptation pathways or route maps. While most applications have been at the project level, notably for sea level rise, there are now examples emerging of more strategic or even national level plans (see Watkiss and Hunt, 2011: Watkiss et al, 2013).
A variation of the approach is to consider major biophysical, human, social or economic thresholds, and the MEDIATION project has developed such assessments using the term ‘Adaptation Turning Points’ looking at socialpolitical thresholds (i.e. a formal policy objective or societal preference).
The advantage of the approach is that rather than taking an irreversible decision now about the ‘best’ adaptation option – which may or may not be needed depending on the level of climate change that arises – it encourages decision makers to ask “what if” and develop a flexible approach, where decisions are made over time, and these plans adjusted as the evidence emerges (Reeder and Ranger, 2011). This allows the right decisions are taken at the right time (EA, 2011), such that additional options can be brought forward– or delayed to a later time period – depending on how climate change actually evolves. The disadvantage is that the identification of suitable risk thresholds can be difficult, especially when there are multiple risks or cross sectoral dimensions involved. Like many of the approaches above, it can be time and resource intensive, especially when addressing multiple scenarios in an economic framework.
Analytic Hierarchy Process (AHP) is a form of multi-criteria analysis (see earlier) that undertakes pairwise comparisons using expert judgements to derive priority scales (Saaty, 1980). The method allows the analysis of tangible and intangible elements together, allowing these to be traded off against each other in a decision-making process.
The method is applied by making comparisons using a scale of absolute judgements that represents how much one element dominates another for a given attribute. The derived priority scales are then synthesised and the various weighted scores are aggregated. The approach is very flexible and can be adapted to specific contexts. Criteria (or attributes) and sub-criteria can be decided in advance by experts or through a participatory process with stakeholders. There is no upper limit to the number of criteria or subcriteria, except for the time that is then required to do the comparison.
The approach can be used to choose options, to rank and/or prioritise them, and for resource allocation and conflict resolution. The tool has a particular relevance where important elements of the decision are difficult to quantify or compare, or where different expertise, goals, and worldviews are a barrier to consensus-building and communication.
AHP has been used in a wide variety of fields. The approach has high relevance for the application to adaptation, as it can evaluate options in situations of high complexity, considering different time horizons, uncertainty, multiple and interdependent variables, and subjectivity, all of which requires multidimensional trade-offs. It allows the comparison of diverse elements that are often difficult to measure in a structured and systematic way using a scale, though this inevitably involves a degree of subjectivity.
At the same time, there has been a shift in the mainstream adaptation literature away from a scenario-based impact assessment framework, where adaptation is assessed on the basis of a predict-then-optimise framework, to a greater focus on adaptation assessment and decision making under uncertainty (Füssel and Klein, 2006; UNFCCC, 2009; IPCC SREX, 2012).
Definitions
of Uncertainty
There are many different definitions of uncertainty.
The IPCC SREX (2012) defines uncertainty as:
An expression of the degree to which a value or relationship is unknown. Uncertainty can result from lack of information or from disagreement about what is known or even knowable. Uncertainty may originate from many sources, such as quantifiable errors in the data, ambiguously defined concepts or terminology, or uncertain projections of human behaviour.
Uncertainty can therefore be represented by quantitative measures, for example, a range of values calculated by various models, or by qualitative statements, for example, reflecting the judgment of a team of experts.
This broad definition can be compared with stricter definitions, notably those which seek to separate uncertainty, where it is impossible to attach probabilities to outcomes, from risk, where probability is defined.
There are many different definitions of uncertainty.
The IPCC SREX (2012) defines uncertainty as:
An expression of the degree to which a value or relationship is unknown. Uncertainty can result from lack of information or from disagreement about what is known or even knowable. Uncertainty may originate from many sources, such as quantifiable errors in the data, ambiguously defined concepts or terminology, or uncertain projections of human behaviour.
Uncertainty can therefore be represented by quantitative measures, for example, a range of values calculated by various models, or by qualitative statements, for example, reflecting the judgment of a team of experts.
This broad definition can be compared with stricter definitions, notably those which seek to separate uncertainty, where it is impossible to attach probabilities to outcomes, from risk, where probability is defined.
As a result there is a growing interest in decision support techniques that can address different elements of uncertainty. A brief description of some of the emerging approaches is presented below.
Real Option Analysis (ROA) is an economic decision support tool that quantifies the investment risk associated with uncertain future outcomes. The approach derives from the financial markets, where it has been used to assess the valuation of financial options and risk transfer. The same insights are also useful when there is risk or uncertainty involved with investment in physical assets, hence ‘real’ options.
The approach can be used to consider the value of flexibility. This includes the flexibility over the timing of a capital investment, but also the flexibility to adjust the investment as it progresses over time, i.e. allowing a project to adapt, expand or scale-back in response to unfolding events or from new information (learning). The approach can therefore assess whether it is better to invest now or to wait – or whether it is better to invest in options that offer greater flexibility in the future. ROA can also be used to support initial enabling steps to help secure projects for future development. The most common application of the approach uses decision trees, defining possible outcomes, and assigning probabilities to these, then using expected values to compare alternatives.
Real Option Analysis has been widely cited as a possible decision tool for adaptation, as the concepts of the approach align closely with iterative decision making and adaptive management. It is a powerful decision-support tool with a number of strengths, most notably that it provides information in quantitative and economic terms. However, it is a technical complex and resource intensive, and requires probabilistic (or probabilistic-like) information on outcomes. In practice, the approach is most relevant for particular types of decisions, primarily those that involve large up-front irreversible investments (e.g. coastal protection, large water storage projects), where there is flexibility in the timing, and the opportunity for new information to emerge. The framework is most likely to be supportive of projects that have some combination of substantial near-term benefits, and the ability to scale-up or down in line with learning regarding potential upside benefits or downside risks.
Robust Decision Making (RDM) is a decision support tool that is used in situations of deep uncertainty, i.e. in the absence of probabilistic information on scenarios and outcomes. The key aim of RDM is to seek strategies that are robust over many future outcomes, i.e. that are 'good enough' and minimize regret. It therefore offers an alternative to a conventional cost-benefit analysis and the identification of optimal options on the basis of economic efficiency.
The formal application of the approach (Lempert et al, 2003: Groves and Lempert, 2007) uses quantitative models, or scenario generators, with data mining algorithms, to evaluate how different strategies perform under large ensembles of scenarios reflecting different plausible future conditions (using hundreds to thousands to millions of input combinations). The aim is to help policymakers minimise the negative impacts of possible future outcomes. Iterative and interactive techniques are then applied to “stress test” different strategies to identify potentia
vulnerabilities or weaknesses of proposed approaches.
RDM has many attributes that align with the concept of adaptive management (Lempert and Groves, 2010) and the approach has been widely recommended for adaptation. The approach can assess robustness using various metrics, including physical effectiveness or economic efficiency. It can also be used in an iterative framework (Lempert, 2010) which aligns it more closely to the iterative adaptation management concepts of monitoring, research, evaluation and learning.
It is a particularly useful tool when future uncertainties are poorly characterised and probabilistic information is limited or not available – a key strength for long-term climate change related decisions – though the formal (computer modelling based) application of the approach has a high demand for quantitative information and computing power. As a result, the broad concepts of the approach have also been applied to the climate domain looking at climate uncertainty, to help in selecting options that are robust across a wide-range of plausible (climate) futures.
Portfolio Analysis (PA) is a decision support tool that helps in developing portfolios of options, rather than single options. It originated in the context of financial markets to explore the potential for portfolios of financial assets to maximise the financial return on investments, subject to a given level of risk. PA helps in the design of such portfolios. It aims to spread investments over a range of asset types to spread risks at the same time, thereby reducing the dependence on a single asset.
The approach highlights the trade-off between the returns on an investment and the riskiness of that investment, measuring risk by estimating the variance (standard deviation) of the portfolio return: thus a portfolio with a relatively high (low) variance is judged to have a higher (lower) risk (Aerts et. al. 2008). The information on returns and risks is used to identify a portfolio that most closely matches (risk) preferences.
The principles of diversification and the use of portfolios have high relevance for climate change adaptation (including iterative adaptive management, see IPCC SREX, 2012), and the technique can quantify the effectiveness of portfolios of options against climate change uncertainty. In the climate change context, the trade-off is then between the possibility of a high degree of effectiveness in reducing climate risks, and the risk that the adaptation options will fail to be effective over a certain range of climate change. PA allows the selection a set of options that, together, are effective over the range of possible projected future climates, rather than one option that is best suited to one possible future climate.
The main strength of the approach is that it provides a structured way to assess adaptation portfolios in a way that other decision tools do not allow. The approach can be undertaken using various metrics, including physical effectiveness, cost effectiveness, or economic efficiency and this provides flexibility for application to a wide range of applications.
However, the formal application is technically complex, and relies on the availability of data on effectiveness and co-variance, which requires probability, and it can be difficult to apply for some risks, notably where climate attribution is less straightforward, and/or the effectiveness of adaptation is not easily measured.
Adaptive Management (Iterative Risk Management) is a long established approach that uses a monitoring, research, evaluation and learning process to improve future management strategies.
The potential for an iterative approach for addressing climate change has been recognised for some time (Tompkins and Adger, 2004) and aligns with the IPCC AR4 Act-Learn-Then act again approach (Klein et al. 2007). The approach has been widely recommended for adaptation, including in the latest IPCC Special Report on Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation (SREX) (IPCC, 2012).
The approach is less formalised than many of the tools above, but the focus is on the management of uncertainty, allowing adaptation to work within a process of learning and iteration. Recent applications have also used the term ‘adaptation pathways’ to reflect a shift from predict-andoptimise based approaches to a dynamic pathway that incorporates uncertainty (Downing, 2012).
The most recent applications identify possible risk or impact thresholds (and accompanying indicators) and assess options (or portfolios of options) that can respond to these threshold levels. These are accompanied by monitoring plans that track key indicators, and through a cycle of evaluation and learning, allows the adjustment of plans over time. The appraisal of options within these pathways can be undertaken using some of the tools above, using qualitative or semi-quantitative decision support tools, such as MCA, or more formalised economic appraisal with CBA (though it would also be possible to use the alternative tools above in such a framework). The results of these iterative assessments are often presented as adaptation pathways or route maps. While most applications have been at the project level, notably for sea level rise, there are now examples emerging of more strategic or even national level plans (see Watkiss and Hunt, 2011: Watkiss et al, 2013).
A variation of the approach is to consider major biophysical, human, social or economic thresholds, and the MEDIATION project has developed such assessments using the term ‘Adaptation Turning Points’ looking at socialpolitical thresholds (i.e. a formal policy objective or societal preference).
The advantage of the approach is that rather than taking an irreversible decision now about the ‘best’ adaptation option – which may or may not be needed depending on the level of climate change that arises – it encourages decision makers to ask “what if” and develop a flexible approach, where decisions are made over time, and these plans adjusted as the evidence emerges (Reeder and Ranger, 2011). This allows the right decisions are taken at the right time (EA, 2011), such that additional options can be brought forward– or delayed to a later time period – depending on how climate change actually evolves. The disadvantage is that the identification of suitable risk thresholds can be difficult, especially when there are multiple risks or cross sectoral dimensions involved. Like many of the approaches above, it can be time and resource intensive, especially when addressing multiple scenarios in an economic framework.
Analytic Hierarchy Process (AHP) is a form of multi-criteria analysis (see earlier) that undertakes pairwise comparisons using expert judgements to derive priority scales (Saaty, 1980). The method allows the analysis of tangible and intangible elements together, allowing these to be traded off against each other in a decision-making process.
The method is applied by making comparisons using a scale of absolute judgements that represents how much one element dominates another for a given attribute. The derived priority scales are then synthesised and the various weighted scores are aggregated. The approach is very flexible and can be adapted to specific contexts. Criteria (or attributes) and sub-criteria can be decided in advance by experts or through a participatory process with stakeholders. There is no upper limit to the number of criteria or subcriteria, except for the time that is then required to do the comparison.
The approach can be used to choose options, to rank and/or prioritise them, and for resource allocation and conflict resolution. The tool has a particular relevance where important elements of the decision are difficult to quantify or compare, or where different expertise, goals, and worldviews are a barrier to consensus-building and communication.
AHP has been used in a wide variety of fields. The approach has high relevance for the application to adaptation, as it can evaluate options in situations of high complexity, considering different time horizons, uncertainty, multiple and interdependent variables, and subjectivity, all of which requires multidimensional trade-offs. It allows the comparison of diverse elements that are often difficult to measure in a structured and systematic way using a scale, though this inevitably involves a degree of subjectivity.
download this briefing note