Navigating Cascades of Uncertainty — As Easy as ABC? Not Quite . . .

The uncertainties in scienti ﬁ c studies for climate risk management can be investigated at three levels of complexity: “ ABC ” . The most sophisticated involves “ Analyzing ” the full range of uncertainty with large multi-model ensemble experiments. The simplest is about “ Bounding ” the uncertainty by de ﬁ ning only the upper and lower limits of the likely outcomes. The intermediate approach, “ Crystallizing ” the uncertainty, distills the full range to improve the computational ef ﬁ ciency of the “ Analyze ” approach. Modelers typically dictate the study design, with decision-makers then facing dif ﬁ culties when interpreting the results of ensemble experiments. We assert that to make science more relevant to decision-making, we must begin by considering the applications of scienti ﬁ c outputs in facilitating decision-making pathways, particularly when managing extreme events. This requires working with practitioners from outset, thereby adding “ D ” for “ Decision-centric ” to the ABC framework.


Introduction -The Cascade of Uncertainty in Scientific Modeling
Science informs complex decisions ranging from climate adaptation strategies to appraisals of investment opportunities. Numerical and physical models, which are used to represent reality, help resource managers explore the outcomes of different trade-offs and decisions. However, modeling is a complex set of activities, with multiple choices around model selection, identifying which processes to include, and how to benchmark performance. Such choices present contrasting modeling options, each of which leads to potentially different but equally plausible outcomes. Wider factors also shape the analysis, including the sensitivity of the system to climate, the level of risk associated with changing or retaining rules and regulations, the capacity (technical, human, economical, institutional) to undertake a climate change impact assessment, and the societal, political and regulatory context. Hence, in addition to their scientific merit, the resource required to implement each strategy (A, B or C) will also influence the final design.
Which pathway to take through the choices in a modeling study is inherently subjective, with the range of choices expanding at each stage, creating a "cascade of uncertainty". This concept was developed by the climate science community to describe how uncertainty ranges expand along the modeling chain. For example, use of high or low greenhouse gas emission scenarios to force a climate model will lead to different global climate responses. Next, the choice of climate model itself will affect the simulated outputs significantly. The initial and boundary conditions and values assigned to model parameters likewise affect model output. These uncertainties grow as more permutations in the modeling chain are explored. This concept was first graphically represented by Schneider (1983) as the "Uncertainty Explosion", and subsequently portrayed by Wilby and Dessai (2010) as a top-down, pyramid shaped, cascade of uncertainty that extends to local impacts and adaptation responses. The cascade applies beyond climate science to any modeling study, whether of the environment or economics, and applies to past, present and future processes. It also relates to interdependent modeling studies (i.e., modeling chains), whereby output(s) from one model (e.g., regional climate change) is used to drive other systems (e.g., environmental impact), and in turn to interrogate decision options (e.g., economic analysis). Quantifying the uncertainty in such cascades, while scientifically justifiable, is complex and resource intensive. Therefore, it is crucial to identify modeling pathways that are technically defensible, yet feasible for decision-makers with varying levels of resource and expertise.

ABC Approaches to Navigating the Cascade of Uncertainty
Strategies for characterizing uncertainty fall into three categories (herein "ABC"). First, uncertainty can be analyzed, by exhaustively characterizing as much of the cascade as possible. Alternatively, uncertainty can be bounded, by defining only the upper and/or lower limits of plausible outcomes. Finally, uncertainty can be crystallized by taking a representative sample from each constituent component allowing a more strategic characterization of the overall distribution. Figure 1 illustrates each approach as might be applied in a climate change risk assessment.
Decisions about how to navigate the cascade are typically made by expert modelers, who understand the limitations of different approaches. However, users that are reliant on model simulations to inform decisions are not always aware of the implications of choices for the robustness of model results or how effectively they capture uncertainty. Hence, information users may find it difficult to adequately situate model results in the context of other, wider uncertainties that may not have been analyzed. Consequently, we stress that there is a need for the ABC strategies to include a "D" for decision-making. For support, an example of each approach and some challenges currently faced by decision makers in their application are discussed. Although we focus on climate science, the concepts apply equally across other disciplines. highlights how decisions made at each step down the modeling cascade lead to potentially different outcomes, such that the uncertainty range amplifies as it passes from one stage in the modeling chain to the next. To comprehensively analyze uncertainty via this strategy requires expertise in a range of scientific disciplines supported by significant financial, computational and human resources. Such analyses cannot be tackled individually, and must rely on collaborative, crossdisciplinary efforts to pool sufficient resources. Examples of such collaborations include the Coupled Model Intercomparison Projects (CMIPs) for climate change science (Taylor et al. 2011), and the Inter-Sectoral Impact Model Intercomparison Projects (ISI-MIPs) for environmental science (Warszawski et al. 2014). Even these substantial projects are not exhaustive; they simply cannot explore every possible model combination or outcome, thus analyzing the entire cascade of uncertainty is unattainable. Projects such as these may undergo revisions and updates to include new data and different models. Decision-makers are then faced with an overwhelming amount of information at the end of the process, leading to a sense of helplessness and indecision about how best to proceed. Sometimes, the default position will be the ensemble mean, however, this can disguise multi-modal results or dampen the variability, and thus the extreme values which are often of most interest.

Bound it
The resources required for the exhaustive "analyze it" approach is an obstacle in many decision contexts. Hence, an informed estimate of upper and/or lower bounds of the final outcomes might be sufficient. In this case, effort is focused on the most extreme end-members of the modeling cascadein effect bracketing the uncertainty. This approach is represented in Figure 1(b), which shows the cascade from the "analyze it" approach, with banks marking the upper and lower limits of each stage down the cascade. Here only the most extreme cases are progressed at each point of investigation. Although appearing straightforward, this approach requires model investment, a capability to interpret complex uncertainty assessments, knowledge of physical limits to key processes (such as extreme rainfall), and expertise to define the cascade boundaries. Nonetheless, it is appealing for those prepared to embrace uncertainty from the outset and seeking to implement decisions that are robust across the range of modeled outcomes. This approach might also be regarded as precautionary.
Decision-makers may only require information on one of the uncertainty bounds (i.e., upper or lower), as may be the case when incorporating allowances for uncertainty in the design of new infrastructure (e.g., flood defense height, reservoir capacity, or nuclear power plant elevation above sea level). In these cases, the question is "how extreme are conditions likely to become during the lifetime of this structure?" or "what is the severity of the 200 year return period event?" For example, the Office for Nuclear Regulation and Environment Agency (2017) joint advice requires that developers of new nuclear build consider a credible maximum scenario defined therein as "a peer-reviewed, high-end, plausible scenario". As an example, the guidance references the Hþþ scenarios (Wade et al. 2015), from the UK Committee on Climate Change, which estimate extreme climate change scenarios outside the range of the commonly employed UKCP09 projections (Defra et al. 2009). Similarly, the Dam Safety Regulation of the Canadian province of Québec requires that every impoundment must be able to withstand the 10,000 year flood or Probable Maximum Flood, depending on the dam category, but without explicitly accounting for climate change (Éditeur Officiel du Québec 2017). The "bound it" approach is currently much more decision-centric than that of the "analyze it" approach. Even so, expert opinion may be divided about what constitutes a plausible extreme high-or low-end bracket.

Crystallize it
In some cases, analyzing the entire cascade is unfeasible, yet more comprehensive information about possible outcomes beyond the "bound it" approach is required. Hence, it should be possible to apply knowledge of both the physical world and our models to adopt a "smarter" approach. Figure 1(c) illustrates this, whereby samples are taken from the spectrum of potential results at each step to reduce or "crystallize" salient outcomes. This approach is utilized in operational weather forecasting, such as the UK Met Office 42 member ensemble forecasting system (MacLachlan et al. 2015). In real-time forecasting, urgency and computational constraints preclude a full "analyze it" assessment. However, uncertainty may still be captured within "crystallize it" ensemble experiments to provide probabilistic statements to decision-makers and the public. A "crystallize it" approach to smartsampling the uncertainty range is widely applied by the research community. Example applications that demonstrate uncertainty analyses in environmental modeling chains include hurricane forecasting (Munsell and Zhang 2014), climate change impacts on maize agriculture in Africa (Dale et al. 2017), and climate impacts on UK water resources (Christierson et al. 2012). Clark et al. (2016) formalized a smart-sampling approach to characterize uncertainty in hydrology modeling as the "storyline approach", and emphasized that certain sources of uncertainty, such as internal variability, are commonly neglected by the water management community. Despite the popularity of this approach, choices about how many, and which sub-samples to select can be made arbitrarily rather than on the basis of scientific assessment or decision context. Hence, there is still room for more objective and decision-relevant sampling of the cascade of uncertainty.

Discussion -Ways Forward
Each of the above ABC strategies has their place in decision-making in managing extremes in an uncertain world. The scientific community will, no doubt, continue to strive for more comprehensive ("analyze it") uncertainty assessments. These ensemble results should be made more widely available so that others can develop smart-sampling approaches. Care should also be taken when communicating the messages from such vast ensembles; hence scientists and decision-makers must work closely to ensure results are appropriately derived and interpreted. Conversely, the "bound it" approach is helpful for lower budget projects, where stress testing under extreme scenarios is required, or whenever climate change has a marginal influence on the application compared with other risks. Similarly, a strategically sampled ("crystallize it") approach can conserve resources whilst enabling decision-makers to capture the essence of the uncertainty space in more interpretable forms. However, to meet their needs and avoid undue confidence in the resultant bounds, it is essential that decision-makers are consulted from the outset regarding how the cascade is sampled. A prime example of "crystallizing" the uncertainty with both the scientific and the decision-making communities in mind is that of the Representative Concentration Pathways and Shared Socio-Economic Pathways (IPCC 2014). These scenarios reflect a carefully considered range of plausible outcomes that have been endorsed and extensively applied in modeling studies to aid decision makers in quantifying climate change uncertainties.
In summary, which one of the strategies (A, B, or C) is adopted for navigating scientific cascades of uncertainty should ultimately depend on the Decision contextthe specific decision question, the risks involved, the resources available, and the scale of investigation. This supposes greater involvement of decisionmakers in the design and execution of uncertainty analysesmore purposeful evaluation and communication of uncertainty would certainly result. We argue that framing ABC in the light of D will enable more meaningful progress in real-world applications, as the importance of tackling uncertainty holistically, yet efficiently, is recognized by both communities.