An effective screening design for sensitivity analysis of large models
Introduction
The use of models to inform and support the decision-making process is becoming extremely important nowadays. Whatever the type of modelling process, certain common steps must be followed if the goal is to obtain credible results and valuable information (see, e.g. Jakeman et al., 2006 for a list of recommended steps in the development of environmental models). Uncertainty and sensitivity analysis are important steps in the model building process. Different approaches can be followed to test the sensitivity of a model (Saltelli et al., 2004, Saltelli et al., 2005, Cacuci and Ionesco-Bujor, 2004). Sensitivity methods range from the quantitative variance-based methods, defined from the decomposition of the total output variance into the contributions of the input factors, to other forms of global sensitivity with regional properties (Pappenberger et al., 2006), down to the simplest class of the One Factor At a Time (OAT) screening techniques, which simply vary one factor at a time and measure the variation in the output. In contrast to the OAT, global methods require a high number of model evaluations, increasing with the number of factors. In 1991 Morris proposed a method, which is particularly well-suited when the number of uncertain factors is high and/or the model is expensive to compute. The method is based on calculating for each input a number of incremental ratios, called Elementary Effects (EE), from which basic statistics are computed to derive sensitivity information. While the EE method was proven to be a very good compromise between accuracy and efficiency, especially for sensitivity analysis of large models (see, for instance, Campolongo and Braddock, 1999), it is still not extensively used. In this paper we aim to recall the attention of the modelling community to the effectiveness of this method and to enhance its efficiency. We propose a revised measure which increases its interpretability in the case of complex models with multiple inputs and outputs and which allows for its applicability to groups of factors. We propose a refined sampling strategy that allows for a better exploration of the space of the input factors. We then employ the method to assess the sensitivity of a large chemical model to its input factors.
Section 2 describes the EE screening sensitivity measure as originally proposed. In Section 3, we present the revised version of the EE measure, show how to apply it to groups of factors and describe the improved sampling strategy. Section 4 investigates by means of analytical examples the relationship between the revised EE method and the variance-based measures. Section 5 illustrates the sensitivity of a model relevant to climate change studies, which describes the tropospheric air and droplet chemistry for dimethylsulphide (DMS). Section 6 contains our conclusions.
Section snippets
The elementary effects method
The guiding philosophy of the original EE method (Morris, 1991) is to determine which input factors may be considered to have effects which are (a) negligible, (b) linear and additive, or (c) non-linear or involved in interactions with other factors. For each input, two sensitivity measures are computed: μ, which assesses the overall influence of the factor on the output, and σ, which estimates the ensemble of the factor's higher order effects, i.e. non-linear and/or due to interactions with
Improving the sampling strategy
The EE method is based on the construction of r trajectories in the input space, typically between 10 and 50. The design is based on generating a random starting point for each trajectory and then completing it by moving one factor at a time in a random order. This strategy could lead to a non-optimal coverage of the input space, especially for models with a large number of input factors.
We propose an improvement of the sampling strategy, which aims at a better scanning of the input domain
μ∗ versus the variance-based measures
Let us compare the sensitivity measures μ∗ and σ, discussed in the previous sections, with a class of well established sensitivity measures, the variance-based measures (Saltelli et al., 1999), which can be regarded as good practice in sensitivity analysis (Saltelli et al., 2004, Santner et al., 2003). The idea of variance-based measures is that the total output variance for a model with input factors can be decomposed as: where ,
The kim model
Dimethylsulphide (DMS, CH3SCH3) is the major biogenic sulphur gas emitted into the atmosphere from phytoplankton in the oceans. DMS has been tentatively identified as the major precursor of condensation nuclei and eventually cloud condensation nuclei in remote marine regions. DMS could thus have a significant influence on the earth's radiation budget and possibly on climate regulation. The effect of DMS on the climate is critically dependent on the production of gas-phase sulphuric acid and new
Conclusions
Although 14 years have passed since the EE sensitivity method was published (Morris, 1991), examples of its application are still rare in the literature. Too often the simplest local analyses, varying one factor at the time around a baseline point, are employed.
In this piece of work, we have recalled the efficacy of the EE design and proposed some improvements for it. We have revised the definition of its sensitivity measures in order to extend its utility to models with multiple outputs and to
References (28)
- et al.
Sensitivity analysis of the IMAGE Greenhouse Model
Environmental Modelling and Software
(1999) - et al.
Sensitivity analysis of an environmental model: an application of different analysis methods
Reliability Engineering and System Safety
(1997) - et al.
Nonlinear sensitivity analysis of multiparameter model systems
Journal of Computational Physics
(1978) - et al.
Analysis of soil carbon outcomes from interaction between climate and grazing pressure in Australian rangelands using Range-ASSESS
Environmental Modelling and Software
(2006) - et al.
Importance measures in global sensitivity analysis of model output
Reliability Engineering and System Safety
(1996) - et al.
Ten iterative steps in development and evaluation of environmental models
Environmental Modelling and Software
(2006) - et al.
Towards coupling 3D hydrodynamic lake model with the Canadian Region Climate Model: Simulation on Great Slave Lake
Environmental Modelling & Software
(2007) - et al.
Sensitivity analysis based on regional splits and regression trees (SARS-RT)
Environmental Modelling and Software
(2006) - et al.
The effect of error in gridded digital elevation models on the estimation of topographic parameters
Environmental Modelling & Software
(2006) - et al.
About the use of rank transformation in sensitivity analysis of model output
Reliability Engineering and System Safety
(1995)
Making best use of model valuations to compute sensitivity indices
Computer Physics Communications
Sensitivity analysis practices. Better strategies for model-based inference
Reliability Engineering and System Safety
A comparative review of sensitivity and uncertainty analysis of large-scale systems-II: statistical methods
Nuclear Science and Engineering
The role of multiphase chemistry in the oxidation of dimethylsulphide (DMS). A latitude dependent analysis
Journal of Atmospheric Chemistry
Cited by (1457)
Building a simple multivariable filtration model to predict irreversible fouling when directly filtering municipal wastewater
2024, Journal of Environmental Chemical EngineeringWhat drives uncertainty surrounding riverine flood risks?
2024, Journal of HydrologyComputational modelling and simulation to mitigate the risk of daylight exposure in tropical museum buildings
2024, Energy and Built Environment