Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index

doi:10.1016/j.cpc.2009.09.018

Computer Physics Communications

Volume 181, Issue 2, February 2010, Pages 259-270

https://doi.org/10.1016/j.cpc.2009.09.018 Get rights and content

Abstract

Variance based methods have assessed themselves as versatile and effective among the various available techniques for sensitivity analysis of model output. Practitioners can in principle describe the sensitivity pattern of a model $Y = f (X_{1}, X_{2}, \dots, X_{k})$ with k uncertain input factors via a full decomposition of the variance V of Y into terms depending on the factors and their interactions. More often practitioners are satisfied with computing just k first order effects and k total effects, the latter describing synthetically interactions among input factors. In sensitivity analysis a key concern is the computational cost of the analysis, defined in terms of number of evaluations of $f (X_{1}, X_{2}, \dots, X_{k})$ needed to complete the analysis, as $f (X_{1}, X_{2}, \dots, X_{k})$ is often in the form of a numerical model which may take long processing time. While the computational cost is relatively cheap and weakly dependent on k for estimating first order effects, it remains expensive and strictly k-dependent for total effect indices. In the present note we compare existing and new practices for this index and offer recommendations on which to use.

Section snippets

Introduction to variance based measures

Sensitivity analysis is the study of how uncertainty in the output of a model (numerical or otherwise) can be apportioned to different sources of uncertainty in the model input factors, factors from now on [30]. Existing regulatory documents on impact assessment recommend the use of quantitative sensitivity analysis [7], [21]. Official guidelines insist on the importance of taking factor interactions into account [7], [9]. Variance based methods [6], [37] are well suited to this task and have

Sensitivity indices

Given a model of the form $Y = f (X_{1}, X_{2}, \dots X_{k})$ , with Y a scalar, a variance based first order effect for a generic factor $X_{i}$ can be written as (see notations in Table 1): $V_{X_{i}} (E_{X_{\sim i}} (Y | X_{i}))$ where $X_{i}$ is the i-th factor and $X_{\sim i}$ denotes the matrix of all factors but $X_{i}$ . The meaning of the inner expectation operator is that the mean of Y is taken over all possible values of $X_{\sim i}$ while keeping $X_{i}$ fixed. The outer variance is taken over all possible values of $X_{i}$ . The associated sensitivity measure (first order

Best practices for the simultaneous computation of $S_{i}$ and $S_{Ti}$

We discuss here existing estimators to compute in a single set of simulations both sets of indices $S_{i}$ and $S_{Ti}$ . By ‘simulation’ we mean here the computation of an individual value for Y corresponding to a sampled set of k factors $X_{1}, X_{2}, \dots, X_{k}$ .

We imagine to have two independent sampling matrices A and B, with $a_{j i}$ and $b_{j i}$ as generic elements. The index i runs from one to k, the number of factors, while the index j runs from one to N, the number of simulations. We now introduce matrix $A_{B}^{(i)}$ $(B_{A}^{(i)})$

Computational scheme for $S_{Ti}$

To compute $S_{Ti}$ from formula (f), which represents the best practice so far, the design matrices A and $A_{B}^{(i)}$ have to be set-up. Different methods may be used. In the following two different designs are compared: the first, called ‘radial design’, has been firstly presented in [25]; the second, called ‘winding design’ derives from the method discussed in [14]. The two designs are illustrated in Table 3. Let us focus first on the left-hand side. This shows how – starting from the fist row made of

Using Sobol' quasi-random sequences

Several types of quasi-random (QR) sequences have been suggested by Faure, Niederreiter, Halton, Hammersley, Sobol' and other investigators, see Bratley and Fox [3] for a review of these works.

QR sequences are specifically designed to generate samples of $X_{1}, X_{2}, \dots, X_{k}$ as uniformly as possible over the unit hypercube Ω.

Unlike random numbers, successive quasi-random points know about the position of previously sampled points and fill the gaps between them. For this reason they are also called

Numerical experiments

The following research questions concerning the estimation of $S_{Ti}$ are tackled here:

1.
Which is the best estimator for $S_{Ti}$ between estimators (e) and (f) in Table 2⁴?
2.
Which is the best strategy between winding stairs and radial sampling?
3.
Is $n > 1$ convenient with either of the above strategies?
4.
Is the answer to the questions above dependent upon the typology of the

Conclusions

The theory and the computational tools available to compute total sensitivity indices $S_{Ti}$ have been revised. The main motivation for the present work is that previous comparisons of different methods to estimate $S_{Ti}$ were based on incomplete combinations of sampling designs and estimators [5] or a limited set of test functions [25]. In this work a larger set of test functions has been employed reflecting different degrees of linearity, additivity and effective dimension. Further the simulations

Acknowledgements

Authors are particularly grateful to an anonymous reviewer who considerably helped in improving the manuscript.

References (48)

F. Campolongo et al.
An effective screening design for sensitivity analysis of large models
Environmental Modelling and Software
(2007)
J. Helton et al.
Survey of sampling-based methods for uncertainty and sensitivity analysis
Reliability Engineering and System Safety
(2006)
T. Homma et al.
Importance measures in global sensitivity analysis of model output
Reliability Engineering and System Safety
(1996)
J. Jacques et al.
Analysis in presence of model uncertainty and correlate inputs
Reliability Engineering and System Safety
(2006)
M.J.W. Jansen
Analysis of variance designs for model output
Computer Physics Communications
(1999)
S. Kucherenko et al.
Monte Carlo evaluation of derivative-based global sensitivity measures
Reliability Engineering and System Safety
(2009)
H. Rabitz et al.
Efficient input–output model representations
Computer Physics Communications
(1999)
M. Ratto et al.
State dependent parameter metamodelling and sensitivity analysis
Computer Physics Communications
(2007)
A. Saltelli
Making best use of model valuations to compute sensitivity indices
Computer Physics Communications
(2002)
A. Saltelli et al.
Screening important inputs in models with strong interaction properties
Reliability Engineering and System Safety
(2009)

A. Saltelli et al.

About the use of rank transformation in sensitivity analysis of model output

Reliability Engineering and System Safety

(1995)

I.M. Sobol'

On the distribution of points in a cube and the approximate evaluation of integrals

USSR Comput. Maths. Math. Phys.

(1967)

I.M. Sobol'

Uniformly distributed sequences with an addition uniform property

USSR Comput. Maths. Math. Phys.

(1976)

I.M. Sobol'

Global sensitivity analysis indices for the investigation of nonlinear mathematical models

Matematicheskoe Modelirovanie

(2007)

I.M. Sobol' et al.

Quasirandom sequence generators

Keldysh Inst. Appl. Maths RAS Acad. Sci.

(1992)

S. Tarantola et al.

Random balance designs for the estimation of first order global sensitivity indices

Reliability Engineering and System Safety

(2006)

S. Da Veiga, F. Wahl, F. Gamboa, Local polynomial estimation for sensitivity analysis for models with correlated...

G. Archer et al.

Sensitivity measures, anova-like techniques and the use of bootstrap

Journal of Statistical Computation and Simulation

(1997)

B. Bratley et al.

Implementation and test of low discrepancy sequences

ACM Transactions on Modeling and Computer Simulation

(1992)

P. Bratley et al.

Algorithm 659 implementing Sobol's quasirandom sequence generator

ACM Transactions on Mathematical Software

(1988)

K. Chan et al.

Winding stairs: A sampling tool to compute sensitivity indices

Statistics and Computing

(2000)

R.I. Cukier et al.

Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. i theory

The Journal of Chemical Physics

(1973)

B. Efron et al.

The jackknife estimate of variance

Annals of Statistics

(1981)

Cited by (2382)

A comprehensive parametric study and multi-objective optimization of turbulent jet array impingement for uniform cooling of gas turbine blades with minimized compression power
2024, International Journal of Thermal Sciences
In the present paper, a comprehensive parametric study and multi-objective optimizations on jet array impingement cooling are conducted for mid-chord sections of gas turbine blades to maximize the heat transfer uniformity on the target plate and minimize the air compression power consumption at different desired Nusselt numbers. The validated numerical method based on RANS equations is utilized to determine the effects of Reynolds number $(2500 \leq R e \leq 35000)$ , jet spacings ( $3 \leq P_{x}, P_{y} \leq 8$ ), and the jet-to-target distance ( $0.75 \leq P_{z} \leq 3$ ) on air compression power ( $W_{c}$ ), average Nusselt number ( $\overline{N u}$ ), and heat transfer uniformity index ( $U I$ ). According to the parametric study, the increase of $R e$ improves $\overline{N u}$ and $U I$ , while intensively increasing $W_{c}$ . The effect of jet-to-target distance is a function of jet spacings; with the increase of $P_{z}$ at low jet spacings, $W_{c}, \overline{N u}$ , and $U I$ are reduced. Although, at large spacings, $\overline{N u}$ and $U I$ increase with $P_{z}$ , and $W_{c}$ is independent of $P_{z}$ . Additionally, the increase of $P_{x}$ decreases $W_{c}$ , $\overline{N u}$ , and $U I$ . Increasing $P_{y}$ reduces $\overline{N u}$ and $W_{c}$ . But at small $P_{x}$ and $P_{z}$ , the $U I$ is descending; while at large $P_{x}$ and $P_{z}$ , the $U I$ tends to ascend. Three high-accuracy surrogate models are developed using backpropagation artificial neural networks (ANN) for estimating $W_{c}$ , $\overline{N u}$ , and $U I$ for input design variables. Sobol global sensitivity analysis is also performed based on the developed models for quantifying the influence of design variables and their interactions on objective functions. As expected, the results indicate that $W_{c}$ and $\overline{N u}$ are the most sensitive to $R e$ , and $U I$ is mainly affected by $P_{x}$ , whereas $P_{y}$ is more affecting the uniformity index rather than compression power. Finally, to find out the best design and flow conditions, optimizations are conducted by the NSGA-II algorithm. The optimal Pareto frontier and final decided solutions by TOPSIS and LINMAP methods are then demonstrated for the desired Nusselt number ( ${N u}_{D}$ ) of 70. The TOPSIS method indicates $P_{x} = 4.70$ , $P_{y} = 3.25$ , $P_{z} = 1.40$ , and $R e$ = 13800 as the best compromise for optimization. The analysis of Pareto solutions in the range of ${N u}_{D}$ from 35 to 130 suggests a variety of optimal flow and geometrical arrangements for a trade-off between objective functions; therefore, at most, 50% less compression power or 5% more uniformity is approachable corresponding to the design requirements.
Evaluation of solar energy potential for residential buildings in urban environments based on a parametric approach
2024, Sustainable Cities and Society
Building integrated photovoltaics is an important measure to promote low-carbon urban growth. The residential buildings, which play an important role in cities, have a great development potential to utilize solar resources. However, the solar utilization performance of buildings in a block is influenced by the shadings from surrounding buildings with diverse layouts and heights. Therefore, this study proposes a parametric approach to evaluate the solar energy potential of residential buildings by randomly generating 6730 block environments. We quantify and prioritize how block parameters influence the solar energy potential, and provide photovoltaics (PV) utilization strategies for the roof and façades of the building in diverse urban environments. The average photovoltaic installation ratio of roof, south and west façade can reach 98%, 46.3% and 38.5% when the target building height is higher than 24 m. Among all the block parameters, the target building height and its interactions with other parameters can collectively contribute more than 80% to the solar potential of building surfaces. Correspondingly, the two most influential parameters on the PV generation and installation are obtained for PV utilization strategies of building roof, south and west façade. The proposed framework and findings are expected to offer inspirations for solar design in urban buildings.
Deep learning based simulators for the phosphorus removal process control in wastewater treatment via deep reinforcement learning algorithms
2024, Engineering Applications of Artificial Intelligence
Phosphorus removal is vital in wastewater treatment to reduce reliance on limited resources. Deep reinforcement learning (DRL) can be used to optimize the processes in wastewater treatment plants by learning control policies through trial and error. However, applying DRL to chemical and biological processes is challenging due to the need for accurate simulators. This study trained six models to identify the phosphorus removal process and used them to create a simulator for the DRL environment. While achieving high accuracy ( $>$ 97%) in one-step ahead prediction of the test dataset, these models struggled as simulators over longer horizons, showing uncertainty and incorrect predictions when using their own outputs for multi-step simulations. Compounding errors in the models’ predictions were identified as one of the causes of this problem. This approach for improving process control involves creating simulation environments for DRL algorithms, using data from supervisory control and data acquisition (SCADA) systems with a sufficient historical horizon without complex system modeling or parameter estimation.
Large-scale sandwich structures optimization using Bayesian method
2024, International Journal of Mechanical Sciences
Benefiting from advanced features like high stiffness-to-weight ratios, sandwich structures are widely used in aerospace for primary and secondary structures. As tasks grow more complex and structures increase in scale, high-dimensional design spaces inevitably arise. Optimizing large-scale sandwich structures efficiently and intelligently presents certain challenges. Additionally, functional requirements and constraints, such as thermal deformation, should be fully considered in the design of practical structures like solar arrays, which involve a large expensive analysis and make the problem more complicated. This paper proposes to use efficient Bayesian optimization with the active subspace (AC) method to address this type of problem. The active subspace method, combined with the adaptive kriging and global sensitivity analysis (GSA), is employed for dimension reduction and reconstructing the design space. Then the structure is optimized within the reconstructed design space using the efficient Bayesian method. The proposed optimization strategy is applied to a case study of solar array with sandwich panels, demonstrating that the developed framework is feasible and effective for structural optimization of large-scale sandwich structures.
Identification of typical district configurations: A two-step global sensitivity analysis framework
2024, Energy
The recent geopolitical conflicts in Europe have underscored the vulnerability of the current energy system to the volatility of energy carrier prices. In the prospect of defining robust energy systems ensuring sustainable energy supply in the future, the imperative of leveraging renewable indigenous energy sources becomes evident. However, as such technologies are integrated into the existing system, it is necessary to shift from the current centralized infrastructure to a decentralized production strategy. This paper presents a method to identify a panel of technological solutions at the district level, intended to reduce complexity for the integration of decentralized models into a national-scale model. The framework’s novelty lies in combining a global sensitivity analysis for solution generation with clustering to identify typical configurations. The global sensitivity analysis is performed on a mixed integer linear programming model, which optimally sizes and operates district energy systems. The sensitivity analysis determines the most influential parameters of the model using the Morris method and provides a representative sampling of the solution space by leveraging the Sobol sampling strategy. The latter is then clustered using a density-based algorithm to identify typical solutions. The framework is applied to a suburban and residential Swiss neighborhood. The first outcome of the research is the high sensitivity of the model to energy carrier prices. As a result, Sobol’s sampling space separates itself into two system types: those based on a natural gas boiler and those relying on a combination of electrical heaters and heat pumps. For both types, the electricity demand is either fulfilled by PV panels or electricity imports. The identified configurations showcase that the framework successfully generates a panel of solutions composed of various system configurations and operations being representative of the overall solution space.
Formation of vascular-like structures using a chemotaxis-driven multiphase model
2024, Mathematical Biosciences
We propose a continuum model for pattern formation, based on the multiphase model framework, to explore in vitro cell patterning within an extracellular matrix (ECM). We demonstrate that, within this framework, chemotaxis-driven cell migration can lead to the formation of cell clusters and vascular-like structures in 1D and 2D respectively. The influence on pattern formation of additional mechanisms commonly included in multiphase tissue models, including cell-matrix traction, contact inhibition, and cell–cell aggregation, are also investigated. Using sensitivity analysis, the relative impact of each model parameter on the simulation outcomes is assessed to identify the key parameters involved. Chemoattractant–matrix binding is further included, motivated by previous experimental studies, and found to reduce the spatial scale of patterning to within a biologically plausible range for capillary structures. Key findings from the in-depth parameter analysis of the 1D models, both with and without chemoattractant–matrix binding, are demonstrated to translate well to the 2D model, obtaining vascular-like cell patterning for multiple parameter regimes. Overall, we demonstrate a biologically-motivated multiphase model capable of generating long-term pattern formation on a biologically plausible spatial scale both in 1D and 2D, with applications for modelling in vitro vascular network formation.

View all citing articles on Scopus

View full text

Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index

Abstract

Section snippets

Introduction to variance based measures

Sensitivity indices

Best practices for the simultaneous computation of Si and STi

Computational scheme for STi

Using Sobol' quasi-random sequences

Numerical experiments

Conclusions

Acknowledgements

Environmental Modelling and Software

Reliability Engineering and System Safety

Reliability Engineering and System Safety

Reliability Engineering and System Safety

Computer Physics Communications

Reliability Engineering and System Safety

Computer Physics Communications

Computer Physics Communications

Computer Physics Communications

Reliability Engineering and System Safety

Reliability Engineering and System Safety

USSR Comput. Maths. Math. Phys.

USSR Comput. Maths. Math. Phys.

Matematicheskoe Modelirovanie

Keldysh Inst. Appl. Maths RAS Acad. Sci.

Reliability Engineering and System Safety

Sensitivity measures, anova-like techniques and the use of bootstrap

Journal of Statistical Computation and Simulation

Implementation and test of low discrepancy sequences

ACM Transactions on Modeling and Computer Simulation

Algorithm 659 implementing Sobol's quasirandom sequence generator

ACM Transactions on Mathematical Software

Winding stairs: A sampling tool to compute sensitivity indices

Statistics and Computing

Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. i theory

The Journal of Chemical Physics

The jackknife estimate of variance

Annals of Statistics

Best practices for the simultaneous computation of $S_{i}$ and $S_{Ti}$

Computational scheme for $S_{Ti}$