Karen Willcox Joint work with Chad Lieberman

SIAM Conference on Uncertainty Quantification

Raleigh, NC April 4, 2012

Model reduction for uncertainty quantification of large-scale systems

Collaborators and Contributors

• Krzysztof Fidkowski • Phuong Hunyh • David Galbally • Omar Ghattas • Bart van Bloemen Waanders

Doug Allaire Tan Bui-Thanh Leo Ng Chad Lieberman

Outline

• Uncertainty quantification of large-scale systems

• Projection-based model reduction and the challenge of high-dimensional parameters

• Model reduction approaches that are – Goal oriented – Exploit problem structure

• Conclusions and outlook

UQ for large-scale complex systems

Characterizing, representing and analyzing uncertainty in large-scale, multidisciplinary simulation tools is essential

– To support decision-making processes (optimization, control, design, policy-making)

– To inform model development

FAA Environmental Tools Suite

The challenge of UQ for large-scale complex systems

• Many sources and types of uncertainty

• Formulation is key (and often overlooked); little may be known about the character of uncertainties

• Forward models are complicated: systems of PDEs, nonlinear, multi-scale, multi-physics

• Cost of forward solve may be prohibitive for repeated evaluations (sampling, scenarios)

• Many problems are characterized by uncertain parameters of high dimension

Main message

• Seemingly intractable challenges of UQ for large-scale complex systems can be overcome if

we use approaches that are teleological† and structure-exploiting

† of or pertaining to teleology, the philosophical doctrine that final causes, design, and purpose exist in nature

From Ancient Greek τέλος (telos, “purpose”) + λόγος (logos, “word, speech, discourse”)

(http://en.wiktionary.org)

Data-fit models

Surrogate modeling

Projection-based reduced models • Exploit problem structure • Embody underlying physics

= + = +

Simplified physics models

=

=

p1 p2

Model reduction for applications in UQ

• Forward propagation of uncertainty (e.g., impact of uncertain geometry parameters in unsteady CFD analyses; Tan Bui-Thanh)

• Sensitivity analysis (e.g., apportionment of variance in estimating aircraft emissions; Doug Allaire)

• Statistical inverse problems (e.g., characterization of combustor reaction parameters; David Galbally, Krzyzstof Fidkowski)

• Optimization under uncertainty (e.g., design of an acoustic horn under uncertain operating conditions Leo Ng, Phuong Hunyh)

Parameterized large-scale dynamical systems

Arising, for example, from systems of ODEs or spatial discretization of PDEs describing the system of interest.

Example: CFD systems

Projection-based state-reduced models

• Approximate state by a linear combination of basis vectors – Define right basis, V

• Project equations onto reduced-order subspace

– Define left basis, W

x V xr

Projection-based state-reduced models

State basis example: Proper orthogonal decomposition (POD)

• Consider K snapshots (Sirovich, 1987) (solutions at selected times or parameter values)

• Choose the n basis vectors to be left singular vectors of the snapshot matrix, with

singular values

• This is the optimal projection in a least squares sense:

• Form the snapshot matrix

(aka Karhunen-Loève expansions, Principal Components Analysis, Empirical Orthogonal Eigenfunctions, …)

• For wide range of applications, orders of magnitude reductions are achievable while maintaining high accuracy

Why does model reduction work?

• Input → Output map is often much simpler than the full simulation model suggests

Inputs State Outputs

x u y

Why does model reduction work? • In the linear case, the complexity of the input—output

map can be quantified in rigorous terms

“Reachable” modes – easy to reach – dominant eigenmodes of a

controllability gramian

“Observable” modes – generate large output energy – dominant eigenmodes of an

observability gramian

x u y

“Hankel singular values are to model order what singular values are to matrix rank.” (Matlab hsvd documentation)

Model reduction: State of the art

• Many projection-based model reduction methods – Krylov-based, POD, balanced truncation, reduced basis,

proper generalized decomposition, modal analysis, model-constrained optimization approaches, etc.

• Methodology “mature” for linear time-invariant systems with few inputs/few outputs – In many cases, rigorous error estimators available

• Recent breakthroughs in parametrically varying and nonlinear systems – Empirical Interpolation Method (Barrault et al., 2004)

– Discrete Empirical Interpolation Method (Chaturantabut & Sorensen, 2010)

• High-dimensional parameter spaces? – Essential component for UQ applications of interest

High-dimensional parameters: Why a challenge?

• Most model reduction methods sample the parameter space to build the basis

→ model unaware sampling breaks for > 5 parameters

• If we have many parameters (hundreds, thousands) can we really expect the input—output map to be low-dimensional?

x p y

y

Distributed parameters characterize subsurface properties, contaminant release initial conditions, etc., resulting in thousands of discretized parameters

hydraulic conductivity

pressure data

High-dimensional parameters: The path forward

• Even if the parameter space is of high dimension, the outputs of interest are often of very low dimension – Engineering decisions are usually of low dimension (~1)

• If you have many outputs, is your system really encompassing the ultimate prediction/decision?

• Our approach: define a purpose (prediction/decision goal) exploit problem structure

x p y d

Model reduction for systems with high-dimensional parameters

• Formulate the problem to account for the ultimate decision/prediction quantity of interest

• Re-parameterize the problem in a goal-oriented manner

• Use optimization to build the basis by searching a high-dimensional space efficiently in a goal-oriented manner

x p y d

Parameter and state reduction

Lieberman, W., Ghattas; SISC 2010

Optimization-based goal-oriented sampling

• Use “greedy” approach of Veroy et al., 2003; Grepl & Patera, 2005 to compute state basis V and parameter basis P

• We formulate the task of finding the parameter sample points as an optimization problem

–Efficient search of a high-dimensional parameter space –Exploit problem structure –Sampling is goal-oriented

(driven by outputs of interest)

• Linear problem: explicit solution via eigenvalue problem (Bashir et al.; Int. J. Num. Meth. In Engr., 2007)

• Nonlinear problem: use residual as error indicator, solve with tailored PDE-constrained optimization algorithm (Bui-Thanh, W., Ghattas; SIAM J. Sci. Comp., 2008)

Statistical inference of distributed parameter: Porous media flow

• Given sparse pressure head data, invert for the hydraulic conductivity field in a porous medium

• Reduced-order model 494 states → 10 states 49 parameters → 10 parameters

Full:

Isocontours of posterior MCMC samples

Reduced:

x

y

Parameter Basis

State Basis

Lieberman, W., Ghattas; SISC 2010

Problem reformulation: Exploiting the data-to-prediction map

• Experimental data: low-dimensional O(102) • Parameter: high-dimensional O(105) • Prediction output of interest: low-dimensional O(1)

• Identify reduced parameter subspace in which to perform the inference, with the goal of accurate predictions → “inference for prediction”

• An example where making the system boundary larger may make the problem “simpler”

Inference for prediction: Linear problem

• Forward model: advection-diffusion – State, x: contaminant concentration – Parameters, p: initial concentration x0

– Experimental outputs, ye: sparse measurements of concentration

• Inverse problem: reconstruct initial contaminant distribution, given sparse measurements of state from distributed sensors over specified time horizon

• Prediction problem: define prediction output of interest (e.g., concentration at later time)

•25

Prediction

Mutual

• Measurements :

A control-theoretic approach to “inference for prediction”

Experiment

Lieberman, W.; SISC, submitted

• Prediction outputs of interest: • Parameter (high-dimensional):

Mode informed by experiment and required for prediction

• Determine modes that are both experiment and prediction observable

• Sacrifice inversion accuracy but maintain accuracy in output predictions; reveal primary contributors of prediction uncertainty

*Primary source of uncertainty in predictions

Mode informed by experiment but not needed for prediction

Mode needed for prediction but not informed by experiment*

Inference for prediction basis

Controllability Gramian

x u y

Observability Gramian

Balanced truncation model reduction

Inference for prediction basis

Controllability Gramian

Prediction Hessian

x u y p ye yp

Observability Gramian

Experiment Hessian

Inference for prediction

Lieberman, W.; SISC, submitted

Balanced truncation model reduction

Inference for prediction: Linear theory

Similar algorithms/results for analogous treatment of Tikhonov-regularized linear inverse problems and linear-Gaussian statistical inverse problems.

Lieberman, W.; SISC, submitted

Inference for prediction: Truncated SVD

Prediction output of interest:

Comparing predictions from IFP and TSVD

approaches

N=4005 r=54 (Ve) s=15 (W)

Synthetic data generated from prescribed initial condition

Inference for prediction: Tikhonov regularization

Prediction output of interest:

Comparing predictions from IFP and Tikhonov-regularized approaches

r=4005 (=N) s=11 (W)

Synthetic data generated from prescribed initial condition

Inference for prediction: Statistical inverse problem

Prediction outputs of interest:

Posterior predictive density function contours

Traditional approach r=4005 (=N)

IFP approach s=2

Conclusions

• Surrogate models play an essential role for many uncertainty quantification applications – Forward propagation of uncertainty – Sensitivity analysis (variance apportionment) – Statistical inverse problems – Optimization under uncertainty

• Three-pronged approach for tackling reduction of problems with high-dimensional parameter spaces: – Formulate the problem to account for the ultimate

decision/prediction quantity of interest – Re-parameterize the problem in a goal-oriented manner – Use optimization to build the basis by searching a high-

dimensional space efficiently in a goal-oriented manner

• Not all problems are amenable to model reduction – But many are, especially if you keep your goal in mind

Outlook

• Even though we are making progress on propagating

and analyzing uncertainties for complex systems, what does it mean?

Communicating uncertainty to decision-makers remains a significant challenge.

Problem formulation is an essential but often poorly understood aspect in engineering design under uncertainty. “The model is a deterministic computer code, whose outputs are not random variables (and most likely not Gaussian), so the fiction begins here.” Anonymous reviewer, December 2010.

Outlook

• Even though we are making progress on propagating

and analyzing uncertainties for complex systems, what does it mean?

Model inadequacy poses a critical risk to the decision

process, but is usually ignored.

“No model is perfect. Even if there is no parameter uncertainty, so that we know the true values of all the inputs required to make a particular prediction of the process being modelled, the predicted value will not equal the true value of the process. The discrepancy is model inadequacy.'' Kennedy and O'Hagan, 2001.

Acknowledgements

• This work was supported by AFOSR Computational Mathematics Program and AFOSR MURI on Uncertainty Quantification (F. Fahroo), Department of Energy Advanced Scientific Computing Research Program (S. Landsberg), Singapore-MIT Alliance Computational Engineering Programme, Kambourides Graduate Fellowship in Computational Engineering