Citation

## Material Information

Title:
Simulation-based optimal experimental design theories, algorithms, and practical considerations
Creator:
Walsh, Scott N. ( author )
Place of Publication:
Denver, Colo.
Publisher:
Publication Date:
Language:
English
Physical Description:
1 electronic file (162 pages) : ;

## Thesis/Dissertation Information

Degree:
Doctorate ( Doctor of philosophy)
Degree Grantor:
Degree Divisions:
Department of Mathematical and Statistical Sciences, CU Denver
Degree Disciplines:
Applied mathematics

## Subjects

Subjects / Keywords:
Uncertainty -- Mathematical models ( lcsh )
Probabilities ( lcsh )
Inverse problems (Differential equations) ( lcsh )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

## Notes

Review:
Several new approaches for simulation-based optimal experimental design are developed based on a measure-theoretic formulation for a stochastic inverse problem (SIP). Here, an experimental design (ED) refers to an experiment defining a particular map from the space of model inputs to the space of observable model outputs. The term optimal experimental design (OED) then refers to the choice of a map from model inputs to observable model outputs with optimal properties. We develop a consistent Bayesian approach to OED, which follows the classical Bayesian approach to OED, and determines the OED based on properties of approximate solutions to a set of representative SIPs. ( , )
Review:
Subsequently, we develop an OED approach based entirely on quantifiable geometric properties of the maps defining the EDs. We prove efficient computable approximations to these quantities based on singular value decompositions of sampled Jacobian matrices of the proposed maps. We examine the similarities and differences among these two approaches on a set of challenging OED scenarios. A new description of the computational complexity is introduced and, under certain assumptions, shown to provide a bound on the set approximation error of an inverse image. Furthermore, a greedy algorithm is described for implementation of these OED approaches in computationally intensive modeling scenarios. Several numerical examples illustrate the various concepts throughout this thesis.
Bibliography:
Includes bibliographic resource.
System Details:
Statement of Responsibility:
by Scott N., Walsh.

## Record Information

Source Institution:
Holding Location:
Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
on10204 ( NOTIS )
1020495212 ( OCLC )
on1020495212
Classification:
LD1193.L622d W35 ( lcc )

Full Text
SIMULATION-BASED OPTIMAL EXPERIMENTAL DESIGN: THEORIES
ALGORITHMS, AND PRACTICAL CONSIDERATIONS
by
SCOTT N. WALSH
B.S., Mathematics, Cleveland State University, 2009 B.A., Physics, Cleveland State University, 2009 M.S., Applied Mathematics, University of Colorado Denver, 2016
A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Philosophy Applied Mathematics
2017

This thesis for the Doctor of Philosophy degree by Scott N. Walsh
has been approved for the Applied Mathematics Program by
Stephen Billups, Chair Troy Butler, Advisor Julien Langou Varis Carey Yail Jimmy Kim
July 29, 2017
n

Walsh, Scott N. (Ph.D., Applied Mathematics)
Simulation-Based Optimal Experimental Design: Theories, Algorithms, and Practical Considerations
Thesis directed by Assistant Professor Troy D. Butler
ABSTRACT
Several new approaches for simulation-based optimal experimental design are developed based on a measure-theoretic formulation for a stochastic inverse problem (SIP). Here, an experimental design (ED) refers to an experiment defining a particular map from the space of model inputs to the space of observable model outputs. The term optimal experimental design (OED) then refers to the choice of a map from model inputs to observable model outputs with optimal properties. We develop a consistent Bayesian approach to OED, which follows the classical Bayesian approach to OED, and determines the OED based on properties of approximate solutions to a set of representative SIPs.
Subsequently, we develop an OED approach based entirely on quantifiable geometric properties of the maps defining the EDs. We prove efficient computable approximations to these quantities based on singular value decompositions of sampled Jacobian matrices of the proposed maps. We examine the similarities and differences among these two approaches on a set of challenging OED scenarios. A new description of the computational complexity is introduced and, under certain assumptions, shown to provide a bound on the set approximation error of an inverse image. Furthermore, a greedy algorithm is described for implementation of these OED approaches in computationally intensive modeling scenarios. Several numerical examples illustrate the various concepts throughout this thesis.
The form and content of this abstract are approved. I recommend its publication.
Approved: Troy D. Butler
iii

ACKNOWLEDGMENTS
First and foremost, I would like to thank my advisor Troy Butler for his dedication and support throughout my graduate career. I am incredibly fortunate to have crossed paths with Troy during his first years as a faculty member at CU Denver. During a time when the future of my graduate career was uncertain, Troy introduced me to a fascinating held of research which encouraged me to continue to pursue this degree. He provided me with opportunity after opportunity to contribute to a fascinating held of research while also building collaborations with researchers at various institutions and laboratories. His willingness to mentor me and provide me with the skills and knowledge required to contribute to this held of research is the reason this thesis is complete today. I would also like to thank my committee members: Stephen Billups, Julien Langou, Varis Carey, and Yail Jimmy Kim. I have benehted greatly from their support and advice in the making of this thesis.
I would like to thank Lindley Graham, Steve Mattis and Clint Dawson for their guidance and support during my summer spent at the Institute for Computational and Engineering Sciences at UT Austin. The discussions we had, and perspectives that they provided, were invaluable to my success. I would like to thank Tim Wildey and John Jakeman for their thoughtful discussions during my summer spent at San-dia National Laboratories. Their open minds and willingness to entertain my ideas enabled me to grow as a researcher, both during that summer and throughout my graduate career.
There are many friends and fellow graduate students who helped me through graduate school in a variety ways. I would like to thank Abulitibu Tuguluke for spending his Saturday mornings studying for the analysis prelim with me. I would like to thank Devon Sigler for his insightful questions and discussions on the patio of The Market. Without a doubt, these conversations contributed greatly to the quality
IV

and clarity of parts of this thesis. I cannot list all of my friends and fellow graduate students who have helped me through this process, but thank you Emileigh Willems, Megan Sorenson, Michael Pilosov, Jordan Hall, Lucas Ortiz, Takao Miller, and Trever Hallock.
Last, but not least, I would like to thank my parents. My Mom, for her encouragement and support during the difficult times of graduate school. My Dad, for his time spent both helping me to overcome the challenges of the prelims and discussing my obscure analysis questions. His ability to explain challenging concepts in various ways provided me with the deep understanding required to complete the prelims and ultimately contribute to the held of applied mathematics.
v

CHAPTER
I. INTRODUCTION................................................ 1
1.1 Stochastic Inverse Problems............................... 1
1.2 Optimal Experimental Design............................... 2
1.3 Overview.................................................. 3
IP LITERATURE REVIEW........................................... 6
II. 1 Bayesian Simulation-Based Optimal Experimental Design .... 6
11.2 Thesis Objectives......................................... 8
III. A MEASURE-THEORETIC FRAMEWORK FOR FORMULATING STOCHASTIC INVERSE PROBLEMS ........................................... 10
III. 1 Formulating the Stochastic Inverse Problem Using Measure Theory 11
111.2 A Consistent Bayesian Solution to the SIP................ 18
111.3 A Measure-Theoretic Solution to the SIP.................. 22
111.4 Comparison of Methods ................................... 24
111.5 Numerical Example: Consistency........................... 27
IV. CONSISTENT BAYESIAN OPTIMAL EXPERIMENTAL DESIGN . . 34
IV. 1 The Information Content of an Experimental Design........ 34
IVY Infeasible Data ......................................... 42
IV. 3 Numerical Examples ...................................... 47
V. MEASURE-THEORETIC OPTIMAL EXPERIMENTAL DESIGN ... 53
V. l Notation................................................. 53
V.2 d-dimensional Parallelepipeds ........................... 54
V.3 Skewness and Accuracy.................................... 57
V.4 Scaling and Precision.................................... 61
V.5 A Multi-Objective Optimization Problem................... 63
V.6 Numerical Examples ...................................... 65
vi

V. 7 Summary .............................................. 70
VI. EXPECTED SCALING EFFECT VS EXPECTED INFORMATION GAIN 71
VI. 1 Technical Lemmas...................................... 72
VI. 2 Computational Considerations.......................... 74
VI. 3 Highly Sensitive Qol.................................. 76
VI.4 Unknown Profile of the Observed Density............... 79
VI.5 Redundant Data ....................................... 95
VI. 6 Summary .............................................. 97
VII. THE COMPUTATIONAL COMPLEXITY OF THE SIP................... 99
VII. 1 A Reformulation of the SIP for Epistemic Uncertainty. 100
VII.2 Single-Objective Optimization Problem................ 105
VII.3 A Bound on the Error of Set Approximations .......... Ill
VII. 4 Summary ............................................. 115
VIII. A GREEDY IMPLEMENTATION FOR SB-OED........................... 119
VIII. 1 A Greedy Algorithm................................... 119
VIII. 2 Numerical Example.................................... 120
IX. CONCLUSIONS.................................................. 129
IX. 1 Summary ............................................. 129
IX. 2 Future Work and Open Problems........................ 130
REFERENCES......................................................... 133
APPENDIX
A. RADIAL BASIS FUNCTIONS FOR DERIVIATIVE APPROXIMATIONS 139
A.l Using RBFs to Interpolate Unstructured Data.......... 139
A.2 Convergence of the Expected Relative Error........... 141
vii

FIGURES
III. 1 (left) The inverse of Q-1(Q(A)) is often set-valued even when A G A specifies a particular Q(A) G V. (middle) The representation of C, as a transverse parameterization, (right) A probability measure described as a density on (V,Bv) maps uniquely to a probability density on (Â£,Bc). Figures adopted from [13] and [10]..........................................
111.2 Illustration of the mapping Q : A > V. Note that Q maps B to a
particular region Q(B) G V, and while the inverse image of this set, given by Q~1(Q(B)), contains B, other points in A may also map to Q(B). (Figure adopted from [16])............................................
111.3 We summarize the steps in Algorithm 3. (top row) We depict Steps 1-
3. We partition A with {Vi}^=l that are implicitly defined by the set of samples {A^}^ we make the MC assumption to determine the volume of each Vj. Then we determine the nominal value qW = Q(A^). Here, N = 2000. (bottom row) We depict Steps 4-5. We partition V with a regular grid and prescribe the observed probablity measure to be constant over the darkened set of V%. Here M = 100.............................
111.4 Summary of Algorithm 3 continued, (top row) We depict Steps 5-6. For
each k, we determine which set of A^ map into Dk In red, we see the union of the Vi corresponding to the \ that map into D\ seen in red in V. We see the same for D2 in blue, (bottom row) We depict Step 8. We compute the approximate solution to the SIP where we proportion out the probability of each Dk (determined by the observed measure) to its corresponding Vi. Darker colors correspond to higher probability......
111.5 (top left) Approximation of p^ obtained using p^s on Q-1-, (top right)
a set of samples from p^, (bottom) and a comparison of the observed density p^s on with the pushforward of p^.............................

111.6 (top left) Approximation of p^ obtained using p^s on Q(top right)
a set of samples from p^\ (bottom) and a comparison of the observed density p^s on Q^ with the pushforward of p^..................... 30
111.7 (left) The approximation of p^ using the joint density p^s on Q^, (right)
a set of samples from p^........................................... 31
/ox
111.8 (top left) The pushforward of p\_ (top right) the exact observed density,
(bottom) the difference between the pushforwad of p\ and the observed density scaled by the maximum of the observed density............ 32
IV. 1 (left) Approximation of p^ obtained using p^s on Q^l\ which gives KLq(i)(pvs) ~ 0.466. (right) p^s on and the pushforward of the
prior................................................................. 37
IV.2 (left) Approximation of p^ obtained using Q^ and p^s on Q(2\ which gives KLq(2)(p\$s) 2 .015. (right) pps on Q^ and the pushforward of
the prior............................................................. 37
IV.3 (top left) The approximation of the pushforward of the prior, (top right)
The exact p^s on Q(3d (bottom) The approximation of p^ using p^s on
Q(3\ which gives KLq(3)(p^s) & 2.98................................... 39
IV.4 (left) The pushforward of the prior and observed densities on Q(right)
The pushforward of the prior and observed densities on (-2\ Notice the support of each observed density is contained within the range of the model, i.e., the observed densities are absolutely continuous with respect
to their corresponding pushforward densities.......................... 44
IV.5 (top left) The pushforward of the prior for the map Q^\ (top right) The observed density using the joint information which now extends beyond the range of the map. (bottom) The normalized observed density that does not extend beyond the range of the map.............................. 46
IX

IV.6 Smoothed plot of the EIG over the indexing set Q for the design space Q. Notice the higher values in the center of the domain and towards the top right (in the direction of the convection vector from the location of the
source), this is consistent with our intuition....................... 49
IV.7 (left) Smoothed plot of the EIG over the indexing set Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q. This location corresponds to placing thermometers near separate ends of the rod. However, we observe multiple local maxima within the design space, denoted Rl, R2, R3 and R4. (right) The EIG for the local OED in each
of the four regions.................................................... 52
V.l The ESE describes how uncertainties of a fixed size on the data space are
reduced under the inverse of the Qol map............................... 63
V.2 The ESE describes how much uncertainty on the data space is reduced to
the size of the unit cube.............................................. 64
V.3 Smoothed plot of the ESE-1 over the indexing set Q for the design space Q. Notice the higher values in the center of the domain and towards the
top right (in the direction of the convection vector from the location of the source), this is consistent with both intuition and the results from
Example IV.3..................................................... 67
V.4 (left) Smoothed plot of the ESE-1 over the indexing set Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q. This location corresponds to placing thermometers near each end of the rod. However, we observe four prominant local maxima, denoted Rl, R2, R3 and R4. (right) The ESE-1 for the local OED in each of the four regions.......................................................... 68

V.5 (left) Smoothed plot of the ESK-1 over the indexing set Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q.
This location corresponds to placing thermometers near each end of the rod. However, we observe four prominant local maxima, denoted Rl, R2,
R3 and R4. (right) The ESK-1 for the local OED in each of the four regions. 69
V. 6 (left) Smoothed plot of (dy^ip, y))_1, with oj = 0.5, over the indexing set
Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q. This location corresponds to placing thermometers near each end of the rod. Here, we observe only one prominant local maxima, (right) (dy^ij), y))-1, with oj = 0.5, for the local OED in the single region of interest......................................................... 69
VI. 1 The two groups of EDs shown in the indexing set Q x Q. The group in
the bottom right (over a red region) is the optimal group of EDs and the group near the upper right (over a blue region) is the sub-optimal group.
For reference, we show these two clusters plotted on top of the ESE-1 (computed using all 5,000 available samples in A as described above). . 77
VI.2 (left) The ESE-1 for the 20 EDs under consideration. Notice the clear difference in ESE-1 values between the optimal group (red) and the sub-optimal group (blue). The optimal group looks to be just a single line, however, there are actually 10 lines plotted here and they simply he so close together that they are indistinguishable, (right) The EIG for the 20 EDs under consideration. In this image, we do not see a clear distinction between the two groups of EDs. Again, there are 10 red lines and 10 blues lines here, they are just not distinguishable. See Table VI. 1 for a quantitative description of these EIG values.............................. 79

VI.3 The two groups of EDs shown in the indexing set Q x Q. The group in the bottom right (over a red region) is the optimal group of EDs and the other group (over a blue region) is the sub-optimal group. For reference, we show these two clusters plotted on top of the ESE-1 (computed using
N = 100 available samples in A as described above)...................
VI.4 (left) We show the ESE-1 as a function of a to emphsize that the ESE is defined independent of the expected profile of the observed density on each Qol. (right) The EIG for the 100 EDs under consideration as a function of a. The optimal group is shown in red and the sub-optimal group is shown in blue. Note that, as a increase, the EIG for the optimal cluster
becomes smaller than the EIG for the sub-optimal group...............
VI.5 The EIG over the indexing set Q x Q and the top 100 OEDs for increasing a. For very small a = IE 05, we see that the top 100 OEDs appear to be random points in Q x Q. For 0.1 < a < 1.0, the top 100 OEDs are all clustered in the bottom right of Q x Q. Then, as sigma increase beyond 1.0, the top 100 OEDs move away from the bottom right of Q x Q and
settle in two regions near the center of the space...................
VI.6 (top left) Approximation of p^ using p^s on with a =lE-07, (top right) a comparison of the observed density p^s on with the pushfor-ward of the prior on (top left) Approximation of p^ using p^s on
Q^ with a =lE-07, (top right) a comparison of the observed density p^s
on Qwith the pushforward of the prior on Q^..........................
VI.7 (top left) Approximation of p^ using p^s on with a =lE-03, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on
Q^ with a =lE-03, (top right) a comparison of the observed density p^s on Qwith the pushforward of the prior on Q^..........................

VI.8 (top left) Approximation of p^ using p^s on with a =lE-02, (top right) a comparison of the observed density p^s on with the pushfor-ward of the prior on (top left) Approximation of p^ using p^s on
with a =lE-02, (top right) a comparison of the observed density p^s
on Qwith the pushforward of the prior on Q^...................... 89
VI.9 (top left) Approximation of p^ using p^s on with a =2E-02, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on
Q^ with a =2E-02, (top right) a comparison of the observed density p^s
on Qwith the pushforward of the prior on Q^...................... 90
VI.10(top left) Approximation of p^ using ppbs on with a =3E-02, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on
Q^ with a =3E-02, (top right) a comparison of the observed density p^s
on Qwith the pushforward of the prior on Q^...................... 91
VI. 11 (top left) Approximation of p^ using p^s on with a =5E-02, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on
Q^ with a =5E-02, (top right) a comparison of the observed density p^s
on Qwith the pushforward of the prior on Q^...................... 92
VI.12(top left) Approximation of p^ using p^s on with a =1E-01, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on
Q^ with a =1E-01, (top right) a comparison of the observed density p^s on Q^ with the pushforward of the prior on Q^.................... 93

VI. 13(top left) Approximation of p^ using p^s on with a =lE+00, (top right) a comparison of the observed density p^s on with the push-forward of the prior on (top left) Approximation of p^ using p^s
on with a =lE+00, (top right) a comparison of the observed density
Pps on Qwith the pushforward of the prior on Q^....................
VI.14Smoothed plots of the ESE-1 and the EIG over the indexing set Q x Q for the design space Q..................................................
VI. 15Smoothed plots of the ESE-1 and the EIG over the indexing set Q for the
design space Q.....................................................
VII. 1 (left) In green, we show the approximate inverse image with the given
Voronoi tesselation of A. The exact inverse image is shown in blue. Notice the true parameter A*e in red is contained within the exact inverse image, however, it is not contained within the approximation, (right) In blue, we show the event E e that defines Q~1(E), the true datum in red, and the observed datum in black that is responsible for the definition of E e
VII.2 In purple we show Q~1(E), in yellow we show the inflated inverse image resulting from extending the boundaries outward, and in green we show the approximation to the inflated inverse image that covers Q~1(E). Note that, this inflation is determined by extending the boundaries outward each direction by a distance of Rn, the maximum radius of the Voronoi
cells in the tesselation...........................................
VII.3 (top left) The inverse image Q~1(E) and the approximation, (top right) The E e Bv defining the uncertainites on the observed data, (bottom right) The inflated E E determined by the Jacobian of Q and R^. (bottom left) The exact inverse image Q~1(E), the inflated inverse image Q~1(E), and the resulting approximation to Q~l(E) that covers Q~1(E).
94
95
96
101
102
104
xiv

VII.4 (left) A representative inverse image resulting from inverting E G Â£>p using the map Q^a\ (right) A representative inverse image resulting from inverting E G using the map Q^. Although produces a smaller inverse image using the same sized E, the correlation amongst the two Qol composing produces an inverse image that is more difficult to
approximate accurately with finite sampling.......................... 109
VII.5The inverse of the inflation effect, for both and Q^b\ as a function of the number of samples available. Although Mq^) < Mq(a), for N < 400, the inflation effect determines that is the OED. This is due to the increased difficulty in approximating the solution to a SIP defined using
Q(b\ For N > 400, the choice changes and is the OED.................. 110
VII.6The EIE-1 over the indexing set Q x Q for the design space Q for various numbers of samples N. In the top left, the EIE-1 looks very similar to the ESK-1 shown in Figure V.5. As we increase the number of samples, the location of the OED does not change, however, the general description of Q changes noticably. We begin to see similar features to that of the
ESE-1 seen in Figure V.4............................................. 112
VII.7The MC approximation to Error(Q~l(E)) and the outer and inner set approximation using a regular grid of samples for N = 192, 400, 784,1600
(from top to bottom)................................................. 117
VII.8The MC approximation to Error(Q~1(E)) and the outer and inner set approximation using uniform random samples for N = 192, 400, 784,1600
(from top to bottom)................................................ 118
VIII.IThe domain 0 is constructed by welding together nine square plates, each
of which has constant, but uncertain, thermal conductivity k........ 121
VIIL2The ESE-1 over the indexing set for the design space Q. Notice the OED
is shown in white, near the center of the domain where the source is located. 123
xv

VIII.^top) The ESE 1 for Q(2\ Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1. (bottom) The ESE-1 for Q^3\ Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of
the ESE-1............................................................ 125
VIII.^top) The ESE-1 for Q(4d Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1. (bottom) The ESE-1 for Q(-5\ Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of
the ESE-1............................................................ 126
VIII.Ii(top) The ESE-1 for Q^. Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1. (bottom) The ESE-1 for Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of
the ESE-1............................................................ 127
VIII.Q> top) The ESE-1 for Q(8\ Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1. (bottom) The ESE-1 for Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of
the ESE-1........................................................ 128
A.l The convergence of the expected relative error for each component of Eq. (A.l). Notice each component of V/ converges at approximately the rate O(j^)............................................................ 142
xvi

A.2 The convergence of the expected relative error for each component of Eq. (A.2). Notice each converges at approximately the rate 0(A)..........
143
xvii

CHAPTER I
INTRODUCTION
1.1 Stochastic Inverse Problems
Scientists and engineers strive to accurately model physical systems with the intent to predict future and/or unobservable quantities of interest (Qol). For example, a scientist modeling the transport of a contaminant source may be interested in predicting the average concentration of the contaminant in a region of the domain at some future time. That same scientist may also be interested in the flux of the contaminant through some boundary that is physically unobservable, i.e., cannot be determined by direct measurements. Both of these scenarios require the construction of a computational model that simulates the physical process of interest. In many applications, the extraordinary complexity of the model induces a set of model inputs that are uncertain. Typically, model inputs are described as belonging to a general set of physically plausible values using either engineering or domain-specific knowledge. Using the computational model, this set of plausible model inputs can be propagated through the model to make predictions on unobservable Qol. When the Qol are sensitive to perturbations in model inputs, the predictive capabilities of even the most sophisticated computational models may be severely limited by a coarse description of uncertainties in the model inputs.
Oftentimes, researchers look to reduce these uncertainties by using data gathered from the actual physical process simulated by the model. The idea being, by fitting the model to observable data, the probability that the model accurately predicts unobservable data is increased. This process is commonly referred to as model calibration. However, fitting or calibrating the model to observed data can be a difficult task. It is common for the observed data to be subject to uncertainty, e.g., measurement instrument error reduces the confidence one has in the exact values provided by the data. Such uncertainties are often described using probability models,
1

and specification of a probability measure on the range of values for a Qol is one approach to represent uncertainties in observable data. Then, rather than calibrating the model to a fixed set of observed data, we formulate and solve a stochastic inverse problem (SIP) in order to determine a probability measure on the space of model inputs informed by the probability measure on the Qol.
There are many frameworks for formulating SIPs based on different assumptions and models for the uncertainties in model inputs, observed data, and even the Qol maps that links these. A popular framework is based on the famous Bayes rule [62] where the Qol map is effectively replaced by a statistical likelihood function. More recently, a framework based on measure-theoretic principles has been developed and analyzed for the formulation and solution of a SIP that does not replace the Qol map for the model [16, 10, 12, 13]. The solution of a SIP formulated in the measure-theoretic framework is defined as a probability measure on the space of model inputs that is consistent with the model and data in the sense that the pushfoward of this probability measure through the Qol map exactly matches the probability measure given on the Qol. In this thesis, we adopt this measure-theoretic framework for formulating a SIP under two scenarios and consider two separate methods for the numerical solution of the SIP.
1.2 Optimal Experimental Design
The collection of data for an observable Qol can be costly and time consuming. For example, exploratory drilling can reveal valuable information about subsurface hydrocarbon reservoirs, but each well can cost upwards of tens of millions of US dollars. In such situations, we can only afford to gather data for some limited number of observable Qol. However, we may be able to define certain aspects of an experiment that define the types of observable Qol for which we collect data. For example, we may be able to choose the locations of a set of exploratory wells from some predetermined feasible domain. The choice of observable Qol for which we may collect data is
2

defined as an experimental design (ED). Thus, an ED defines a particular Qol map from model inputs to model outputs. Since the Qol map plays a significant role in the solution of a SIP, it is evident that two different EDs may lead to significantly different solutions to a SIP. For example, consider a simple contaminant transport problem in which the diffusivity properties of the domain are uncertain. We expect that some measurements in space-time are more useful than others at reducing uncertainties in the diffusion tensor.
To reduce the expense of collecting data on less useful (i.e., less informative) Qol, we consider a simulation-based approach to automatically select the useful Qol from a set of possible EDs. Simulation-based optimal experimental design (SB-OED, or sometimes just OED) looks to determine the optimal ED by way of computational simulations of the physical process. The main idea being to exploit the sensitivity information in a Qol map corresponding to an ED to determine the optimal Qol to gather data in the held. The criteria for optimality of an ED can be defined in various ways depending on the goals of the researchers. Here, we consider batch design over a discrete set of possible EDs. Batch design, also known as open-loop design, involves selecting a set of experiments concurrently such that the outcome of any experiment does not effect the selection of the other experiments. Such an approach is often necessary when one cannot wait for the results of one experiment before starting another, but is limited in terms of the number of observations we can consider. The pursuit of a computationally efficient approach for coupling the SB-OED methods developed within this thesis with continuous optimization techniques is an intriguing topic that we leave for future work.
1.3 Overview
The rest of this thesis is outlined as follows. In Chapter II, we provide a review of past and recent work in SB-OED. For completeness, in Chapter III, we review the measure-theoretic framework for formulating a SIP and two methods for numerical
3

solution of the SIP. The two methods are referred to as a consistent Bayesian approach and a measure-theoretic approach. The consistent Bayesian approach adopts the use of a prior probability density in the solution to the SIP whereas the measure-theoretic approach does not require any prior knowledge of the probability on the model inputs to solve the SIP. We provide relevant algorithms and figures depicting the implementation of each of these approaches.
We follow a Bayesian methodology for SB-OED to develop a consistent Bayesian approach to SB-OED in Chapter IV. The consistent Bayesian approach to SIPs proves useful in reducing the cost of the iterated integral defining the utility of a given ED. In Chapter V, we develop an entirely unique approach to SB-OED inspired by the set-based approximations of the measure-theoretic approach. This approach deviates from the Bayesian methodology for SB-OED in that it quantifies the utility of a given ED entirely in terms of the local geometric properties of the corresponding Qol map and does not require solution to any SIP. Thus, this approach is independent of both the methodology employed to solve the chosen SIP and the specification of a probability measure on the space of observable Qol. Consequently, under certain assumptions, these two SB-OED approaches produce similar qualitative descriptions of a space of EDs. Thus, in Chapter VI, we provide an in depth comparison between these two methods focused specifically on a handful of challenging SB-OED scenarios.
Following Chapter VI, we focus entirely on the measure-theoretic approach to SB-OED and look to resolve and improve upon the current state of the method. In Chapter VII, we focus on three major topics. First, Section VII. 1 proposes a reformulation of the SIP so that, under certain assumptions, we guarantee the true parameter1 is contained within a region of nonzero probability in the solution to the SIP. Although this approach is developed to improve reliability in a solution to a given SIP, it provides useful insight into SB-OED interests. In Section VII.2, we
1By true parameter, we are referring to an application in which the SIP fundamentally is defining a problem of parameter identification under epistemic uncertainties.
4

scalarize the multi-objective optimization problem, defined in Chapter V, implicitly as a function of the sensitivities of the Qol map, the dimension of the model input space, and the number of model solves available to ultimately approximate the solution to a chosen SIP. In Section VII. 3, we provide a bound on the error in the approximation to an inverse image which offers an alternative description of the computational complexity, i.e., condition number, of a given SIP. In Chapter VIII, we illustrate the measure-theoretic approach to SB-OED on a toy problem that yields high-dimensional parameter, data, and design spaces. We employ a greedy algorithm to abate this curse of dimensionality, thus, we approximate the OED from a combinatorial enumeration of a discretization of the design space. In Chapter IX, we provide concluding remarks and discuss future research opportunities and directions.
5

CHAPTER II
LITERATURE REVIEW
It is worth noting that the term optimal experimental design can mean a great many things depending on the context. The term experimental design (sometimes referred to as design of experiments), was likely first popularized within the statistical community, e.g., see [42, 43, 29, 41, 5] and the references therein for earlier references on the topic. In this context, an experimental design refers to some placement of samples within the predictor space. An optimal experimental design refers to an optimal placement of samples within the predictor space, where the optimality criterion may be application specific. The so-called alphabetic optimality criteria [5, 40] look to optimize properties of parameter estimates defining a model, e.g., D-optimality to maximize the differential Shannon entropy, A-optimality to minimize the average variance of parameter estimates, and G-optimality to minimize the maximum variance of model predictions. These criteria have been developed in both Bayesian and non-Bayesian settings [5, 40, 6, 25, 18, 19]. In summary, this held of experimental design research focuses on optimizing the quality of a statistical model relating the predictors to the responses.
II. 1 Bayesian Simulation-Based Optimal Experimental Design
Although developing appropriate statistical models is of utmost importance in many predictive modeling applications, we emphasize that this differs substantially from the approaches proposed in this thesis. The approaches proposed below are developed specifically for physics-based computationally expensive and potentially nonlinear models with uncertain model input parameters. Given a parameter (predictor), the computational model provides the value of the Qol (responses). Hence, we need not approximate an optimal model.
However, uncertainties in model input parameters lead to uncertainties in model predictions. If uncertainties on the parameters can be quantified and reduced using
6

experimental data, then the uncertainties in model predictions can also be quantified and reduced. We accomplish this with the formulation and solution of a SIP. Given a probability distribution on some collection of observable Qol, the solution of the SIP is a probability distribution on the space of model input parameters. The probability distribution on the observable Qol may be determined by combining available information on the experimental data and observation instrument error.
Different collections of Qol may provide information about different directions within the parameter space. Hence, in this context, an optimal experimental design (OED) refers to an optimal collection of Qol, where optimality criterion are often derived from an information theoretic foundation [46, 65, 8]. The directional information contained in a particular collection of Qol can be analyzed by numerical simulations of the model. Thus, we may be able to rank different EDs by some performance metric based entirely on simulations. We follow recent literature [35, 36, 54] and refer to this type of experimental design as simulation-based optimal experimental design (SB-OED) to emphasize the necessity of a computational model relating the uncertain parameters to the Qol.
Given a probability distribution on the chosen space of observable Qol, the Bayesian paradigm is often used to formulate and solve a SIP to determine a proba-blity distribution on the space of model input parameters [59, 62], This perspective has lead to a Bayesian approach to SB-OED, where approximate Bayesian solutions to the SIP determine the expected utility of a given ED [20, 8, 63]. The expected utility of an ED is often written as an iterated integral over both the parameter space and the space of Qol defining the ED. Hence, it is computationally expensive to approximate many useful forms of the expected utility, e.g., the commonly used expected information gain (EIG) [8] derived from the Kullback-Leibler (KL) divergence [44, 64], Therefore, much of the literature in this held is directed towards reducing computational complexity in approximating the expected utility of an ED.
7

Locally optimal techniques reduce cost but require a best guess of the value of the unknown parameter [30]. This approach is not suitable when the posterior parameter distribution is broad. In [57], Markov chain Monte Carlo (MCMC) methods and Laplacian approximations are used to compute the EIG. In [9, 21], the authors reduce computational cost by approximating the posteriors with Gaussian distributions. When the posterior differs significantly from a Gaussian profile, such an approach is not applicable. In [48, 49, 50], the authors use Laplace approximations to accelerate the computations of the EIG. In [35], the authors utilize a generalized polynomial chaos surrogate to accelerate computations of the EIG and stochastic optimization techniques to determine an OED in a high-dimensional setting. In [55, 4], the authors use MCMC and simulated annealing methods to accelerate the computation of the expected utility. In [1, 2], the authors develop an OED framework for inverse problems with infinite dimensional parameter spaces resulting from an unknown held in the physics-based model.
All of the methods described here have proved valuable in the appropriate context. However, the general formulation of Bayesian SB-OED requires the solutions, or approximate solutions, of SIPs to determine an OED. Thus, these approaches require first formulating a specific type of SIP within a particular framework, knowledge of the method to be used to solve the resulting SIP, and, finally, an estimate of the uncertainties on the observed Qol.
II.2 Thesis Objectives
The objective of the approaches described above is to approximate the expected utility of an ED based on approximate solutions to the classical Bayesian SIP [59, 62], Although this approach has proved immensely useful in the context of classical Bayesian inference, in this thesis, we focus specifically on developing SB-OED approaches inspired by consistent solutions to the SIP [16, 10, 12, 13]. The consistent Bayesian SB-OED approach, developed in Chapter IV, contributes to the
8

line of research regarding reduction in computational cost by employing the recently developed consistent Bayesian formulation and solution to the SIP [16] to efficiently integrate over the space of observed densities to approximate the EIG. Future work will explore the coupling of this method with continuous optimization techniques to determine OEDs in high-dimensional design spaces.
The measure-theoretic approach to SB-OED, developed in Chapter V, derives a computationally tractable formulation for SB-OED in which not a single SIP is solved and, therefore, is independent of both the framework used to formulate the SIP, the method used to solve the SIP, and also the estimated uncertainties on the observed Qol. An additional benefit is that the measure-theoretic method provides a quantification of the computational complexity involved in solving the final SIP using sample-based approaches.
In Chapter VI, we present a thorough comparison between these two SB-OED approaches and illustrate the strengths and weaknesses of each. Chapter VII continues to extend the measure-theoretic approach to SB-OED by both scalarizing a previously defined multi-objective optimization problem and contributing further to the quantification of the computational complexity of a SIP. In Chapter VIII, we illustrate a greedy implementation of the measure-theoretic approach to SB-OED in a high-dimensional setting.
9

CHAPTER III
A MEASURE-THEORETIC FRAMEWORK FOR FORMULATING STOCHASTIC INVERSE PROBLEMS
In this chapter, we review two recently developed methods for solving a SIP formulated within a measure-theoretic framework. In Section III. 1, we review the measure-theoretic formulation of a SIP solved by both methods. The main result is a form of the disintegration theorem that states the conditions under which a probability measure solves the SIP. Conceptually, this involves decomposing the probability measure into a marginal and family of conditional probability measures. The two methods we present for solving the SIP go about defining these conditional probabilities in fundamentally different ways that lead to different approximation procedures for the solution to the SIP.
In Section III. 2, we review the consistent Bayesian method which follows the Bayesian philosophy to define a unique solution to the SIP. The solution is defined in terms of a probability density function as is common in Bayesian approaches. In Section III.3, we present a measure-theoretic method which exploits the geometry of the Qol map to define a unique solution to the SIP. There, the solution is defined as the actual probability measure on the prescribed u-algebra.
Whereas the Bayesian approach leads to algorithms for which we can easily draw samples from the distribution using simple schemes (e.g. accept/reject), the measure-theoretic approach leads to an algorithm that is based on set (i.e. event) approximation resulting in a direct approximation of probabilities of these sets. In Section III.4, we compare these two methods. In particular, we focus on the family of conditional probabilities each method employs to define a unique solution to the SIP and the subsequent numerical methods utilized to approximate solutions resulting from each method. In Section III.5, we illustrate both the consistency property of the solution to a SIP formulated within a measure-theoretic framework and the dependence of
10

this solution on the choice of observable Qol that compose the Qol map.
III. 1 Formulating the Stochastic Inverse Problem Using Measure Theory
Here, we summarize the work of [10, 12, 13]. In order to provide a complete description of the measure-theoretic framework to the SIP, we first introduce some necessary notation and then carefully discuss a taxonomy of three forward problems, with increasing levels of uncertainty, along with their direct inverse problems. For a more thorough description and analysis of these problems and solutions, see [10, 13].
Let M(Y, A) denote a deterministic model with solution T(A) that is an implicit function of model parameters Ag Ac R. The set A represents the largest physically meaningful domain of parameter values, and, for simplicity, we assume that A is compact. In practice, modelers are often only concerned with computing a relatively small set of Qol, {Qi(Y)}f=1, where each Qi is a real-valued functional dependent on the model solution Y. Since Y is a function of parameters A, so are the Qol and we write Qi(A) to make this dependence explicit. Given a set of Qol, we define the Qol map Q(A) := (Qi(A),--- ,Q V C W1 where V := Q(A) denotes the range of the Qol map.
Level 1: Deterministic Point-Based Analysis
The first forward problem is the simplest in predictive science which is to evaluate the map Q for a fixed A G A to determine the corresponding output datum Q(A) = q G V. In other words, once the inputs of a model are specified, the simplest forward problem is to solve the model in order to predict the output datum. There is no uncertainty, save for model evaluation error, in these point values.
The corresponding inverse problem is to determine the parameter(s) A G A that produce a particular value of q G V. Oftentimes Q~1(q) defines a set of values in A either due to non-linearities in the map Q and/or the dimension of A being greater than the dimension of T>. Thus, the simplest inverse problem often has uncertainty as to the particular value of A G A that produced a fixed output datum. However, there
11

is no uncertainty about which set-valued inverse produced the fixed output datum. When n = 2 and d = 1, a set-valued inverse, Q~1(q), defines a contour in A that we are familiar with from contour maps, see the left plot in Figure III. 1. In general, we refer to the set-valued inverses of the map Q as generalized contours.
Assuming d < n and that the Jacobian of Q has full row rank everywhere in A, the generalized contours exist as piecewise-defined (n d)-dimensional manifolds in A. The assumption that the Jacobian of Q has full row rank is a practically important one since we prefer to use Qol that are sensitive to the parameters since otherwise it is difficult to infer useful information about the uncertain parameters. Moreover, we prefer Qol to exhibit these sensitivities in unique directions in A as this allows different Qol to inform about different uncertain model input parameters. We formally characterize the property that the Jacobian of Q has full row rank in the following definition.
Definition III. 1 The components of the d-dimensional vector-valued map Q(A) are geometrically distinct (GD) if the Jacobian of Q has full row rank at every point in A. When the component maps are GD, we say that the map Q is GD.
Henceforth, we assume that any map Q is GD. This implies that d < n, i.e., the number of observable Qol is no larger than the number of uncertain model parameters.
Level 2: Deterministic Set-Based Analysis
The second type of forward problem can often be written mathematically as analyzing Q(B) C V given a set B C A. In other words, there is generally uncertainty as to the precise value A takes in a set B C A and subsequently there is uncertainty as to the particular datum to predict in the set Q(B). This is a commonly studied, and well understood, topic in the mathematical and engineering sciences. For example, it is common in dynamical systems to study how sets of initial conditions evolve in time. Oftentimes we are interested in describing the size of the sets and how that
12

Figure III. 1: (left) The inverse of Q_1(Q(A)) is often set-valued even when A G A specifies a particular Q(A) G V. (middle) The representation of C, as a transverse parameterization, (right) A probability measure described as a density on (V, Bp) maps uniquely to a probability density on (C,Bc). Figures adopted from [13] and [10]-
size changes from model inputs to model outputs. Thus, we assume (A,Â£>a,^/a) and (V, Bp, fiv) are measure spaces where Â£>A and Bp are the Borel u-algebras inherited from the metric topologies on E and Ed, respectively, and the measures fj,\ and /ip are volume measures. The volume measures on A and V allow us to quantitatively assess how model inputs change to model outputs in terms of the the measures of A and Q(A), respectively.
The corresponding deterministic inverse sensitivity analysis problem is to analyze Q-1(A) for some A G Bp. While Q-1(A) G Â£>A, the practical computation of Q-1(A) is complicated by the fact that Q-1 does not map individual points in V to individual points in A. As described above, assuming d < n and Q is GD, Q-1 maps a point in V to a generalized contour described as an (n d)-dimensional manifold embedded in A. Thus, Q-1(A) defines a generalized contour event, or just contour event, belonging to an induced contour a-algebra CA C Â£>A (where the inclusion is often proper). To observe that the inclusion is often proper, observe that for any B G Â£>A,
BCQ-\Q(B)),
(III.l)
but in many cases B yt Q 1(Q(B)) even when n = cl (see Figure III.2 where n = 2
13

and cl = 1 for an illustration). In other words, solutions to the corresponding inverse problem can be described in the measurable space (A, Ca) and the volume measure can be used to provide quantitative assessments of solutions to this inverse sensitivity analysis problem.
Figure III.2: Illustration of the mapping Q : A > V. Note that Q maps B to a particular region Q(B) E V, and while the inverse image of this set, given by Q~1(Q(B)), contains B, other points in A may also map to Q(B). (Figure adopted from [16]).
It is useful to consider solutions to this deterministic inverse problem in a space of equivalence classes. Specifically, we may use the generalized contours to define an equivalence class representation of A where two points are considered equivalent if they he on the same generalized contour. We let Â£ denote the space of such equivalence classes and let ttc : A > Â£ denote the projection map where 7qc(A) = Â£ E Â£ defines the equivalence class corresponding to a particular A and ic~^l{Â£) = Ct is the generalized contour in A corresponding to the point Â£ E Â£. It is possible to explicitly represent Â£ in A by choosing a specific representative element from each equivalence class. As described in [10, 13], such a representation of Â£ can be constructed by piecewise d-dimensional manifolds that index the (n cl)-dimensional generalized contours. We refer to any such indexing manifold as a transverse parameterization (see the middle plot of Figure III. 1). Given a particular indexing manifold representing Â£, Q defines a Injection between Â£ and V. The measure space (Â£,Bc,tl>c) can
14

defined as an induced space using the bijection Q and (P, Pp, pp). Then, solutions to the deterministic inverse problem can be described and analyzed as measurable sets of points in Â£ instead of measurable generalized contour events in A.
Level 3: Stochastic Analysis
We now assume that there is uncertainty, described in terms of probabilities, as to which set A e PA the model parameters A belong, and the goal is to analyze the probabilities of sets in Pp. In the language of probability theory, measurable sets are referred to as events. Thus, we assume that a probability measure PA is given on (A, PA) describing uncertainty in the events for which parameters may belong, and the goal is to determine Pp on (V, Pp). We refer to this as the stochastic forward problem. When PA (resp., Pp) is absolutely continuous with respect to the volume measure La (resp., pp), the corresponding Radon-Nikodym derivative (i.e., the probability density function) pA (resp., pp) is usually given in place of the probability measure. We assume this is the case so that we can refer to probabilities of events in terms of the more common representation using integrals and density functions, e.g.,
Solution to this stochastic forward problem is given by the induced pushforward probability measure Pp defined for any A e Pp by
This is a familiar problem in uncertainty quantification. The approximate solution can be obtained by a classic Monte Carlo method. Assuming that Q is GD, the pushforward volume measure, by the map Q, is absolutely continuous with respect to the usual Lebesgue measure on Rd [13, 16]. Therefore, unless otherwise noted, we assume pA and pp are the usual Lebesgue measures on E and Ed, respectively.
(III.2)
(III.3)
15

The corresponding SIP assumes that we are given an observed probability measure, Pgbs, on (V,Bt>), which can be described in terms of an observed probability density, p^s, i.e., p^s is the Radon-Nikodym derivative of Pgbs with respect to pp. The SIP is then defined as determining a probability measure, Pa, described as a probability density, pa, such that the pushforward measure of Pa agrees with Pgbs on We use P^(-Pa-) to denote the pushforward of Pa through Q, i.e.,
/>yp*>(.4) = Pa(C-1(.4)).
for all A e Bv- Using this notation, a consistent solution to the SIP is defined formally as follows:
Definition III.2 (Consistency) Given a probability measure Pfibs described as a density prfis with respect to pp on (V,Bx>), the SIP seeks a probability measure Pa described as a probability density pa with respect to pa on (A, Â£>A) such that the subsequent pushforward measure induced by the map Q satisfies
Pa(Q-'(A)) = fg(PAp4) = Pg(.4), (1114)
for any A e Â£>p. We refer to any probability measure Pa that satisfies (III.4) as a consistent solution to the SIP.
To determine such a solution to the SIP, we first consider the SIP posed on (Â£, Bfi)-Since Q defines a bijection between Â£ and V, Pgbs defines a unique Pc on (Â£, Be) (see the right-hand plot in Figure III. 1 and [13]). We can then use the projection map 7tc to prove the following theorem as was done in [13].
Theorem III.3 The SIP has a unique solution on (A,C\).
However, the goal is to define a probability measure Pa on the measurable space
16

(A,Ba) not on a space involving contour events that are complicated to describe. This requires an application of the Disintegration Theorem, which allows for the rigorous description of conditional probability measures defined on sets of zero /iA-nieasure [24, 13, 32],
Theorem III.4 (The Disintegration Theorem) Assume Q : A V is BA-
rneasurable, Pa is a probability measure on (A,Ba) and Pfibs is the pushforward measure of Pa on (V,Bx>)- There exists a Pfibs-a.e. uniquely defined family of conditional probability measures {Pq}qeT> on (A, Â£>A) such that for any A E Ba,
Pq(A) = Pq(AnQ~1(q)),
so Pq(A \ Q~1(q)) = 0, and there exists the following disintegration of Pa,
P^(A)= f P,(A)dPg(q) = f l f dP,(X))dPg(q), (III.5)
Jv Jv yJAnQ-1^) J
for A E Ba Equivalently, because there is a bijection between C, and V, we may write this iterated integral in terms of C,
Pa(A)= [ ([ dPe(\))dPc(e), (IH.6)
Jc KJArmp1^) J
for A E Ba where Pc is the unique probability measure induced on C, by Pfibs.
For proofs related to the above disintegration theorem see [13, 16]. It follows from the disintegration theorem that if a probability measure Ppbs on (V, Â£>p) is given, then the specification of a family of conditional probability measures {Pq}qeT> (or, equivalently, {Pe}Â£ec) on (A, Ba) can be used to define a specific probabilty measure Pa using Eq. (III.5) (or, equivalently, Eq. (III.6)) that is consistent in the sense of Definition III.2. However, a consistent solution may not be unique, i.e., there may be
17

multiple families of conditional probability measures on (A, Ha) that define multiple consistent probability measures. A unique solution may be obtained by imposing additional constraints or structure on the SIP that leads to a unique specification of the family of conditional probability measures on (A, Ha). The methods explored in Section III.2 and Section III.3 describe how we may use either assumed or existing measure-theoretic information to define the families of conditional probabilities.
III.2 A Consistent Bayesian Solution to the SIP
Here, we summarize the work of [16].
III.2.1 Mathematical Formulation
Following the Bayesian philosophy [62], we introduce a finite prior probability measure PÂ£nor described as a probability density p^lor on (A, Ha) with respect to //a-The prior probability measure encapsulates any existing knowledge about the uncertain parameters and serves to define the necessary family of conditional probability measures on (A, Ha) to obtain a unique solution to the SIP.
The prior information can come from various sources. For example, engineering knowledge of the feasible parameter domain may inform bounds on the support of the prior density and past experimental results may inform an appropriate distribution for the prior over this feasible region. In the absence of a physically meaningful prior measure, or any previous results from which to build one, it may be possible to develop a prior using a coarse solution to the SIP using the measure-theoretic method which does not require a prior probability measure, see Section III.3. The quality of a prior probability measure and suitable methods for development of such a prior is a current topic of scientific research and debate, and this topic is beyond the scope of this exposition of the consistent Bayesian solution to the SIP.
Assuming that Q is at least measurable, then the prior probability measure on (A, Ha), HÂ£nor, and the map, Q, induce a pushforward measure pP(pnob Gn V, which
18

is defined for all A e Â£>:
v,
P^(prior) (A) = PfOI(Q-\A)). (III. 7)
Recall from classical probability theory that if P is a probability measure, B is an event of interest, and A is an arbitrary event, then
P(B) = P{B\A)P{A) + P(B\AC)P(AC).
Furthermore, if B C A, then this reduces to
P(B) = P(B\A)P(A).
(III.8)
In this context, the term P(B) represents the consistent probability measure of the event B that we seek and we let A := Q~1(Q(B)), which ensures that B C A. We are motivated to use P(A) = P-^S(Q(A)) in Eq. (III.8) since the definition of the pushforward measure implies that the consistency condition is then satisfied because Q(A) = Q(B). The remaining term in Eq. (III.8) is determined using the prior measure PÂ£nor, the classical Bayes theorem for events, and the fact that B C A,
P(B\A)
PZnor(A\B)PZnor(B)
Pprior(7l)
prr(B)
Pfor(A)'
It follows from the definition of p^(pnod pPnor(^4) = Pp<-prior-)((5(R)). Thus, we
utilize the following formal expression for Pa,
Pa(B) :
pprior /g\ PyS(Q(B)) a J p^prior\Q(B))
if PfOI(B) > 0,
0,
otherwise,
(III.9)
The following proposition and ensuing theorem highlight the major steps needed to
19

prove the existence and uniqueness of a consistent Bayesian solution to the SIP given by a probablity measure on (A, Ba)-
Proposition III.5 The probability measure given by P(/.(IIL9) defines a probability measure on (A ,CA).
We note that, although Proposition III.5 says that Pa, as defined above, defines a probability measure on the contour Theorem III.6 The probability measure Pa on (A, Pa) defined by
Pa(A)= [ ( [ dPfrior(X)\ dPfibs(q), MA e BA, (IH.10)
Jv ydanq-pg) J
where
prior / \ \
dPFnor{A) = JA K }-------diaAqW,Vq e V
q p%{pnor)(Q( A)) ,q
is a consistent solution to the SIP in the sense of Definition III.2.
We note that PÂ£nor is used only to define the family of conditional probability measures {PfmoY}qeV on (A, Ba ), needed to obtain a unique solution to the SIP, in terms of the prior probability density and the map Q. It follows that Pa in Equation (IIP 10) is given by
Pa(A)
Ppr( a)
AnQ~Hq) pv
Q(prior)
(Q(A))
dpA,qifii)
dPfihS
(q)
(iii.ii)
for all A E Ba- Furthermore, since dPffis(q) = p^sdpx>(q), we can substitute this into
20

Eq. (III. 11) to obtain the density
MA)=prr(A) Pv vbA'vl
which, when integrated in terms of the above iterated integral, induces a probability measure P\ on (A,Ba) that is a consistent solution to the SIP.
III.2.2 Numerical Approximation of Solutions
Approximating pa using the consistent Bayesian approach requires an approximation of the pushforward of the prior probability measure on the model parameters, which is fundamentally a forward propagation of uncertainty. We consider the most basic of methods, namely Monte Carlo sampling, to sample from the prior. We evaluate the computational model for each of the samples from the prior and use a standard kernel density estimator (KDE) to approximate the pushforward of the prior. Given the approximation of the pushforward of the prior, we can evaluate pa at any point A G A if we have Q(A). Thus, we can construct an approximation of pa-
We summarize the steps required to numerically approximate the pushforward of the prior in Algorithm 1, and summarize the steps required to approximate the solution to the SIP in Algorithm 2. We note that, the two prominent costs in producing a numerical approximation to pa are: (1) the evaluation of the model, and (2) the construction of an approximation to p^wlor\
Algorithm 1: Computing the Pushforward Density Induced by the Prior and the Model______________________________________________________________
1. Given a set of samples from the prior density: A^, i = 1,... N;
2. Evaluate the model at each A^ and compute the Qol: = Q(A^);
3. Use the set of Qol and a standard technique, such as KDE, to estimate
pU'W./).
III.3 A Measure-Theoretic Solution to the SIP
21

Algorithm 2: Numerical Approximation of the Consistent Bayesian Solution
1. Given a prior density on A, p^nor;
2. Given an observed density on V, p^s;
3. Given the pushforward of the prior density on V, p^(prior)
4. Given a set of samples in A: A^, i = 1,..., N;
5. Evaluate the model at each A^;
6. Compute the value of pa at each A^:
PA(A(i>)
ftA"or)(<2(A(>))
(III. 13)
7. Use Pa(A<-*-)) to construct an approximation to pa-
We follow [13, 32] and adopt what is referred to as the standard Ansatz determined by the disintegration of the volume measure pa to compute probabilities of events inside of a contour event, i.e., to define the family of probability measures on (A, Â£>A) required to obtain a unique solution to the SIP. The standard Ansatz is given by
pm
(P)
HCiiCe)
VB G BCe,
for all Â£ G C, where pcÂ£ is the disintegrated volume measure on the generalized contour Ce. Note that, this is the same as pa,9 discussed above where Cg = Q~1(q)- Also, Bce denotes the standard Borel u-algebra on the Cg.
The standard Ansatz can be used as long as pcÂ£ is finite for a.e. Â£ G C, e.g., as happens when A C Rd is compact, pa is the standard Lebesgue measure, and Q is continuously differentiable. Assuming such conditions hold to apply the standard Ansatz, the approximation method and resulting non-intrusive computational algorithm can be easily modified for almost any other Ansatz. See [13] for more details and theory regarding general choices of the Ansatz. Combining an Ansatz with the
22

disintegration theorem proves the following theorem.
Theorem III.7 The probability measure Pa on (A,Â£>a) defined by
Pa(A)= [ ( [ dPe(\))dPc(e), (in* 14)
Jc yJArm^ie) J
where
Pt{B) = VB e BCi, WeÂ£
is a consistent solution to the SIP in the sense of Definition III.2.
The standard Ansatz results in a probability measure PA that inherits key geometric features from the generalized contour map. Any other choice of Ansatz imposes some other geometric constraints on the solution to the inverse problem that are not coming from the map Q. If we choose not to use any Ansatz, then we can always solve the SIP on (A,Ca), i.e., we can always compute the unique probability measure when restricted to contour events.
III.3.1 Numerical Approximation of Solutions
Fundamental to approximating solutions PA to the SIP is the approximation of
events in the various u-algebras, Â£>p, CA and Â£>A. Since CA C Â£>A, we can simultaneously approximate events in both of these u-algebras using the same set of events partitioning A. Let {denote such a partition of A where Vi G Â£>A for each i. Assume that we are given a collection of sets {Dk}^=1 C partitioning V, see Figure III.3. The basic algorithmic procedure for approximating PA(Vi) for each i is to determine which of the {Vfif=1 approximate Q~1(Dk), and then to apply the Ansatz on this approximation of the contour event which has known probability Pv(Dk), see Figure III.4. Letting pA;i denote the approximation of PA(Vi), we can then define an approximation to PA as
N
Pa(A) Pa,n(A) = Sa(A n Vi), A G Ba,
i= 1
23

We summarize this basic procedure for approximating Pa in Algorithm 3.
Algorithm 3: Numerical Approximation of the Measure-Theoretic Solution
1. Let {Vic Â£>a partition A, often defined by a set of samples {A^}^ .
2. Compute Vi = /^(W) for i = 1,N.
3. Determine a nominal value qW for the map Q(A) on Vi for i = 1,N, often q = Q{ A).
4. Choose a partitioning of V, {Dk}^=1 C V.
5. Compute P^s(Dk) for k = 1,M.
6. Let Ck = {i\q^ E Dk} for k = 1,M.
7. Let Oi = {k\q^ E Dk} for i = 1,N.
8. Set Pa(K) = V3)Pvs(D0z) for = 1,N.
In Algorithm 3, Ck is used to determine which sets from {Vi}^=l approximate the contour event Q~1(Dk) for each k. Similarly, Oi is used to determine which contour event Q~1(Dk) is associated to Vi for each i. The Ansatz is applied in the final step where the probability of each Vi is determined by the probability P^s(D0i) multiplied by the ratio of the volume of Vi to the volume of the approximate contour event of
Q~\D0i).
We note that, the prominent costs in producing a numerical approximation to Pa are : (1) the evaluation of the model, and (2) the nearest neighbor searches employed in the various steps of Algorithm 3, sometimes referred to as binning. Depending on the assumption we make, we may reduce this cost in certain ways, i.e., given a set of i.i.d. random samples defining {Vi}f=1, we may make the MC assumption that fj,A(Vi) = //a(A)/A for each i.
III.4 Comparison of Methods
A brief discussion on the difference in how each of the above methods defines
a unique solution to the SIP is warranted. As noted, the definition of a family of
24

A
V
Figure III.3: We summarize the steps in Algorithm 3. (top row) We depict Steps 1-3. We partition A with {Vi}f=1 that are implicitly defined by the set of samples A}" we make the MC assumption to determine the volume of each Vj. Then we
determine the nominal value = Q(A^). Here, N = 2000. (bottom row) We depict Steps 4-5. We partition V with a regular grid and prescribe the observed probablity measure to be constant over the darkened set of W Here M = 100.
25

A
V
Figure III.4: Summary of Algorithm 3 continued, (top row) We depict Steps 5-6. For each k, we determine which set of A^ map into Dk. In red, we see the union of the Vi corresponding to the Ai that map into D\ seen in red in V. We see the same for D2 in blue, (bottom row) We depict Step 8. We compute the approximate solution to the SIP where we proportion out the probability of each Dk (determined by the observed measure) to its corresponding Vi. Darker colors correspond to higher probability.
26

conditional probability measures on (A, B\) defines a unique solution to the SIP. The consistent Bayesian method utilizes the data space V and prior information on the parameter space to define this family of conditional probability measures which are ultimately described in terms of their corresponding probability densities, see Eq. (III. 11). This method requires the pushforward of the prior density and results in an approximation to the value of Pa (A) at any point A G A that we have Q( A). We may then use the {pa(A(-A)}^1 to construct an approximation to pa- Alternatively, we may generate samples from pA using an accept/reject algorithm without actually constructing pA.
The measure-theoretic method utilizes the transverse parameterization C, implicitly, and defines the family of conditional probability measures on (A, B\) based on the underlying volume measure This does not require any prior information on the input parameters. This method requires various nearest neighbor searches and results in an approximation to Pa on a discretized parameter space. We may use this approximation to Pa to construct an approximation to p\ and generate samples from pa using an accept/reject algorithm. However, the samples used to discretize and approximate Pa need not come from an underlying probability measure and can be placed optimally using only information about the map Q and data space discretization, see [15].
III.5 Numerical Example: Consistency
Here, we consider an accessible nonlinear system of equations to display the consistency of the solution to the SIP. We note that, given a uniform prior, the consistent Bayesian solution is identical to the measure-theoretic solution under the standard Ansatz [16]. In other words, the two methods produce the same consistent solution to the SIP under assumption of a uniform prior. Therefore, in this example, we utilize the measure-theoretic algorithm (i.e., Algorithm 3), under the standard Ansatz, for approximating solutions to the SIP. The following consistent solutions to the SIP are
27

computed using the Butler, Estep, Tavener (BET) Python package, which is designed to solve measure-theoretic stochastic inverse and forward problems [32],
We formulate and solve three SIPs in order to both illustrate the consistency property of the solution to a given SIP and to motivate the remaining chapters. Consider the following 2-component nonlinear system of equations introduced in [10]:
Xixf + x\ = 1
x\ A2^2 = 1
The parameter ranges are given by Ai G [0.79,0.99] and A2 G [1 4.5-\/0T, 1 + 4.5-\/0T] which are chosen as in [10] to induce an interesting variation in the solution (aq(A),Â£2(A)). We consider three Qol maps defined by the solution to this system,
Q[l) =ari(A),
Q{2) = x2(X),
Q(3) = (xi(X),x2(X)).
The corresponding solutions to the SIPs are denoted, respectively, as Pa\pa^> and
(3)
Pa-
Consider and assume that the observed density on is a truncated normal distribution with mean 1.015 and standard deviation of 0.007, see Figure III.5 (bottom). We generate IE + 5 independent identically distributed (i.i.d.) samples in A, use the BET Python package to construct an approximation to P^\ and a simple accept/reject algorithm to generate a set of samples from the resulting p^. We propagate this set of samples from p^ through the model and approximate the resulting pushforward density using a KDE, see Figure III.5. The bottom image in Figure III.5 displays the consistency property of this solution to the SIP, i.e., the
28

pushforward of p^A
matches the observed density.
0.80 0.85 0.90 0.95
Al
5.6
4.8
3.2 CL>
3D
(D
1.6
0.8
Samples from A*
0.80 0.85 0.90 0.95
Ai
0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08
Xl(A)
Figure III.5: (top left) Approximation of p^ obtained using p^s on Q^, (top right) a set of samples from pA\ (bottom) and a comparison of the observed density Pps on Qd) with the pushforward of pA\
Consider Qd) and assume that the observed density on Qd) js a truncated normal
(2)
distribution with mean 0.3 and standard deviation of 0.035. We approximate PA
(2)
using the same IE + 5 samples, generate a set of samples from the resulting pA and
(2)
propagate these samples through the model to approximate the pushforward of pA ,
see Figure III.6. Again, we observe the consistency of pA in that the pushforward of
(2)
pA matches the observed density.
Next, we consider using the vector-valued Qol map Qd). We take the observed
(joint) density on Qd) to |3e the product of the (marginal) densities on Qd) and
29

(2)
0.80 0.85 0.90 0.95
Al
Samples from p?
0.1 0.2 0.3 0.4 0.5
*2 (A)
Figure III.6: (top left) Approximation of p^ obtained using p^s on Q<'2\ (top right) a set of samples from p^\ (bottom) and a comparison of the observed density Pps on Q^ with the pushforward of p^\
30

Q^ as specified above. Again, we approximate PA^ and generate a set of samples from the resulting pA using the same IE + 5 samples, see Figure III.T. With the information from both ;ri(A) and #2(A), we see a substantial decrease in the support
("S')
of pA Intuitively, with the assumptions above, we expect that
supp(p^) C supp(p^) Pi supp(p^).
(M
Figure III.7: (left) The approximation of p^ using the joint density p^s on Qd, (right) a set of samples from pA\
("3)
We propagate the set of samples from pA through the model and calculate the means and standard deviations of the resulting set of samples to construct the push-forward of pA In Figure III.8, we show the pushforward of pA the observed density, and the error between the two scaled by the maximum of the observed density to display a relative error.
In the scenario in which we gather data leading to an observed density on both
Qd) and compared to the scenarios where we only gather data on Qd) or Q(2)
but not both, there is a clear reduction of uncertainties as is evidenced by the reduced
support in pA and its overall structure. However, suppose we could only gather data
on one of these Qol in an experimental/held setting. Choosing either Qd) or q(2)
31

Pushforward of p^3)
Observed
0.5
0.4
-c
0.3 -
0.2
0.1 -
607.5
540.0
472.5 ^
=5
405.0
(U
337.5 -g
_l
270 0 T,

202.5 _H*
Q.
135.0
67.5
0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08
x\ (A)
0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08
Xl (A)
Error
xi(A)
Ho.0300
Bo.0225
Bo.0150
- 0.0075 0.0000 -0.0075 B 0.0150 B 0.0225 B -0.0300 B 0.0375
594
528
462
396
330
264
198
132
66
Figure III.8: (top left) The pushforward of p^\ (top right) the exact observed
density, (bottom) the difference between the pushforwad of p and the observed density scaled by the maximum of the observed density.
32
Pxux, (Lebesgue)

defines a separate ED. It is not immediately clear from the figures which ED is more useful.
In the remaining chapters, we seek to quantify the utility of an ED under consideration in order to determine the optimal Qol map to be used in formulating and solving a SIP. In Chapter IV, we quantify the expected utility of a given ED using the KL divergence. In Chapter V, we quantify the expected utility of an ED in terms of local geometric properties of the corresponding Qol map that describe the expected precision and accuracy of the resulting solution to a SIP. In Chapter VI, we provide an in depth comparison between these two SB-OED methods focused specifically on a handful of challenging SB-OED scenarios. In Chapter VII, we continue to extend the measure-theoretic approach to SB-OED by contributing further to the quantification of the computational complexity of a SIP. In Chapter VIII, we illustrate a greedy implementation of the measure-theoretic approach to SB-OED in a high-dimensional setting.
33

CHAPTER IV
CONSISTENT BAYESIAN OPTIMAL EXPERIMENTAL DESIGN
In this chapter, we develop a SB-OED approach based on the consistent Bayesian method for solving SIPs. We follow the Bayesian methodology for SB-OED discussed in Chapter II and [35, 36, 9]. This methodology quantifies the utility of a given ED defined by a Qol map Q based on a set of densities corresponding to solutions of SIPs for Q which, in turn, are based on the types of data that we expect to observe in the held for the given Q. Since the consistent Bayesian method for solving SIPs has different assumptions from the classical Bayesian method, we modify the classical Bayesian SB-OED methodology as needed.
In Section IV. 1, we consider the information content of a given ED and define the OED problem. In Section IV.2, we address the issue of infeasible data and discuss computational considerations. We provide numerical examples in Section IV.3 to display the efficacy of this method on two toy problems for which we have significant intuition regarding the location of the OED.
IV. 1 The Information Content of an Experimental Design
Conceptually, an ED is considered informative if the resulting solution to the SIP is significantly different from the prior probability measure. To quantify the information gain of an ED we use the KL divergence [64], which is a measure of the difference between two probability densities. While the KL divergence is by no means the only way to compare two probability densities, it does provide a reasonable measure of the information gained in the sense of Shannon information [22] and is commonly used in Bayesian OED [35, 9]. In this section, we discuss computation of the KL divergence and define an OED formulation based upon the expected information gain over a specified space of probable observed densities.
IV. 1.1 Information Gain: Kullback-Leibler Divergence
34

Suppose we are given a description of the uncertainty on the observed data for a Qol map Q in terms of a probability measure Pgbs that can be represented as a density p^s. In other words, Pgbs is absolutely continuous with respect to pp> and p^s is the Radon-Nikodym derivative of Ppbs. As presented in Chapter III, given prior information about the parameters described as a probability measure PÂ£nor, the SIP has a unique solution given by a probability measure Pa that is absolutely continuous with respect to pa, and we let pa denote the probability density. The KL divergence, i.e., information gain, of pa from p^nor, denoted KLq to emphasize the role Q plays in determining pA, is given by
KLq{pT Pa) := PA log (IV. 1)
Since p^rior is fixed for a given ED, KLq is simply a function of p\. However, from Eq. (III. 12), we see that p\ is simply a function of pps. Therefore, to make this dependence explicit, we write KLq as a function of the observed density,
KLq(j%*) := KLQ(pf^ : pA). (IV.2)
The observation that KLq is a function of only p^s is critical to defining the expected information gain on a well-defined space of observed densities, as done in Section IV. 1.2.
Example IV. 1 (Information Gain) Consider the follovjing 2-component nonlinear system of equations with two parameters discussed in Section III. 5,
A\Xi T X2 = 1, x\ A2^2 = 1.
35

Recall that the parameter ranges are given by Ai G [0.79,0.99] and A2 G [1 4.5-\/oT, 1 + 4.5v/0^I] As before, we consider three Qol maps defined by the solution to this system,
Q{1) =ari(A),
Q{2) = x2(X),
Q(3) = (xi(X),x2(X)).
The corresponding solutions to the SIPs are denoted, respectively, as pA >Pa >Pa
Consider QAl and assume the observed density cm QAl is a truncated normal distribution with mean 0.3 and standard deviation of 0.01, see Figure IV. 1. We generate 40,000 samples from the uniform prior and construct an approximation to the pushforward of the prior with Algorithm 1, see Figure IV. 1 (right). Then, we use Algorithm 2 to construct an approximation to p^A using the same 40,000 samples, see Figure IV.1 (left). Notice the support of p^A lies in a relatively large region of the parameter space. The information gain from p^A approximated using Equation (IV. 1), is KLq(i)(p))s) 0.466.
Next, we consider p^A associated with solving the SIP using QA\ We assume the observed density on QAl is a truncated normal distribution with mean 1.015 and standard deviation of 0.01. We approximate the pushforward of the prior and pA with Algorithm 1 and Algorithm 2, using the same 40,000 samples, see Figure IV.2.
Although the observed densities cm QAl and QAl have the same standard deviation, clearly p^A and p^A are different. Visually, p^A has a much larger region of support within the parameter space compared to pA This is quantified with the information gain from p^A KLq(2)(p))s) ~ 2.015. Given these two maps, QA) and QA), and the specified observed densities on each of these data spaces, the experiment leading to QAl and the corresponding pfi)s is more informative than the experiment
36

Observed / pushforward of prior
Figure IV. 1: (left) Approximation of p^ obtained using p^s on which gives ALq(i)(Pps) 0 .466. (right) p^s on Qd) anc[ the pushforward of the prior.
Observed / pushforward of prior
Figure IV.2: (left) Approximation of p^ obtained using Qd) anc[ pg>s on Q(2\ which gives K Lq(2)(p^s) ~ 2.015. (right) p^s on Qd) anc[ the pushforward of the prior.
37

leading to Q^ and the corresponding pfifs.
Next, we consider using the vector-valued Qol map We take the observed (joint) density on Q^ to be the product of the (marginal) densities on and Q^ as specified above. Again, we approximate the pushforward of the prior and pA using the same 40,000 samples, see Figure IV.3. With the joint information, we see a substantial decrease in the support of pA This is quantified in the information gain of this KLQ(3)(p\$s) 2 .98. Thus, we see the largest information gain by using Q^ and the joint density observed on Q^. However, in practice, we may be limited to specify a single Qol for which we can collect data to construct an observed density.
Based on the computed information gains, we may conclude that Q^ is more informative of the parameters than However, due to nonlinearities of the maps, it is not necessarily true that Q^ is more informative than if different observed densities are specified. Since we do not know the observed densities prior to collecting actual data, we are left to determine which of these Qol we expect to produce the most informative p\.
IV. 1.2 Expected Information Gain
OED must select an ED before experimental data becomes available. In the absence of data, we use the computational model to quantify the expected information gain (EIG) of a given ED defined by a Qol map Q. Given Q and V = Q(A), let Ov denote the space of all probability density functions over V. Defining the EIG over Ox> is computationally impossible since Ox> contains uncountably infinitely many densities. Moreover, Ox> includes densities that are unlikely to be observed in practice for a given Q. Therefore, we restrict Ox> to be a space more representative of densities that may be observed in practice for a particular ED defined by Q.
With no experimental data available to specify an observed density on V for a
given Qol map Q, we assume the observed density is a Gaussian with mean belonging
to V and a diagonal m x m covariance matrix, i.e., we assume the uncertainties in
38

Pushforward of prior
Observed
Figure IV.3: (top left) The approximation of the pushforward of the prior, (top right) The exact p^s on Q{~i\ (bottom) The approximation of p^ using p^s on Q which gives KLQ(3){p^s) 2.98.
39

each Qol are uncorrelated. For a given Qol map Q, we let Oq denote the space of all densities of this profile with mean belonging to V = Q(A),
Oq = jpp G Ot> pv ~ N(p, E), p G V. (IV.3)
Thus, px> is assumed to be a Gaussian density function with mean p G V and diagonal covariance matrix E.
We may restrict Oq in other ways as well. For example, we may expect the uncertainty in each Qol to be described by a uniform density of a given width, in which case we would define the restriction on Oq accordingly. This choice of profile for the observed densities within Oq is largely dependent on the application and specification of Q. The only limitation on the observed density space comes from the pushforward measure on the data space, p^pnof>) ag described below.
The restriction of possible p^s to this specific space of densities allows us to index each density by a unique q G V. Based on our prior knowledge of the parameters and the sensitivities of the map Q, the model informs us that some data are more likely to be observed than other data, this is seen in the plot of p^(prior) in Figure IV.3. Therefore, we average over V with respect to the pushforward of the prior. This utilizes both the prior knowledge of the parameters and the sensitivity information provided by the model. We define the EIG, denoted E(KLq), as
E(.KLq) := [ KLQ^r^'iqWv- (IV.4)
Jv
From Eq. (IV. 1), KLq itself is defined in terms of an integral. The expanded form for E(KLq) is then an iterated integral,
E(A'io) = jf MA;9)log (^^WdfÂ£ 40

where we make explicit that pa is a function of the observed density, and, by the restriction of the space of observed densities in Eq. (IV. 3), is therefore a function of q E V. We utilize Monte Carlo sampling to approximate the integral in Eq. (IV.4) as described in Algorithm 4.
Algorithm 4: Approximating the EIG of an ED

1. Given a set of samples from /ia- A^, i = 1,
2. Given a set of samples from the pushforward of the prior: q^\ j = 1,
3. Given the values of Pa(A; q^) at : i = 1,... V;
4. For j = 1,... M approximate KLq(q^):
,/r ( (7-p ^a(A) mP) (7)m ( q^)
KLq(QU)) ~ )lQS 1 pprior(A(i))
N
i= 1
M
5. Approximate E(KLq):
M

i=l
Remark IV. 1 In Step 1 of Algorithm f, we assume we are given a set of samples
from the volume measure (j,a on the parameter space. Assuming this volume measure
is the Lebesgue mesure, then given a uniform prior, this set of samples is equivalent to
a set of samples from this prior. In this thesis, we focus primarily on applications in
vjhich we have a uniform prior and, hence, use the samples from the prior in Step 1 of
Algorithm f. Given an application vnth a nonuniform prior, we have two remedies:
consider a set of samples from the posterior in Step 1 and modify the integral in
Eq. (IV. 1) to be vnth respect to Pa, or approximate Eq. (IV. 1) in terms of the values
of the observed density and pushforward of the prior density vMch allows integration
vnth respect to the prior measure on the parameter space. This topic is beyond the
focus of this thesis; the interested reader is referred to [66] for a complete discussion
41

on this topic.
Algorithm 4 appears to be a computationally expensive procedure since it requires solving M separate SIPs and, as noted in [16], approximating p^(pnor) can be expensive. However, we only need to compute this approximation once, as each KLq in Step 4 of Algorithm 4 is computed using the same prior and map Q and, therefore, the same pp('prior'>. In other words, the fact that the consistent Bayes method only requires approximating the pushforward of the prior implies that this information can be used to approximate pas for different observed densities without requiring additional model evaluations. This significantly improves the computational efficiency of the consistent Bayesian approach in the context of SB-OED.
IV. 1.3 Defining the OED
Given a deterministic model, prior information on the model parameters, a space of potential EDs, and a generic description of the uncertainties for each Qol, we define the OED as follows.
Definition IV.2 (OED) Let Q represent the design space, i.e., the space of all possible EDs, and Q E Q be a specific ED. Then, the OED is the Q E Q that maximizes the EIG,
Qopt := argmaxE(VLQ). (IV.6)
In general, the design space Q, can either be a continuous or discrete space. In this chapter, our focus is on the utilization of the consistent Bayesian methodology within the OED framework, hence, we simply fold the OED over a discrete set of candidate EDs that may come from a sampling of a continuous design space.
IV. 2 Infeasible Data
The OED procedure proposed above is based upon consistent Bayesian inference
which requires that the pushforward measure, induced by the prior and the model,
42

dominates the observed measure, that is, any event that we observe with non-zero probability will be predicted using the model and prior with non-zero probability, see Section III.2. During the process of computing E(KLq), it is possible that we violate this assumption. Specifically, depending on the mean and variance of the observational density, we may encounter pv Â£ Oq such that fv pvdpv < 1, i.e., the support of px> extends beyond the range of the map Q, see Figure IV.5 (upper right). We refer to observed data that lies outside the data space V as infeasible data. Here, we discuss potential causes of infeasible data and options for dealing with infeasible data when estimating an OED.
IV.2.1 Infeasible Data and Consistent Bayesian Inference
When inferring model parameters using consistent Bayesian inference, a common cause for infeasible data is that the model being used to estimate the OED is inadequate. That is, the deviation between the computational model and reality is large enough to prohibit the model from predicting all of the observational data. The deviation between the model prediction and the observational data is often referred to as model structure error and can often be a major source of uncertainty. This is an issue for most, if not all, inverse parameter estimation problems [38]. Recently there has been a number of attempts to quantify this error [58]. Such approaches are beyond the scope of this thesis. In the following, we assume that the model structure error does not prevent the model from predicting all the observational data.
IV.2.2 Infeasible Data and OED
When V is a proper subset of Rm, the densities in Oq given by Eq. (IV. 3) may produce infeasible data. The effect of this violation increases as p approaches the boundary of V. To remedy this violation, we must modify Oq. We choose to truncate and normalize px> Â£ Oq so that supp(pp) C V, where px> is the truncated density.
43

We define this observed density space Oq accordingly,
Oq \ pv Pv Â£ Oq, C
PvPv
(IV.7)
iv
Example IV.2 (Infeasible Data) In this example, we use the nonlinear model from Section III.5 to demonstrate that infeasible data can arise from relatively benign assumptions. Suppose the observed density on Qd) {s a truncated normal distribution with mean 0.982 and standard deviation of 0.01. In this one-dimensional data space, this observed density is absolutely continuous with respect to the pushforward of the prior on Qd); see Figure IV.4 (left). Next, suppose the observed density on Qd) {s a truncated normal distribution with mean 0.3 and standard deviation of 0.04- Again, in this one-dimensional data space, this observed density is absolutely continuous with respect to the pushforward of the prior on Q^2\ see Figure IV.4 (right). Both of these observe densities are dominated by their corresponding pushforward densities, i.e., the model can reach all of the observed data in each case.
zi(A)
Figure IV.4: (left) The pushforward of the prior and observed densities on Qd)_ (right) The pushforward of the prior and observed densities on d). Notice the support of each observed density is contained within the range of the model, i.e., the observed densities are absolutely continuous with respect to their corresponding pushforward densities.
44

However, consider the data space defined by Q^ = (aq(A),Â£2(A)) and the corresponding pushforward and observed densities on this space, see Figure IV. 5. The non-rectangular shape of the combined data space is induced by the nonlinearity in the model and the correlations between aq(A) and Â£2(A). As we see in Figure IV.5, the observed density using the product of the 1-dimensional Gaussian densities is not absolutely continuous with respect to the pushforward density on QAl, i.e., the support of pfi)s extends beyond the support of p^pnor\ Referring to Eq. (IV. 7), we normalize this observed density overV, see Figure IV. 5 (bottom). Now that the new observed density obeys the assumptions needed, we can solve the SIP as described in Section III.2.
IV.2.3 Computational Considerations
Next, we address the computational issue of normalizing pfifs over V. Prom the plot of pp('pnor') in Figure IV.5 (top left), it is clear the data space may have a complex shape and be non-convex. Hence, normalizing pfifs, as in Figure IV.5 (bottom), over V may be computationally expensive. Fortunately, the consistent Bayesian approach provides a means to avoid this expense. Note that from Eq. (III.9) we have,
-Pa (A)
pprior
rA
(A)
Pfihs(Q( A))
Hg(prior)(Q(A))
where PÂ£nor(A) = P^('pnoT\Q(A)) = 1 which implies,
(IV.8)
Pa( A) = Pfihs(Q( A)).
(IV.9)
Therefore, normalizing p^s over V is equivalent to solving the SIP and then normalizing pA (where the tilde indicates this function may not integrate to 1 over V) over A. Although A may not always be a generalized rectangle, our previous assumption (that we have a description of the prior knowledge of the parameters) implies we have a clear definition of A, and we can therefore efficiently integrate pa over A and
45

Pushforward of prior
Observed
Normalized observed
576
- 144
0.1 - - ' 72
0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08
Xl(A)
360
320
280
240
200 160 120 80 40
Figure IV.5: (top left) The pushforward of the prior for the map Q^\ (top right) The observed density using the joint information which now extends beyond the range of the map. (bottom) The normalized observed density that does not extend beyond the range of the map.
46

normalize pa by
Pa
Pa
/a Pa dp a
(IV.10)
IV.3 Numerical Examples
In this section, we consider two models of physical systems. First, we consider a stationary convection-diffusion model with a single uncertain parameter defining the magnitude of the source term. Then, we consider a time-dependent diffusion model with two uncertain parameters defining the thermal conductivities of a metal rod. In each example, we have a parameter space A, a set of possible Qol, and a specified number of Qol for which we can gather data in an experiment. This in turn defines a design space Q, and we let Q E Q represent a single ED and V = Q(A) the corresponding data space. For each ED, we let Oq denote the associated observed density space.
The following two examples have continuous design spaces. We choose to approximate the OED by selecting it from a large set of candidate designs. This approach was chosen because it is much more efficient to perform the forward propagation of uncertainty using random sampling only once and to compute all of the candidate measurements for each of these random samples. Alternatively, one could pursue a continuous optimization formulation which would require a full forward propagation of uncertainty for each new design.
Example IV.3 (Stationary Convection-Diffusion: Uncertain Source Amplitude)
Here, we consider a stationary convection-diffusion problem with a single uncertain parameter controlling the magnitude of a source term. This example serves to demonstrate that the OED formulation can give intuitive results for simple problems.
Problem setup
47

Consider a stationary convection-diffusion model on a square domain:
DV2u + V {vu) = S,
< Vu n = 0, i G C <912, (IV. 11)
u = 0, x G Vd C <912,
with
S(x) = Aexp(- 11XsTC2h2X11 ),
where = [0,l]2, u is the concentration field, the diffusion coefficient D = 0.01, the convection vector v = [1,1], and S is a Gaussian source with the following parameters: xsrc is the location, A is the amplitude, and h is the width. We impose homogeneous Neumann boundary conditions on Yn (right and top boundaries) and homogeneous Dirichlet conditions on Yd (left and bottom boundaries). For this problem, we chose xsrc = [0.5,0.5], and h = 0.05. We let A be uncertain within [50,150]. Thus, the parameter space for this problem is A = [50,150].
Given a source amplitude A, we approximate solutions to the PDE in Eq. (IV. 11) using a finite element discretization with continuous piecewise bilinear basis functions defined on a uniform (25 x 25) spatial grid.
Results
Assume that we have only one sensor to place in the spatial domain to gather a single concentration measurement. In other words, there is a bijection between 12 and Q so points (xq,x\) G 12 can be used to index the EDs. The goal is to place this single sensor in 12 to maximize the EIG about the amplitude of the source. We discretize the continuous design space Q using 2,000 uniform random points in 12, {(a^, x^)}2k=i, which corresponds to 2,000 EDs denoted by {Q(-fc')}|(L010 where Q^ = u(x%f\x^). We assume the uncertainty in any measurement for each possible Qol is described by a truncated Gaussian profile with a fixed standard deviation of 0.1. This produces 2, 000
48

observed density spaces, {(9^)}!as described in Eq. (IV.7).
We generate 5,000 samples, from the uniform prior on A and com-
pute {{Q(fc) (^W)}500}201_ yye caicuiate {E(/\ Lq(k))using Algorithm 4 and plot E(KLq) as a function of the indexing set defined by Q in Figure IV.6. Notice the EIG is greatest when the ED is defined by measurements taken near the center of the domain (near the location of the source) and in the direction of the convection vector away from the source. This result matches intuition, as we expect data gathered in regions of the domain that exhibit sensitivity to the parameters to produce high EIG values.

x0
Figure IV.6: Smoothed plot of the EIG over the indexing set for the design space Q. Notice the higher values in the center of the domain and towards the top right (in the direction of the convection vector from the location of the source), this is consistent with our intuition.
Example IV.4 (Time-Dependent Diffusion: Uncertain Diffusion Coefficients)
Here, we consider a time-dependent diffusion problem with two uncertain parameters defining the thermal conductivities of a metal rod.
Problem setup
49

Consider the time-dependent diffusion equation:
pcfifi = V (kVu) + S, x G 12, t G (t0, tf)
x G <912
(IV.12)
u(x; 0) = 0,
V
with
S(x) = A exp
where 12 = [0,1] represents an alloy rod, p = 1.5 is the density of the rod, c = 1.5 is the heat capacity, k is the thermal conductivity (which is uncertain in this problem), and S is a Gaussian source with the following source parameters: xsrc = [0.5] is the location, A = 50 is the amplitude, and w = 0.05 is the width. We let to = 0, tf = 1, and impose homogeneous Neumann boundary conditions on each end of the rod.
Suppose the rod is manufactured by welding together two rods of equal length and of similar alloy type. However, due to the manufacturing process, the actual alloy compositions may differ slightly leading to uncertain thermal conductivity properties k on the left-half and right-half of the rod, denoted by Ai and X2, respectively. Specifically, we let A = [0.01, 0.2]2.
Given the thermal conductivity values for each side of the rod, we approximate solutions to the PDE in Eq. (IV.12) using the state-of-the-art open source finite element software FEniCS [3, f7] with a 40 x 40 triangular mesh, piecewise linear finite elements, and a Crank-Nicolson time stepping scheme.
Suppose the experiments involve placing two contact thermometers along the full rod that can record two separate temperature measurements at tf. Thus, there is a
The goal is to place the two sensors in 12 to maximize the EIG about the uncertain
Results
bijection between 12 x 12 and Q so points (xq, xf) G 12 x 12 can be used to index the EDs.
50

thermal conductivities. We discretize the continuous design space Q using 20,000 uniform random points in Q x Q, {(a^,x^)}2k=\, which corresponds to 20,000 EDs denoted by , where Q^ = (u(xQk\tf),u(x(i\tf)) for each k. We assume
the uncertainty in any measurement for each possible Qol is described by a truncated Gaussian profile vnth a fixed standard deviation of 0.1. This produces observed density spaces, as described in Eq. (IV.7).
We generate 5,000 samples, {A^lf0, from the uniform prior on A and compute {{(5(fc)(A(A)}Â£010|^0i00. We calculate {E(KLQ(k))}2k=i using Algorithm f and plot E(KLq) as a function of the indexing setQxQ for Q in Figure IV.7. We note that, due to the symmetry within kl x Â£1 along the line x0 = X\, we only show the EIG over half of the indexing set Q x Q because the other half corresponds to a renaming of contact thermometer one as contact thermometer two and vice versa. Moreover, E(KLq) is clearly nonconvex and substantially more complicated than the results observed in Example IV. 3. We label the four notable local maxima of the EIG as Rl, R2, R3, and Rf. Placing the contact thermometers symmetrically away from the midpoint-near the ends of the rod (Rl)-can lead to optimal designs, vMch matches physical intuition (see the lower right corner of Figure IV.7). In the table in the right Figure IV.7, we show the local OED locations and EIG values for each of the four regions labeled in Figure IV.7. The global OED is located in Rl.
In the following chapter, we propose an OED formulation based solely on local linear approximations of Qol maps. In this way, we provide a description of the expected information gain of an ED independent of the uncertainties on the observed data. Moreover, the method described in the following chapter allows for the consideration of optimization objectives other than expected information gain. In particular, we are interested in the accuracy of the solution to a given SIP. We consider a multi-objective optimization problem in which we optimize both the expected information gain and the expected accuracy over a design space. We provide
51

EIG over indexing set R x 0
XO
Region Optimal EIG
Rl 4.909
R2 4.769
R3 4.729
R4 4.269
Figure IV. 7: (left) Smoothed plot of the EIG over the indexing set H x SI for the design space Q. Notice the high values in the bottom right corner of SI x SI. This location corresponds to placing thermometers near separate ends of the rod. However, we observe multiple local maxima within the design space, denoted Rl, R2, R3 and R4. (right) The EIG for the local OED in each of the four regions.
a thorough comparison of the two SB-OED methods in Chapter VI.
52

CHAPTER V
MEASURE-THEORETIC OPTIMAL EXPERIMENTAL DESIGN
In this chapter, we focus on two major goals: a quantification of how a map Q impacts the accuracy of the solution to a given SIP [14], and a definition of OED independent of the uncertainties on the data space. Therefore, we do not solve any SIPs during the sections leading to the definition of the OED, and we do not specify uncertainties on the data space. We consider local linear approximations of a given map Q defining an ED, and quantify two driving geometric properties of inverse images: the expected skewness and expected scaling effect [17]. The expected skewness describes the expected accuracy of the solution to a given SIP, and the expected scaling effect describes the expected precision of the solution to a given SIP. This expected scaling effect can be thought of as a type of quantification of the expected information gained by solution of a given SIP. Further comparison between the expected scaling effect and the EIG can be found in Chapter VI.
We introduce notation in Section V.l. In Section V.2, we introduce linear algebra results needed to efficiently approximate the local skewness and local scaling effect of a given Qol map. In Sections V.3 and V.4, we define and provide a means for approximating the expected skewness and the expected scaling effect, respectively. We define the OED in Section V.5 and provide several numerical examples in Section V.6. Remark V.l In [14], the computational complexity of a given SIP was characterized in terms of local linear properties of the Qol map under consideration. However, this work did not provide a computationally efficient method for approximating the expected computational complexity nor provide an optimality objective for describing the expected precision of the solution to a SIP.
V.l Notation
We let Q denote the space of all piecewise differentiable Qol maps, i.e., the space
of EDs, under consideration by the modeler for collecting data to solve a stochastic
53

inverse problem. For example, suppose the modeler is considering two different EDs that lead to Qol maps,
Q(o) : A -G V(a\ or Q(b) : A -G V(b\
Then,
Q={Q{a),Q{b)}.
For any map Q E Q, we are interested in how this map affects the local geometric properties related to the shape (specifically the skewness) and scaling of the volume measure of the inverse of an output event E E Â£>p. Conceptually, the skewness of Q describes a general increase in geometric complexity of E under Q-1, which quantitatively is related to the number of samples in A required to accurately estimate hA(Q~1(E)). The local scaling effect of E by the map Q~l describes the precision of using the map Q to identify parameters that map to E. In Sections V.3 and V.4, we show that both the local skewness of Q~l(E) and the local scaling of E by the map Q~l can be described in terms of singular values of a Jacobian of Q. We let J\tQ E E For simplicity in describing sets, we ignore any boundaries of A or T>. However, in practice, we assume that A is compact, which, by the assumed smoothness of Q, gives that V is also compact. In other words, we assume that na(A) and /rp('ZJ) are finite, which is often the case in practice by the introduction of known or assumed bounds of parameter and data values. Second, since the skewness and scaling properties of Q may vary significantly throughout A when Q is nonlinear, we must account for this variability in determining optimal Qol.
V.2 d-dimensional Parallelepipeds
Determining the size of d-dimensional parallelepipeds embedded in n-dimensional
54

space is fundamental to efficiently approximating local skewness and scaling effects of a given map Q. We are interested in two cases: d-dimensional parallelepipeds defined by the rows of a given matrix J, and d-dimensional parallelepipeds determined by the cross sections of u-dimensional cylinders that are defined by the pre-image of a d-dimensional unit cube under J. The following technical lemma and ensuing corollary describe a method for determining the sizes of these objects based on the singular values of Jacobians of Q.
Lemma V.2 Let J be a full rank d x n matrix with d < n, and Pa(J) denote the d-dimensional parallelepiped defined by the d rows of J. The Lebesgue measure pd in of Pa{J) is given by the product of the d singular values {<7fc}^=1 of J, i.e.,
d
pd(Pa(J)) = \\<7k. (V.l)
k= 1
Proof: The singular values of J are equal to the singular values of JT. Consider the reduced QR factorization of JT,
JT = QR, (V.2)
where Q is n x d and R is d x d. By the properties of the QR factorization, we know the singular values of R are the same as the singular values of JT. Let x e Rd, then
| |Qx| |2 = (Qx)t(Qx.) = xtQtQx = xTx = | |x| |2, (V.3)
so Q is an isometry. This implies the Lebesgue measure of the parallelepiped defined by the rows of R is equal to the Lebesgue measure of the parallelepiped defined by the columns of JT, or the rows of J,
d d
pd(Pa(J)) = pd(Pa(R)) = X\lk = X\<7k, (V.4)
k=1 k=1
55

where {7fc}1 of J. u
We now turn our attention to the second case of describing the size of a d-dimensional parallelepiped determined by the cross section of an n-dimensional cylinder defined by the pre-image of a d-dimensional unit cube under J. In this case, we consider the pseudo-inverse of J, J+ = JT(JJT)-1. As is evident from the formula of the pseudo-inverse, the range of J+ is equal to the row space of J. This implies that the /id-measure of the cross-section of the pre-image of a unit cube under J is equal to the the /id-measure of the parallelepiped defined by the columns of J+.
Corollary V.3 Let J be a full rank dxn matrix with d < n. Then Pa((J+)T) is a d-dimensional parallelepiped defining a cross-section of the pre-image of a d-dimensional unit cube under J and its Lebesgue measure pd is given by the inverse of the product of the d singular values {cp;}fc=1 of J, i.e.,
w(P Proof: Consider the pseudo-inverse of J
J+ = JT(JJT)-\
From this equation, it is clear that the column space of J+ is equal to the row space of J. The row space of J defines a subspace orthogonal to the n-dimensional cylinder that is the pre-image of a unit cube under J. Therefore, the column space of J+ is orthogonal to the pre-image cylinder and Pa((J+)T) is a d-dimensional parallelepiped defining an orthogonal cross-section of this cylinder.
From basic results in linear algebra, the singular values of (J+)T are equal to those of J+. Then, from properties of the pseudo-inverse, the singular values of J+
56

are the inverse of the singular values of J. Finally, from Lemma V.2, it follows that
-l
m(Pa((J+)T)) = I U <7*
(V.6)
where {<7k}
d
k= 1
are the singular values of J.
V.3 Skewness and Accuracy
In [14], it was shown that the number of samples defining regular grids (and thus also uniform i.i.d. sets of samples) in A required to obtain accurate approximations in
Below, we define the local skewness of a Qol map, and provide a means for computing the expected skewness.
First, assume that Q is linear and E Â£ a generalized rectangle. When d = n, Q~1(E) is a d-dimensional parallelepiped in A (ignoring any affects from possible intersections with the boundary of A) that is in 1-to-l correspondence with E. If d < n, then Q~1(E) is given by a n-dimensional cylinder in A, with cross-sections given by d-dimensional parallelepipeds, where each such parallelepiped is in 1-to-l correspondence with E.
A fundamental decomposition result [23] from geometry gives Theorem V.4 Let J be a full rank d x n matrix with d < n, and Pa(J) denote the d-dimensional parallelepiped defined by the d row vectors {ji, jd} C E of J. There exists G E such that
hA-measure of Q 1(E) is proportional to the maximum local skewness of the map Q.
ji=ji'+ji, Ji'-Lji, j? G span{j2,- ,jd}
and
Hd(Pa(J)) = (j^l x nd-i(Pa(Ji))
where
57

J\ denotes the submatrix of J with the first row removed,
Pa(Ji) is the (d l)-dimensional parallelepiped defined by the d 1 row vectors of J\, and
pd and pd-i represent the d- and (d l)-dimensional Lebesgue measures, respectively.
We now relate this fundamental decomposition result to inverse sets, where, for a fixed Q E Q, we let Jk,\ denote the submatrix of J\ with the kVa row removed. For simplicity, we assume that Q is linear and 1 < d < n. We then have that the Jacobian J = J\ is independent of A, and Q = J A. Consider the map Q = J^A, which maps A to a (d 1)-dimensional data space V. Let P denote the projection matrix from V to
V. Suppose we observe the event E E Bv using Q, which corresponds to observing the event E = PE if we were to use Q. We then have that Q~l(E) is a cylinder with cross sections given by d-dimensional parallelepipeds, and Q~1(E) is a cylinder with cross sections given by (d 1)-dimensional parallelepipeds. By construction, Q~l(E) C Q~1(E) and the (d 1)-dimensional parallelepiped cross sections of the cylinder described by Q~l(E) are faces of the d-dimensional parallelepipeds defining the cross sections of Q~1(E). In other words, the use of all d component maps truncates the cylinder given by Q~l(E) in a particular direction. The length of the truncation can be quantified by |j^|, which conceptually corresponds to the amount of additional, or unique, information contained in the kVa Qol component used to define Q compared to all the other components of Q. With this in mind, we define the local skewness of a map Q G Q, which was originally introduced in [14].
Definition V.5 For any Q E Q, X E A, and a specified row vector j*, of the Jacobian J\,q, we define
.Sg(V.Q.j) := (V.7)
Ufc I
58

Then, we define the local skewness of the map Q G Q at a point A as
Sq{ A)
max
Kk SQ(J\,Q,jk)-
(V.8)
Conceptually, Sq(J\, jfc) describes the amount of redundant information present in the fcth component of the Qol map compared to what is present in the other d 1 components when inverting near the point A G A. The smallest value Sq(A) can be is one. There is no largest value since there exists maps Q that have GD component maps, but the condition of the Jacobian may be arbitrarily large. If the Jacobian were to ever fail to be full rank, then Sq(A) would be infinite. However, the assumption of GD Qol prevents this from occurring.
The fundamental decomposition of Theorem V.4 along with Lemma V.2 provides a convenient method for determining the skewness in terms of the d-dimensional parallelepipeds described by Q.
Corollary V.6 For any Q G Q, Sq(A) can be completely determined by the the norms of n-dimensional vectors and products of singular values of the Jacobian of Qol maps of dimensions d 1 and d,
Sq( A)
max
1 \jk\hd-i(Pa(Jk,\))
Hd{Pa{J\))
(V.9)
Proof:
Sq( A)
max SQ(Jx,jk)
Kk I jfc
max r
1 max
1 \jk\hd-i(Pa(Jk,\))
Hd{Pa{J\))
(V.10)
then applying Lemma V.2 we have
max
1 \jk\hd-i(Pa(Jk,x))
Hd(Pa(J\))
max
1 nd 1
r=\ &k,r
Yidr=l r
(V.ll)
59

where {<7r}i of Jk,a-
Corollary V.6 implies that we can exploit efficient singular value decompositions to algorithmically approximate Sq(A) at any point A G A. Since Sq(A) may vary substantially over A, we must quantify this variability in order to optimally choose Q G Q. Although we have assumed that Q G Q is GD, we realize that, in practice, this may not be true. In order to accommodate this scenario, and to reduce the impact of outliers, we define the expected value of Sq inspired by the harmonic mean of a finite set of positive numbers, i.e., if A is a random variable and /(A) > 0, then the harmonic mean of / is given by
This motivates the following
Definition V.7 For any Q G Q, we define the expected skewness (ESK) as
H(SQ)={]IIwIAs^(xjtW) ' Generally, we approximate H(Sq) using Monte Carlo techniques to generate a set of independent identically distributed (i.i.d.) samples {A^}^ C A and compute
Remark V.8 Notice the descriptions of skewness are independent of the generalized rectangle E G Â£>p. In other words, skewness is a property inherent to the map Q itself and describes the way in which the geometry of E G is changed by applying Q~l.
V.4 Scaling and Precision
60

To motivate what follows, consider the simple problem where we must choose between two different EDs leading to two distinct Qol maps and so that Q = Let and represent the set of all probable observations
from using either Qor Q(b\ respectively. Then, depending on which experiment we observe, we would conclude that either the parameters belong to (Q1(E^) or 1(Ealmost surely. Suppose
W))fA((Qm)~l(E,t>)),
then we generally expect that ensembles of parameter samples generated from results based on the experiment leading to will have smaller variance (which is a description of precision in statistical terms) than those based on the experiment leading to Q^. This motivates a general measure-theoretic goal for designing experiments where events of high probability on a data space are made small in volume on the parameter space by inverting the Qol map.
We begin with a simplifying assumption that Q E Q is linear with GD component maps. Then, there exists a dxn matrix J, such that Q(A) = JA. If d = n and A = E, it is easily shown from standard results in measure theory and linear algebra that
Aa(Q~l(E)) = ME) det(J1) = iiv{E) (j! ^ j , (V.14)
where are the singular values of J. Note that if A C R" is proper, then
the above equation is not necessarily true as Q~1(E) may intersect the boundary of
A. We neglect such boundary effects in the computations, and simply note that in
certain cases they may play an important role although this is not the typical case
in our experience. If d < n, then recall that Q~l(E) is defined by a cylinder in A
with cross sections given by d-dimensional parallelepipeds. We use Corollary V.3 to
compute the measure of these d-dimensional parallelepipeds. As with the expected
61

skewness of Q, in order to reduce the impact of outliers, we define the expected value of Mq by the harmonic mean of a random variable. This motivates the following Definition V.9 For any Q E Q and A G A, we define the local {/aA-measure) scaling effect of Q as
mQ{a)=(n a^j > (y-15)
where { H(Mq)
1
1
-l
Ta(A) JA Mq(A)
djiA
(V.16)
As with the expected skewness of Q, we generally approximate H(Mq) using a set of i.i.d. random samples {A^}^ C A and computing
H(MQ) HN(MQ) = (i
We summarize the above results into the following
Corollary V.10 For any Q E Q, the local skewness, local scaling effect, average skewness, and average scaling effect of Q can be computed using norms of row-vectors and singular values of the Jacobian J\,q.
Remark V.ll An alternative to H(Mq) that accounts for possibly different pv(E) is to use simple multiplication of pv(E) as suggested by Eq. (V.14).
The geometric interpretation of the ESE offered above describes how the size of
sets are scaled as they are inverted with the map Q. That is, given a Qol map with
an ESE of 10, we expect the cross section of the inverse image of an arbitrary set
E E Bx> to increase in size by a factor of 10. Likewise, given a Qol map with an
ESE value of 0.1, we expect the cross section of the inverse image of an arbitray set
62

E G Bx> to decrease in size by a factor of fO. An ESE value of 1 suggests the size of the cross section of the inverse image of E G Â£>p will remain unchanged from its size in V. This is illustrated in the schematic in Figure V.l.
We offer an alternative geometric interpretation of the ESE that shows clearly how the ESE is related to the reduction of uncertainty, i.e., the precision of solving a SIP with a particular Qol map. Consider two Qol maps QC) and Qd>) gUCp that the ESE of is 0.1 and the ESE of is 0.01. We expect (Q(a))_1 maps E G with fix>(E) = 10 to a set the size of a unit cube in A, and we expect (Q^)-1 maps E G Bx> with Ht>(E) = 100 to a set the size of a unit cube in A. That is, we expect the map (Q(6))-1 to map larger sets in T> into a region the size of the unit cube in A. This is illustrated in the schematic in Figure V.2.
D
(a)
(q (a)r
(q (b)r
A
Figure V.l: The ESE describes how uncertainties of a fixed size on the data space are reduced under the inverse of the Qol map.
V.5 A Multi-Objective Optimization Problem
We let S C R denote the set of all possible values of H(Sq) and M. C R denote the set of all possible values of H(Mq) for all Q belonging to a specified Q. Clearly, S is bounded below by 1, which represents the case of optimal global skewness in a Qol map Q. Similarly, M. is bounded below by 0, which represents the case where
63

A
Figure V.2: The ESE describes how much uncertainty on the data space is reduced to the size of the unit cube.
a set of output data exactly identifies a particular generalized contour of parameters responsible for the data.
For Q, we can clearly order all Q G Q according to either the values of H(Sq) or H(Mq). In other words, the values of H(Sq) and H(Mq) separately describe an ordering index on the space of all possible Qol given by Q. However, the mapping Q i y (H(Sq), H(Mq)) describes a double indexing with no natural ordering. We define metrics in order to quantify the distance to the optimal point of (1,0) in the Cartesian product space described by the pairs (H(Sq), H(Mq)). While there are many options for defining metrics, a thorough investigation on the effect of different metrics is beyond the scope of this thesis which is focused on the exposition of this general approach. Below, we choose a particular form for the metrics on S and M. that minimizes the effect of the possibly disparate ranges of values we may observe in S and M. on the solution to the multi-objective optimization problem.
We define (S,ds) and (M.,(Im) using the metrics
ds(x, y) = ^ - for all x, y G S, (V.18)
f t h y\
64

and
(V.19)
1 + \x y\
Choose to G (0,1) and let denote the Cartesian product space S x A4, with metric defined by
Note that uj determines the relative importance we place on either precision or accuracy. Choosing uj = 0 implies we disregard skewness in the objective whereas choosing uj = 1 implies we disregard the scaling effect.
Definition V.12 Given Q and uj G (0,1), the OED problem is defined by the multi-objective optimization problem
where p = (1, 0) is the ideal point and y = (H(Sq), for Q G Q.
V.6 Numerical Examples
In this section, we again consider two models of physical systems originally shown in Chapter IV. First, we consider a stationary convection-diffusion model with a single uncertain parameter defining the magnitude of the source term. In this example, we consider choosing a single Qol to define an OED. Hence, we optimize only for the ESE, as skewness is not defined for one-dimensional data spaces. Next, we consider a time-dependent diffusion model with two uncertain parameters defining the thermal conductivities of a metal rod.
In Definition V.12, we define the OED in terms of a minimization problem. In this section, we consider maximizing the inverse of the optimization objective for two reasons: (1) to compare results to those in Chapter IV in a more obvious way, and,
dY(x,y) = uds(xi,yi) + (1 uj)dM(x2,y2) for all x,y eYu
(V.20)
min dY^(p,y)
(V.21)
65

(2) by considering the inverse of (p, y) (and the ESE and ESK) we accentuate the smaller values of the optimization objectives in such a way that makes the OED more obvious in plots over indexing spaces.
Example V.l (Stationary Convection-Diffusion: Uncertain Source Amplitude)
Here, we consider a stationary convection-diffusion problem with a single uncertain parameter defining the magnitude of a source term. This example serves to demonstrate that the OED formulation gives intuitive results for simple problems. See Example IV. 3 for a description of the model and problem setup.
Results
We discretize the continuous design space Q from Example IV. 3 using 2,000 uniform random points in which corresponds to 2,000 EDs denoted
by {}Â£
We generate 5,000 uniform samples in A and compute { ) }f= }^ We
approximate { {} fc using a radial basis function (RBF) interpolation method. Essentially, this method constructs a local surrogate of about and differentiates the surrogate to obtain an approximation to J\a) qw, see Appendix A for details. We calculate {HN(MQ(k))}fVf using Eq. (V.17) and plot the ESE~l as a function of the indexing set defined by Q in Figure V.3. We see results similar to those in Example IV. 3, i.e., we observe higher values near the center of the domain (near the location of the source) and in the direction of the convection vector away from the source.
Example V.2 (Time Dependent Diffusion: Uncertain Diffusion Coefficients)
Here, we consider a time-dependent diffusion problem with two uncertain parameters defining the thermal conductivities of a metal rod. We choose the data spaces under consideration to be two-dimensions as well. We consider optimizing the ESE,
ESK, and solving a multi-objective optimization problem. See Example IV.4 for a description of the model and problem setup.
66

Figure V.3: Smoothed plot of the ESE-1 over the indexing set Q for the design space Q. Notice the higher values in the center of the domain and towards the top right (in the direction of the convection vector from the location of the source), this is consistent with both intuition and the results from Example IV.3.
Results
We discretize the continuous design space Q from Example IV. 3 using 20,000 uniform random points in QxQ, {(^\ ^)}|, which corresponds to 20,000 EDs
denoted by where Q(fc) = (u(x<^\tf),u(x^\tf)) for each k. We gen-
erate 5,000 uniform samples, {Ah)}0, in A and compute {(Ah'A)}^0-We approximate {{VA(i) qcao j^.00^0 using an RBF interpolation method, see Appendix A for details. We calculate and {^v(5,Q(fc))}|{)0 using
Equations (V.17) and (V.13), respectively.
In Figures V.f-V.6, we show the ESE~l, ESK~l, and (chg(p,y)) 1 with uj = 0.5 over the indexing set QxQ for the design space Q. We note that, due to the symmetry within Q x Q along the line xo = x\, we only show each optimization objective over half of the indexing set Q x Q because the other half corresponds to a renaming of contact thermometer one as contact thermometer two and vice versa. Each of the optimization objectives produce nonconvex functions over the indexing set and are substantially more complicated than the results observed in Example V.l. For each objective, we
67

label the notable local maxima within Q xfi, For the ESE~l and ESK~l, we observe four notable local maxima, and for dff we observe one notable local maxima. However, in each case, placing the contact thermometers symmetrically away from the midpoint (i.e., near the ends of the rod (Rl)) can lead to optimal designs, which matches physical intuition (see the lower right corner of the images in Figures V.f-V.6). The tables in the right of Figures V.f V.6 show the local OED optimization objective values for each of the regions labeled in their corresponding plots over the indexing set flxO.
In Figure V.f, the ESE~l for the local OEDs in R2 and R3 are relatively similar. This is expected, as the symmetry of the physical domain and the location of the source suggest that these two ED locations would yield a similar reduction in uncertainties of the model input parameters. The ESE~l for these two local OED are not exactly the same due to various sources of approximation errors: the discretization of the domain, the sampling of the A, and the sampling ofQxQ.
634.5
564.0 Region Optimal (ESE) 1
493.5 Rl 6.54 E + 02
423.0 R2 7.00F + 02
352.5 ST R3 6.45F + 02
K) 282.0 R4 6.06E + 02

211.5
141.0
170.5 0.0
Figure V.4: (left) Smoothed plot of the ESE-1 over the indexing set Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q. This location corresponds to placing thermometers near each end of the rod. However, we observe four prominant local maxima, denoted Rl, R2, R3 and R4. (right) The ESE-1 for the local OED in each of the four regions.
68

Region Optimal (ESK) 1
Rl 9.95E-01
R2 9.42E 01
R3 9.611! 01
R4 6.131! 01
Figure V.5: (left) Smoothed plot of the ESK-1 over the indexing set Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q. This location corresponds to placing thermometers near each end of the rod. However, we observe four prominant local maxima, denoted Rl, R2, R3 and R4. (right) The ESK-1 for the local OED in each of the four regions.
Figure V.6: (left) Smoothed plot of ( 69

V.7 Summary
In this chapter, we developed a SB-OED approach based entirely on local linear approximations of Qol maps. This approach optimizes two geometric properties of inverse images which are related to the accuracy and precision of the solution to a given SIP. The novelty of this approach lies within two key features: (1) the approach defines an OED independent of the expected (and often unknown) uncertainties on the observed data, and, (2) provides a quantification of the expected computational limitations in accurately approximating solutions to a given SIP. In the following chapter, we make direct comparisons between this approach and the consistent Bayesian approach to SB-OED developed in Chapter IV.
The multi-objective optimization problem defined in Eq. (V.21) is somewhat unsatisfying. The metrics defined on S and M. are chosen to reduce the chances that one optimization objective dominates the decision because the values of that objective are larger. Then uj is chosen to determine a weighting of the priority of optimizing each objective. Determining an optimal oj to satisfy the needs of a specific problem is an open question. In Chapter VII, we address this question, in a somewhat unconventional way, to define a single objective OED problem that is implicitly a function of the ESE, the number of model solves available to approximate the solution to a given SIP, and the dimension of the parameter space.
70

CHAPTER VI
EXPECTED SCALING EFFECT VS EXPECTED INFORMATION
GAIN
In this chapter, we provide an in-depth exploration and comparison of the two previously developed SB-OED approaches: the consistent Bayesian approach, described in Chapter IV, and the measure-theoretic approach, described in Chapter V. The consistent Bayesian approach is based on the classical Bayesian methodology for SB-OED. This approach quantifies the utility of a given ED using the EIG, which is defined in terms of solutions to a set of probable SIPs of interest. Thus, this approach requires knowledge of the profile of the observed density on each Qol in order to solve the SIP and then approximate the OED.
The measure-theoretic approach quantifies the utility of a given ED in terms of the ESE1, which describes how measurable sets in the data space are scaled as they are inverted into the parameter space by a given Qol map. This approach is based entirely on local linear approximations of the Qol map under consideration. Thus, computation of the ESE requires gradient approximations. However, this approach does not require prior knowledge of the profile of the observed density on each Qol. The generality of this approach, based on properties of a given Qol map not on properties of solutions to a set of SIPs, proves valuable in robustly approximating OEDs with limited numbers of model solves available and unknown profiles of the observed density on each Qol.
In Section VI. 1, we introduce two technical lemmas used in the analysis of the results presented in this chapter. In Section VI.2, we discuss computational considerations for each approach. We focus on the computational cost as a function of the number of model solves. In Section VI.3, we provide an example to illustrate the ability of each method to distinguish between highly sensitive Qol given a lim-
1The measure-theoretic approach also considers the ESK, however, for simplicity, we focus only on the ESE in this chapter.
71

ited number of model solves. In the context of OED, we often must prescribe an expected utility systematically to a broad set of EDs. This set of EDs may include EDs where the profile of the observed density is large relative to the range of the data space. Moreover, it may include EDs composed of Qol providing redundant data, i.e., the components may fail to be linearly independent in regions of, or throughout, the parameter space. In Section VI.4, we consider the stability of each OED method with respect to varying profiles of the observed density, with a specific focus on observed densities whose support extends well beyond the range of the data space. In Section VI.5, we explore the ability of each OED method to distinguish optimal properties amongst EDs defined by Qol maps with Jacobians that are not locally, or globally, full rank in A. In Section VI.6, we provide summarizing remarks for the chapter.
VI. 1 Technical Lemmas
In this section, we introduce two technical lemmas used later in this chapter. The first lemma states that, under a certain assumption, the MC approximation to the KL-divergence is bounded from above by log(N), where N is the number of samples used in the MC approximation. The second lemma states that the KL-divergence of pA from Pa lor is equal to the KL-divergence of ppbs from p^(pnor).
Lemma VI. 1 Suppose Q : A > V is BA-measurable, pv^0T is uniform ouer A, and ho: x is the set of i.i.d. samples (with respect to pf[WT) in A used to approximate both pa and KLQ(pf[lor : pa). Then the MC approximation to KLQ(pf[lor : pa), denoted KLq(pPj[wt : pA), is bounded by log(V),
klq(p[
^prior
Pa) < log(iV).
Proof: The uniform prior is given by
(VI. 1)
72

We observe that KLq(pvy : p\) is maximum when there is only one A^ G supp(pA). In this case, using the standard MC approximation that Pa(W) ~ fj,\(A)/N implies that the computed p\ is given by
N
A G Vi.
Pa(A)=^a(A)
0, A Vi
where Vi is the Voronoi cell associated with A^ G supp(pa). Then,
KL
Q
(PT = 1 (^3j
i= 1
ua(A) N (N/h a (A)
< -------- log
N Pa(A) lg (A)
1/Pa(A)
(VI.2)
(VI.3)
(VI.4)
where Eq. (VI.3) comes from the MC estimate of the integral and Eq. (VI.4) results from the substitution of p^rior and p\ above into Eq. (VI.3).
Lemma VI.2 Assume Q : A > V is BA-measurable. Then, the KL-diuergence of Pa from pf[lor is equal to the KL-diuergence of p^s from p^fVTl0T\
KLQ(f,rp0T : pA) = KLcipf' : pp).
73

Proof: Using the definition of the KL-divergence given by Eq. (IV. 1), we have
KLQ(f/r-.PA)= I A(A)iog(-Â£hh
> A \Pa (a)
\dpA{ A)
f oriorfM P\$S(QW) ,/ p\$s(QW) m
LPa (A)pg<-'w(A))log Lr>(A))1 ^A(}
f A)) Y pg"(Q(A)) Y.pnno,,',
Lp^\Q(\)) S \9g([,"or,(Q(A))/ A
*\$()
pg()
, 5*07Tlog I I (4)
IV Pv Vi) \Pv Vi)
[ Pvr"c>(g)Pv('i)
k pg(p")(9)
log
P\$s(q) pTm(Q)
I dpv(q)
1ST /VKprior) obs\
K Lq ypv Pv )i
where we use the definition of pA given in Eq. (III. 12) and the fact that
f(Q(X)) dPr(X) = I f(q) dFf^ (q)
>V
VI.2 Computational Considerations
In this section, we discuss the computational requirement of each of the OED
approaches in the context of several relevant computational modeling scenarios. The
various scenarios do not largely impact our ability to compute approximations to
OEDs using the consistent Bayesian approach. The consistent Bayesian approach
requires approximate solutions to SIPs, and this simply necessitates a forward pro-
gagation of uncertainty through the computational model in order to compute the
pushforward of the prior density. As discussed previously, we employ standard KDE
techniques to construct a non-parametric approximation of the pushforward of the
74

prior density2. This requires a set of samples drawn from the prior and the solution of the model for each sample. Thus, for implementation of the consistent Bayesian approach to OED, we simply require a set of samples from the prior and corresponding data from the model solutions. However, accuracy of the approximations to the solutions of each SIP, and therefore the accuracy of the OED, may suffer given a small number of samples from the prior and/or high-dimensional spaces.
The measure-theoretic approach to OED requires gradient information at a set of samples in A for each ED under consideration. There are several common methods for obtaining gradient approximations. First, it is often the case that the Qol defining the ED are defined by linear functionals of a solution to a computationally expensive model. Therefore, solving the corresponding adjoint problems for each componenet of the Qol map is an affordable method for approximating Jacobians [33, 56, 60, 69, 53]. The solution of the adjoint problems produces a computable Jacobian approximation at a cost of approximately one extra model solve per component of the Qol map. That is, for each sample in A and each component of the Qol map, we require solution to the forward problem and the adjoint problem (which has similar computational cost) to approximate the gradient of each component of the Qol map at a point in A.
However, many computational models may not have adjoints available [51]. In this scenario, we are left to approximate gradients using other techniques, e.g., standard finite-difference techniques. This may be computationally intractable given a high-dimensional parameter space as these methods require at least dim(A) +1 model solves to obtain a gradient approximation at a single point in A. Moreover, we may not be given the freedom to place samples optimally to employ standard finite-difference techniques. For example, we may be given a set of N samples in A, e.g., coming from sampling a prior, and must approximate gradient information in order
2We note that, it is well known that these KDE techniques do not scale well with the dimension of the data space [67].
75

to determine an OED3. In this scenario, we choose to utilize radial basis functions (RBFs) to approximate local response surfaces [70, 26]. Essentially, these methods can operate on an unstructured set of samples to construct a local surrogate about a given A G A and differentiate the surrogate to obtain an approximation to the gradient, see Appendix A for details. This is, in many ways, the most challenging scenario for implementation of the measure-theoretic approach to OED. Hence, in the ensuing examples, we assume we do not have an adjoint and that we have N uniform random samples in A, with corresponding data, for which we can use to determine an OED.
VI.3 Highly Sensitive Qol
In this section, we explore the ability of each OED method to distinguish between EDs composed of Qol that are highly sensitive to changes in parameters. We begin with an example to illustrate our findings, and follow with an explanation.
Remark VI.3 We note that, in Example VI. 1 below, the profile of the observed density on each Qol differs substantially from that chosen for the same model in Example IV.4- This choice is made to highlight a specific limitation of the Bayesian approach to OED, specifically, that the method has difficulty distinguishing between two highly sensitive EDs when limited model solves are available.
Example VI. 1 (Time-Dependent Diffusion: Limited Model Solves) We consider a time-dependent diffusion problem with two uncertain parameters defining the thermal conductivities of a metal rod. See Example IV.4 for a complete description of the model and problem setup.
Recall, the experiments involve placing two contact thermometers along the full rod that can record two separate temperature measurements at tf. Thus, there is a bijection between Q x Q and Q so points (xq,x\) G Q x Q can be used to index the EDs. For the consistent Bayesian approach, we let the uncertainty in each Qol be
3Or, even worse, to use these N samples to both determine an OED and then solve the SIP with the same set of N samples.
76

described by a truncated Gaussian profile with a fixed standard deviation of IE 04. This choice of uncertainties suggests the measurement instrument has a high degree of accuracy relative to the range of the data that we expect to observe based on the prior information, i.e., p.p(supp(p^s)) pv(T>).
Here, we focus on the ability of each OED approach to correctly classify two specific groups of EDs: one optimal group and one sub-optimal group. In particular, we are interested in the abilities of each method as a function of the number of model solves available. We choose the optimal group of EDs to be 10 points chosen randomly from a square with side length 0.04 centered at the indexing point (0.97, 0.03) G OxQ, and the sub-optimal group to be 10 points chosen randomly from a square with side length 0.04 centered at the indexing point (0.9, 0.7) GfixH, see Figure VI.1.

634.5 ^ 564.0 -493.5 -423.0
I
- 352.5 S'
to
K)
- 282.0 w
U 211.5
141.0
70.5
0.0
Figure VI. 1: The two groups of EDs shown in the indexing set QxQ. The group in the bottom right (over a red region) is the optimal group of EDs and the group near the upper right (over a blue region) is the sub-optimal group. For reference, we show these two clusters plotted on top of the ESE-1 (computed using all 5,000 available samples in A as described above).
Results
77

For both OED methods, we consider approximating the ESE and EIG values for increasing numbers of model solves (up to N = 5000,), where each set of samples is drawn from the uniform prior. We assume that we do not have an adjoint problem available to approximate therefore, we employ a RBF method to
obtain the required gradient information, see Appendix A for details.
For a given set of N samples, we approximate the ESE and EIG values for each of the 20 EDs. Recall, our goal is to correctly classify each ED as belonging to either the optimal or sub-optimal group. In Figure VI.2, we show the approximate ESE~l and EIG values, for each of the 20 EDs, as a function of the number of model solves available. Notice the clear difference in ESE~l between the optimal and sub-optimal clusters of EDs for all N. However, from this figure, the EIG values of each of the 20 EDs appear to be the same for all N.
In Table VI. 1, we choose one optimal and one sub-optimal ED and show the EIG values for increasing N. We see that the EIG values for the chosen optimal and sub-optimal EDs are the same, out to twelve significant digits, for N < 1000. This poses a concern, as the ESE informs us that the optimal cluster of EDs is approximately ten times more sensitive to (informative of) the parameters than the sub-optimal cluster,
i.e., we expect an ED from the optimal cluster to scale a set in the data space of (admeasure approximately 650 units to a set in A of pi \-measure one unit and an ED from the sub-optimal cluster to scale a set in the data space of (id-measure approximately 70 units to a set in A of pi \-measure one unit, see Figure V.2 in Chapter V for an illustration of this geometric interpretation of the ESE. Thus, using an ED from the optimal cluster to solve a SIF reduces uncertain sets in the data space by an order of magnitude more in pi \-measure compared to using one of the suboptimal EDs. However, the EIG does not differentiate between the two groups until we use more than 1000 model solves. Even for 1000 < N < 5000, the EIG values amongst the two clusters are relatively similar. Consider N = 5000, the relative difference between the
78

two EIG values is approximately 0.0041%.
Figure VI.2: (left) The ESE-1 for the 20 EDs under consideration. Notice the clear difference in ESE-1 values between the optimal group (red) and the sub-optimal group (blue). The optimal group looks to be just a single line, however, there are actually 10 lines plotted here and they simply he so close together that they are indistinguishable, (right) The EIG for the 20 EDs under consideration. In this image, we do not see a clear distinction between the two groups of EDs. Again, there are 10 red lines and 10 blues lines here, they are just not distinguishable. See Table VI. 1 for a quantitative description of these EIG values.
Lemma VI. 1 explains why the EIG values for the optimal and sub-optimal EDs in Example VI. 1 are the same for N < 1000. Consider N = 1000, for any of the 20 EDs, each produced in Algorithm 4 contains only a single sample and therefore the corresponding KLq computed is equal to log(1000) and then EIG = log(1000) ~ 6.908. Hence, as we increase N, we increase the maximum value the EIG can take and its ability to differentiate between highly sensitive EDs. This suggests that in a scenario in which more than one ED has an EIG of approximately log(V), we should be wary of the EIGs ability to differentiate effectively between EDs. Figure VIA shows that the ESE does not suffer from this same limitation.
VI.4 Unknown Profile of the Observed Density
Some applications look to determine an OED to subsequently reduce uncertainties in model input parameters from uncertain observational data without prior knowledge
of the profiles of the observed density on each Qol. For example, consider a contami-
79

N EIG for optimal ED EIG for sub-optimal ED
10 2.30258509299 2.30258509299
50 3.91202300543 3.91202300543
100 4.60517018599 4.60517018599
300 5.70378247466 5.70378247466
500 6.21460809842 6.21460809842
1000 6.90775527898 6.90775527898
2000 7.60090245954 7.60073425065
3000 8.00636756765 8.00620455542
4000 8.2940496401 8.29361794088
5000 8.51719319142 8.51684558669
Table VI. 1: The EIG for a single ED from the optimal cluster and a single ED from the sub-optimal cluster. The EIG values are identical to twelve significant figures for N < 1000. However, from the ESE-1 values in Figure VI.2, we know that the optimal ED is approximately 10 times more sensitive to (informative of) the input parameters than the sub-optimal ED.
nant transport model in which we seek to reduce uncertainties in the diffusivity properties of the domain based on concentration measurement at some specified points in space-time. Suppose we may measure each Qol multiple times and use these multiple measurements to define the profile of the observed density on each Qol. Prior to gathering these multiple measurements, it is unlikely that the profile of the observed density is known. This presents a potential issue for the consistent Bayesian approach as this method requires the prior knowledge of the profiles of the observed density on each Qol in order to determine an OED.
Alternatively, within the context of OED, we often must prescribe an expected
utility systematically to a broad set of EDs. This set of EDs may include EDs where
the standard deviation of the profile of the observed density is large relative to the
range of the data space. In the following example, we consider both of these scenarios:
the dependence of the EIG on the profiles of the observed density on each Qol, and
the impact of highly uncertain observed densities relative to the /ip-measure of the
data space. After the example, we provide an explanation of our findings.
Example VI.2 (Time-Dependent Diffusion: Unknown Observed Density Profile)
We consider a time-dependent diffusion problem with two uncertain parameters defin-
80

ing the thermal conductivities of a metal rod. See Example IV.4 for a complete description of the model and problem setup.
Recall, the experiments involve placing two contact thermometers along the full rod that can record two separate temperature measurements at tf. Thus, there is a bijection between 11x11 and Q so points (x0, X\) G H x H can be used to index the EDs. For this example, we consider the scenario in vMch we do not know the uncertainties in each Qol, i.e., a in Eq. (IV.3) is unknown. Moreover, we assume we are given limited resources to determine an OED, specifically, we have just N = 100 model solves at a set of i.i.d random samples in A.
As in the last example, we specify two groups of EDs: one optimal group and one sub-optimal group. We define optimal and sub-optimal in terms of the ESE values approximated using the same N = 100 i.i.d. random samples that are available for the consistent Bayesian approach. In particular, we are interested in the robustness of the consistent Bayesian approach to varying profiles of the observed density on each Qol. We choose the optimal group of EDs to be 50 points chosen randomly from a square with side length 0.04 centered at the indexing point (0.97, 0.03) 6 HxO, and the sub-optimal group to be 50 points chosen randomly from a square with side length
0.04 centered at the indexing point (0.73,0.26) G H x H, see Figure VI.3.
Results
For both OED methods, we consider approximating the ESE and EIG values for N = 100 samples. We assume that we do not have an adjoint problem available to approximate {{^aw.qO)}^!}]00!; therefore, we employ a RBF method to obtain the required gradient information, see Appendix A.
We approximate the ESE and EIG values for each of the 100 EDs. Recall, our goal is to correctly classify each ED as belonging to either the optimal or sub-optimal group. In Figure VI.4, we show the approximate ESE~l and EIG as a function of the standard deviation a of the Gaussian uncertainty defined on each Qol. Notice the
81

(.ESE)_1 over the indexing set fi x H
1607.5
540.0 -472.5 -405.0
- 337.5 Sf
to KJ
- 270.0 y 202.5
1135.0
167.5
Bo.o
0.0 0.2 0.4 0.6 0.8 1.0
X0
Figure VI.3: The two groups of EDs shown in the indexing set fixfi. The group in the bottom right (over a red region) is the optimal group of EDs and the other group (over a blue region) is the sub-optimal group. For reference, we show these two clusters plotted on top of the ESE-1 (computed using N = 100 available samples in A as described above).
82

clear difference in ESE between the optimal and sub-optimal dusters of EDs for all a. This is because the ESE is not a function of the profile of the obserued density on each Qol.
Howeuer, the EIG ualues are dearly dependent on a. For small a, the EIG ualues for all of the 100 EDs are indistinguishable in Figure VI. f. Then, this method correctly classifies each ED as either sub-optimal or optimal for IE-03 < a < 1.5. Finally, for a > 1.5, the EIG ualues for the optimal group of EDs lie in the middle of the EIG ualues for the sub-optimal group of EDs.
In Table VI.2, we choose one optimal and one sub-optimal ED and show the EIG ualues for increasing a. We see that the sub-optimal ED has the same EIG ualue as the optimal ED for IE-Oh < o < li?-04, a smaller EIG ualue for li?-04 < a < 1.5, and a larger EIG ualue for a > 1.5. Moreouer, notice that the EIG ualue for the sub-optimal ED actually begins to increase as a increases beyond 1E+00. Giuen uery noisy obseruational data and, hence, a large uncertainty on the Qol defining an ED, the EIG potentially misclassifies optimal and sub-optimal EDs and produces non-intuitiue results regarding the increase of the EIG as the uncertainties on the data increase.
To prouide a more global illustration of the impacts that a has on the EIG ualues for the entire design space, in Figure VI. 5, we show the EIG ouer the indexing set Q x Q for various values of a and scatter plot the top 100 OEDs. It is clear from these images that the OED (as defined by the EIG) is a function of a. The top 100 OED locations begin as, apparently, uniform random points in VL x Q, then begin to cluster towards the bottom center ofQxQ, then cluster in the bottom right o/fixfl, and finally settle in two separate regions near the center ofQxQ as a becomes large.
As depicted in Example VI. 2, the EIG produces inconsistent OED results as the profile of the observed density on each Qol varies. Moreover, in some cases, the EIG value increases as the standard deviation of the profile of the observed density
83

Full Text

PAGE 1

PAGE 2

ThisthesisfortheDoctorofPhilosophydegreeby ScottN.Walsh hasbeenapprovedforthe AppliedMathematicsProgram by StephenBillups,Chair TroyButler,Advisor JulienLangou VarisCarey YailJimmyKim July29,2017 ii

PAGE 3

Walsh,ScottN.Ph.D.,AppliedMathematics Simulation-BasedOptimalExperimentalDesign:Theories,Algorithms,and PracticalConsiderations ThesisdirectedbyAssistantProfessorTroyD.Butler ABSTRACT Severalnewapproachesforsimulation-basedoptimalexperimentaldesignare developedbasedonameasure-theoreticformulationforastochasticinverseproblem SIP.Here,anexperimentaldesignEDreferstoanexperimentdeningaparticular mapfromthespaceofmodelinputstothespaceofobservablemodeloutputs.The termoptimalexperimentaldesignOEDthenreferstothechoiceofamapfrom modelinputstoobservablemodeloutputswithoptimalproperties.Wedevelopa consistent BayesianapproachtoOED,whichfollowstheclassicalBayesianapproach toOED,anddeterminestheOEDbasedonpropertiesofapproximatesolutionstoa setofrepresentativeSIPs. Subsequently,wedevelopanOEDapproachbasedentirelyonquantiablegeometricpropertiesofthemapsdeningtheEDs.Weproveecientcomputableapproximationstothesequantitiesbasedonsingularvaluedecompositionsofsampled Jacobianmatricesoftheproposedmaps.Weexaminethesimilaritiesanddierences amongthesetwoapproachesonasetofchallengingOEDscenarios.Anewdescription ofthecomputationalcomplexityisintroducedand,undercertainassumptions,shown toprovideaboundonthesetapproximationerrorofaninverseimage.Furthermore, agreedyalgorithmisdescribedforimplementationoftheseOEDapproachesincomputationallyintensivemodelingscenarios.Severalnumericalexamplesillustratethe variousconceptsthroughoutthisthesis. Theformandcontentofthisabstractareapproved.Irecommenditspublication. Approved:TroyD.Butler iii

PAGE 4

PAGE 5

PAGE 6

TABLEOFCONTENTS CHAPTER I.INTRODUCTION..............................1 I.1StochasticInverseProblems....................1 I.2OptimalExperimentalDesign...................2 I.3Overview...............................3 II.LITERATUREREVIEW..........................6 II.1BayesianSimulation-BasedOptimalExperimentalDesign....6 II.2ThesisObjectives..........................8 III.AMEASURE-THEORETICFRAMEWORKFORFORMULATINGSTOCHASTICINVERSEPROBLEMS........................10 III.1FormulatingtheStochasticInverseProblemUsingMeasureTheory11 III.2AConsistentBayesianSolutiontotheSIP............18 III.3AMeasure-TheoreticSolutiontotheSIP.............22 III.4ComparisonofMethods......................24 III.5NumericalExample:Consistency.................27 IV.CONSISTENTBAYESIANOPTIMALEXPERIMENTALDESIGN..34 IV.1TheInformationContentofanExperimentalDesign.......34 IV.2InfeasibleData...........................42 IV.3NumericalExamples........................47 V.MEASURE-THEORETICOPTIMALEXPERIMENTALDESIGN...53 V.1Notation...............................53 V.2 d -dimensionalParallelepipeds...................54 V.3SkewnessandAccuracy.......................57 V.4ScalingandPrecision........................61 V.5AMulti-ObjectiveOptimizationProblem.............63 V.6NumericalExamples........................65 vi

PAGE 7

V.7Summary..............................70 VI.EXPECTEDSCALINGEFFECTVSEXPECTEDINFORMATIONGAIN71 VI.1TechnicalLemmas..........................72 VI.2ComputationalConsiderations...................74 VI.3HighlySensitiveQoI........................76 VI.4UnknownProleoftheObservedDensity.............79 VI.5RedundantData..........................95 VI.6Summary..............................97 VII.THECOMPUTATIONALCOMPLEXITYOFTHESIP.........99 VII.1AReformulationoftheSIPforEpistemicUncertainty......100 VII.2Single-ObjectiveOptimizationProblem..............105 VII.3ABoundontheErrorofSetApproximations..........111 VII.4Summary..............................115 VIII.AGREEDYIMPLEMENTATIONFORSB-OED.............119 VIII.1AGreedyAlgorithm........................119 VIII.2NumericalExample.........................120 IX.CONCLUSIONS...............................129 IX.1Summary..............................129 IX.2FutureWorkandOpenProblems.................130 REFERENCES...................................133 APPENDIX A.RADIALBASISFUNCTIONSFORDERIVIATIVEAPPROXIMATIONS139 A.1UsingRBFstoInterpolateUnstructuredData..........139 A.2ConvergenceoftheExpectedRelativeError...........141 vii

PAGE 8

FIGURES III.1leftTheinverseof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 Q isoftenset-valuedevenwhen 2 speciesaparticular Q 2D .middleTherepresentationof L asa transverseparameterization.rightAprobabilitymeasuredescribedas adensityon D ; B D mapsuniquelytoaprobabilitydensityon L ; B L Figuresadoptedfrom[13]and[10]......................13 III.2Illustrationofthemapping Q : !D .Notethat Q maps B toa particularregion Q B 2D ,andwhiletheinverseimageofthisset,given by Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 Q B ,contains B ,otherpointsin mayalsomapto Q B Figureadoptedfrom[16]..........................14 III.3WesummarizethestepsinAlgorithm3.toprowWedepictSteps13.Wepartition with fV i g N i =1 thatareimplicitlydenedbythesetof samples i N i =1 ,wemaketheMCassumptiontodeterminethevolume ofeach V i .Thenwedeterminethenominalvalue q i = Q i .Here, N =2000.bottomrowWedepictSteps4-5.Wepartition D witha regulargridandprescribetheobservedprobablitymeasuretobeconstant overthedarkenedsetof V i .Here M =100.................25 III.4SummaryofAlgorithm3continued.toprowWedepictSteps5-6.For each k ,wedeterminewhichsetof i mapinto D k .Inred,weseethe unionofthe V i correspondingtothe i thatmapinto D 1 seeninredin D .Weseethesamefor D 2 inblue.bottomrowWedepictStep8.We computetheapproximatesolutiontotheSIPwhereweproportionout theprobabilityofeach D k determinedbytheobservedmeasuretoits corresponding V i .Darkercolorscorrespondtohigherprobability.....26 III.5topleftApproximationof obtainedusing obs D on Q ,topright asetofsamplesfrom ,bottomandacomparisonoftheobserved density obs D on Q withthepushforwardof ..............29 viii

PAGE 9

III.6topleftApproximationof obtainedusing obs D on Q ,topright asetofsamplesfrom ,bottomandacomparisonoftheobserved density obs D on Q withthepushforwardof ..............30 III.7leftTheapproximationof usingthejointdensity obs D on Q ,right asetofsamplesfrom ...........................31 III.8topleftThepushforwardof ,toprighttheexactobserveddensity, bottomthedierencebetweenthepushforwadof andtheobserved densityscaledbythemaximumoftheobserveddensity..........32 IV.1leftApproximationof obtainedusing obs D on Q ,whichgives KL Q obs D 0 : 466.right obs D on Q andthepushforwardofthe prior......................................37 IV.2leftApproximationof obtainedusing Q and obs D on Q ,which gives KL Q obs D 2 : 015.right obs D on Q andthepushforwardof theprior....................................37 IV.3topleftTheapproximationofthepushforwardoftheprior.topright Theexact obs D on Q .bottomTheapproximationof using obs D on Q ,whichgives KL Q obs D 2 : 98....................39 IV.4leftThepushforwardofthepriorandobserveddensitieson Q .right Thepushforwardofthepriorandobserveddensitieson .Noticethe supportofeachobserveddensityiscontainedwithintherangeofthe model,i.e.,theobserveddensitiesareabsolutelycontinuouswithrespect totheircorrespondingpushforwarddensities................44 IV.5topleftThepushforwardofthepriorforthemap Q .toprightThe observeddensityusingthejointinformationwhichnowextendsbeyond therangeofthemap.bottomThenormalizedobserveddensitythat doesnotextendbeyondtherangeofthemap................46 ix

PAGE 10

IV.6SmoothedplotoftheEIGovertheindexingsetforthedesignspace Q Noticethehighervaluesinthecenterofthedomainandtowardsthetop rightinthedirectionoftheconvectionvectorfromthelocationofthe source,thisisconsistentwithourintuition.................49 IV.7leftSmoothedplotoftheEIGovertheindexingset forthedesign space Q .Noticethehighvaluesinthebottomrightcornerof .This locationcorrespondstoplacingthermometersnearseparateendsofthe rod.However,weobservemultiplelocalmaximawithinthedesignspace, denotedR1,R2,R3andR4.rightTheEIGforthelocalOEDineach ofthefourregions...............................52 V.1TheESEdescribeshowuncertaintiesofaxedsizeonthedataspaceare reducedundertheinverseoftheQoImap..................63 V.2TheESEdescribeshowmuchuncertaintyonthedataspaceisreducedto thesizeoftheunitcube............................64 V.3SmoothedplotoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingsetforthedesignspace Q .Noticethehighervaluesinthecenterofthedomainandtowardsthe toprightinthedirectionoftheconvectionvectorfromthelocationof thesource,thisisconsistentwithbothintuitionandtheresultsfrom ExampleIV.3.................................67 V.4leftSmoothedplotoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 overtheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof .Thislocationcorrespondstoplacingthermometersneareachend oftherod.However,weobservefourprominantlocalmaxima,denoted R1,R2,R3andR4.rightTheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 forthelocalOEDineachofthe fourregions...................................68 x

PAGE 11

V.5leftSmoothedplotoftheESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 overtheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof Thislocationcorrespondstoplacingthermometersneareachendofthe rod.However,weobservefourprominantlocalmaxima,denotedR1,R2, R3andR4.rightTheESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthelocalOEDineachofthefourregions.69 V.6leftSmoothedplotof d Y p;y )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,with =0 : 5,overtheindexingset forthedesignspace Q .Noticethehighvaluesinthebottomright cornerof .Thislocationcorrespondstoplacingthermometersnear eachendoftherod.Here,weobserveonlyoneprominantlocalmaxima. right d Y p;y )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,with =0 : 5,forthelocalOEDinthesingleregion ofinterest....................................69 VI.1ThetwogroupsofEDsshownintheindexingset .Thegroupin thebottomrightoveraredregionistheoptimalgroupofEDsandthe groupneartheupperrightoverablueregionisthesub-optimalgroup. Forreference,weshowthesetwoclustersplottedontopoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 computedusingall5,000availablesamplesin asdescribedabove...77 VI.2leftTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthe20EDsunderconsideration.Noticetheclear dierenceinESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 valuesbetweentheoptimalgroupredandthesuboptimalgroupblue.Theoptimalgrouplookstobejustasingleline, however,thereareactually10linesplottedhereandtheysimplylieso closetogetherthattheyareindistinguishable.rightTheEIGforthe20 EDsunderconsideration.Inthisimage,wedonotseeacleardistinction betweenthetwogroupsofEDs.Again,thereare10redlinesand10 blueslineshere,theyarejustnotdistinguishable.SeeTableVI.1fora quantitativedescriptionoftheseEIGvalues.................79 xi

PAGE 12

VI.3ThetwogroupsofEDsshownintheindexingset .Thegroupin thebottomrightoveraredregionistheoptimalgroupofEDsandthe othergroupoverablueregionisthesub-optimalgroup.Forreference, weshowthesetwoclustersplottedontopoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 computedusing N =100availablesamplesin asdescribedabove............82 VI.4leftWeshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 asafunctionof toemphsizethattheESEis denedindependentoftheexpectedproleoftheobserveddensityoneach QoI.rightTheEIGforthe100EDsunderconsiderationasafunction of .Theoptimalgroupisshowninredandthesub-optimalgroupis showninblue.Notethat,as increase,theEIGfortheoptimalcluster becomessmallerthantheEIGforthesub-optimalgroup..........84 VI.5TheEIGovertheindexingset andthetop100OEDsforincreasing .Forverysmall =1 E )]TJ/F15 11.9552 Tf 12.028 0 Td [(05,weseethatthetop100OEDsappearto berandompointsin .For0 : 1 1 : 0,thetop100OEDsareall clusteredinthebottomrightof .Then,assigmaincreasebeyond 1 : 0,thetop100OEDsmoveawayfromthebottomrightof and settleintworegionsnearthecenterofthespace..............85 VI.6topleftApproximationof using obs D on Q with =1E-07,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E-07,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............87 VI.7topleftApproximationof using obs D on Q with =1E-03,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E-03,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............88 xii

PAGE 13

VI.8topleftApproximationof using obs D on Q with =1E-02,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E-02,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............89 VI.9topleftApproximationof using obs D on Q with =2E-02,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =2E-02,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............90 VI.10topleftApproximationof using obs D on Q with =3E-02,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =3E-02,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............91 VI.11topleftApproximationof using obs D on Q with =5E-02,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =5E-02,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............92 VI.12topleftApproximationof using obs D on Q with =1E-01,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E-01,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............93 xiii

PAGE 14

VI.13topleftApproximationof using obs D on Q with =1E+00,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E+00,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ...........94 VI.14SmoothedplotsoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andtheEIGovertheindexingset forthedesignspace Q .............................95 VI.15SmoothedplotsoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 andtheEIGovertheindexingsetforthe designspace Q .................................96 VII.1leftIngreen,weshowtheapproximateinverseimagewiththegiven Voronoitesselationof .Theexactinverseimageisshowninblue.Notice thetrueparameter true inrediscontainedwithintheexactinverseimage, however,itisnotcontainedwithintheapproximation.rightInblue,we showtheevent E 2B D thatdenes Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,thetruedatuminred,and theobserveddatuminblackthatisresponsibleforthedenitionof E 2B D .101 VII.2Inpurpleweshow Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,inyellowweshowtheinatedinverseimage resultingfromextendingtheboundariesoutward,andingreenweshow theapproximationtotheinatedinverseimagethatcovers Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Note that,thisinationisdeterminedbyextendingtheboundariesoutward eachdirectionbyadistanceof R N ,themaximumradiusoftheVoronoi cellsinthetesselation.............................102 VII.3topleftTheinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E andtheapproximation.topright The E 2B D deningtheuncertainitesontheobserveddata.bottom rightTheinated ~ E 2B D determinedbytheJacobianof Q and R N bottomleftTheexactinverseimage Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,theinatedinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E ,andtheresultingapproximationto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E thatcovers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E .104 xiv

PAGE 15

VII.4leftArepresentativeinverseimageresultingfrominverting E 2B D usingthemap Q a .rightArepresentativeinverseimageresultingfrom inverting E 2B D usingthemap Q b .Although Q b producesasmaller inverseimageusingthesamesized E ,thecorrelationamongstthetwo QoIcomposing Q b producesaninverseimagethatismoredicultto approximateaccuratelywithnitesampling.................109 VII.5Theinverseoftheinationeect,forboth Q a and Q b ,asafunctionof thenumberofsamplesavailable.Although M Q b 400,thechoicechangesand Q b istheOED........110 VII.6TheEIE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 overtheindexingset forthedesignspace Q forvarious numbersofsamples N .Inthetopleft,theEIE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 looksverysimilarto theESK )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 showninFigureV.5.Asweincreasethenumberofsamples, thelocationoftheOEDdoesnotchange,however,thegeneraldescription of Q changesnoticably.Webegintoseesimilarfeaturestothatofthe ESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 seeninFigureV.4...........................112 VII.7TheMCapproximationto Error d Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheouterandinnerset approximationusingaregulargridofsamplesfor N =192 ; 400 ; 784 ; 1600 fromtoptobottom.............................117 VII.8TheMCapproximationto Error d Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheouterandinnerset approximationusinguniformrandomsamplesfor N =192 ; 400 ; 784 ; 1600 fromtoptobottom.............................118 VIII.1Thedomainisconstructedbyweldingtogetherninesquareplates,each ofwhichhasconstant,butuncertain,thermalconductivity .......121 VIII.2TheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingsetforthedesignspace.NoticetheOED isshowninwhite,nearthecenterofthedomainwherethesourceislocated.123 xv

PAGE 16

VIII.3topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensorsare showninblack,andtheoptimallocationforthenextsensorisshown inwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q Noticethatthepreviouslyplacedsensorsareshowninblack,andthe optimallocationforthenextsensorisshowninwhiteatthemaximumof theESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ...................................125 VIII.4topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensorsare showninblack,andtheoptimallocationforthenextsensorisshown inwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q Noticethatthepreviouslyplacedsensorsareshowninblack,andthe optimallocationforthenextsensorisshowninwhiteatthemaximumof theESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ...................................126 VIII.5topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensorsare showninblack,andtheoptimallocationforthenextsensorisshown inwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q Noticethatthepreviouslyplacedsensorsareshowninblack,andthe optimallocationforthenextsensorisshowninwhiteatthemaximumof theESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ...................................127 VIII.6topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensorsare showninblack,andtheoptimallocationforthenextsensorisshown inwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q Noticethatthepreviouslyplacedsensorsareshowninblack,andthe optimallocationforthenextsensorisshowninwhiteatthemaximumof theESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ...................................128 A.1Theconvergenceoftheexpectedrelativeerrorforeachcomponentof Eq.A.1.Noticeeachcomponentof r f convergesatapproximatelythe rate O 1 N ...................................142 xvi

PAGE 17

A.2Theconvergenceoftheexpectedrelativeerrorforeachcomponentof Eq.A.2.Noticeeachconvergesatapproximatelytherate O 1 N .....143 xvii

PAGE 18

CHAPTERI INTRODUCTION I.1StochasticInverseProblems Scientistsandengineersstrivetoaccuratelymodelphysicalsystemswiththe intenttopredictfutureand/orunobservablequantitiesofinterestQoI.Forexample, ascientistmodelingthetransportofacontaminantsourcemaybeinterestedin predictingtheaverageconcentrationofthecontaminantinaregionofthedomain atsomefuturetime.Thatsamescientistmayalsobeinterestedintheuxofthe contaminantthroughsomeboundarythatisphysicallyunobservable,i.e.,cannotbe determinedbydirectmeasurements.Bothofthesescenariosrequiretheconstruction ofacomputationalmodelthatsimulatesthephysicalprocessofinterest.Inmany applications,theextraordinarycomplexityofthemodelinducesasetofmodelinputs thatareuncertain.Typically,modelinputsaredescribedasbelongingtoageneralset ofphysicallyplausiblevaluesusingeitherengineeringordomain-specicknowledge. Usingthecomputationalmodel,thissetofplausiblemodelinputscanbepropagated throughthemodeltomakepredictionsonunobservableQoI.WhentheQoIare sensitivetoperturbationsinmodelinputs,thepredictivecapabilitiesofeventhemost sophisticatedcomputationalmodelsmaybeseverelylimitedbyacoarsedescription ofuncertaintiesinthemodelinputs. Oftentimes,researcherslooktoreducetheseuncertaintiesbyusingdatagatheredfromtheactualphysicalprocesssimulatedbythemodel.Theideabeing,by ttingthemodeltoobservabledata,theprobabilitythatthemodelaccuratelypredictsunobservabledataisincreased.Thisprocessiscommonlyreferredtoasmodel calibration.However,tting"orcalibratingthemodeltoobserveddatacanbea diculttask.Itiscommonfortheobserveddatatobesubjecttouncertainty,e.g., measurementinstrumenterrorreducesthecondenceonehasintheexactvaluesprovidedbythedata.Suchuncertaintiesareoftendescribedusingprobabilitymodels, 1

PAGE 19

andspecicationofaprobabilitymeasureontherangeofvaluesforaQoIisone approachtorepresentuncertaintiesinobservabledata.Then,ratherthancalibrating themodeltoaxedsetofobserveddata,weformulateandsolveastochasticinverse problemSIPinordertodetermineaprobabilitymeasureonthespaceofmodel inputsinformed"bytheprobabilitymeasureontheQoI. TherearemanyframeworksforformulatingSIPsbasedondierentassumptions andmodelsfortheuncertaintiesinmodelinputs,observeddata,andeventheQoI mapsthatlinksthese.ApopularframeworkisbasedonthefamousBayes'rule [62]wheretheQoImapiseectivelyreplacedbyastatisticallikelihoodfunction. Morerecently,aframeworkbasedonmeasure-theoreticprincipleshasbeendeveloped andanalyzedfortheformulationandsolutionofaSIPthatdoesnotreplacethe QoImapforthemodel[16,10,12,13].ThesolutionofaSIPformulatedinthe measure-theoreticframeworkisdenedasaprobabilitymeasureonthespaceofmodel inputsthatis consistent withthemodelanddatainthesensethatthepushfoward ofthisprobabilitymeasurethroughtheQoImapexactlymatchestheprobability measuregivenontheQoI.Inthisthesis,weadoptthismeasure-theoreticframework forformulatingaSIPundertwoscenariosandconsidertwoseparatemethodsforthe numericalsolutionoftheSIP. I.2OptimalExperimentalDesign ThecollectionofdataforanobservableQoIcanbecostlyandtimeconsuming. Forexample,exploratorydrillingcanrevealvaluableinformationaboutsubsurface hydrocarbonreservoirs,buteachwellcancostupwardsoftensofmillionsofUS dollars.Insuchsituations,wecanonlyaordtogatherdataforsomelimitednumber ofobservableQoI.However,wemaybeabletodenecertainaspectsofanexperiment thatdenethetypesofobservableQoIforwhichwecollectdata.Forexample,wemay beabletochoosethelocationsofasetofexploratorywellsfromsomepredetermined feasibledomain.ThechoiceofobservableQoIforwhichwemaycollectdatais 2

PAGE 20

denedasanexperimentaldesignED.Thus,anEDdenesaparticularQoImap frommodelinputstomodeloutputs.SincetheQoImapplaysasignicantroleinthe solutionofaSIP,itisevidentthattwodierentEDsmayleadtosignicantlydierent solutionstoaSIP.Forexample,considerasimplecontaminanttransportproblemin whichthediusivitypropertiesofthedomainareuncertain.Weexpectthatsome measurementsinspace-timearemoreusefulthanothersatreducinguncertaintiesin thediusiontensor. Toreducetheexpenseofcollectingdataonlessusefuli.e.,lessinformativeQoI, weconsiderasimulation-basedapproachtoautomaticallyselecttheusefulQoIfrom asetofpossibleEDs.Simulation-basedoptimalexperimentaldesignSB-OED,or sometimesjustOEDlookstodeterminetheoptimalEDbywayofcomputational simulationsofthephysicalprocess.Themainideabeingtoexploitthesensitivity informationinaQoImapcorrespondingtoanEDtodeterminetheoptimalQoIto gatherdataintheeld.ThecriteriaforoptimalityofanEDcanbedenedinvarious waysdependingonthegoalsoftheresearchers.Here,weconsiderbatchdesignover adiscretesetofpossibleEDs.Batchdesign,alsoknownasopen-loopdesign,involves selectingasetofexperimentsconcurrentlysuchthattheoutcomeofanyexperiment doesnoteecttheselectionoftheotherexperiments.Suchanapproachisoften necessarywhenonecannotwaitfortheresultsofoneexperimentbeforestarting another,butislimitedintermsofthenumberofobservationswecanconsider.The pursuitofacomputationallyecientapproachforcouplingtheSB-OEDmethods developedwithinthisthesiswithcontinuousoptimizationtechniquesisanintriguing topicthatweleaveforfuturework. I.3Overview Therestofthisthesisisoutlinedasfollows.InChapterII,weprovideareview ofpastandrecentworkinSB-OED.Forcompleteness,inChapterIII,wereviewthe measure-theoreticframeworkforformulatingaSIPandtwomethodsfornumerical 3

PAGE 21

solutionoftheSIP.ThetwomethodsarereferredtoasaconsistentBayesianapproach andameasure-theoreticapproach.TheconsistentBayesianapproachadoptsthe useofapriorprobabilitydensityinthesolutiontotheSIPwhereasthemeasuretheoreticapproachdoesnotrequireanypriorknowledgeoftheprobabilityonthe modelinputstosolvetheSIP.Weproviderelevantalgorithmsandguresdepicting theimplementationofeachoftheseapproaches. WefollowaBayesianmethodologyforSB-OEDtodevelopaconsistentBayesian approachtoSB-OEDinChapterIV.TheconsistentBayesianapproachtoSIPsproves usefulinreducingthecostoftheiteratedintegraldeningtheutilityofagivenED. InChapterV,wedevelopanentirelyuniqueapproachtoSB-OEDinspiredbythe set-basedapproximationsofthemeasure-theoreticapproach.Thisapproachdeviates fromtheBayesianmethodologyforSB-OEDinthatitquantiestheutilityofa givenEDentirelyintermsofthelocalgeometricpropertiesofthecorrespondingQoI mapanddoesnotrequiresolutiontoanySIP.Thus,thisapproachisindependent ofboththemethodologyemployedtosolvethechosenSIPandthespecicationof aprobabilitymeasureonthespaceofobservableQoI.Consequently,undercertain assumptions,thesetwoSB-OEDapproachesproducesimilarqualitativedescriptions ofaspaceofEDs.Thus,inChapterVI,weprovideanindepthcomparisonbetween thesetwomethodsfocusedspecicallyonahandfulofchallengingSB-OEDscenarios. FollowingChapterVI,wefocusentirelyonthemeasure-theoreticapproachto SB-OEDandlooktoresolveandimproveuponthecurrentstateofthemethod.In ChapterVII,wefocusonthreemajortopics.First,SectionVII.1proposesareformulationoftheSIPsothat,undercertainassumptions,weguaranteethetrue parameter 1 iscontainedwithinaregionofnonzeroprobabilityinthesolutiontothe SIP.Althoughthisapproachisdevelopedtoimprovereliabilityinasolutiontoa givenSIP,itprovidesusefulinsightintoSB-OEDinterests.InSectionVII.2,we 1 Bytrueparameter,wearereferringtoanapplicationinwhichtheSIPfundamentallyisdening aproblemofparameteridenticationunderepistemicuncertainties. 4

PAGE 22

scalarizethemulti-objectiveoptimizationproblem,denedinChapterV,implicitly asafunctionofthesensitivitiesoftheQoImap,thedimensionofthemodelinput space,andthenumberofmodelsolvesavailabletoultimatelyapproximatethesolutiontoachosenSIP.InSectionVII.3,weprovideaboundontheerrorinthe approximationtoaninverseimagewhichoersanalternativedescriptionofthecomputationalcomplexity,i.e.,conditionnumber,ofagivenSIP.InChapterVIII,we illustratethemeasure-theoreticapproachtoSB-OEDonatoyproblemthatyields high-dimensionalparameter,data,anddesignspaces.Weemployagreedyalgorithm toabatethiscurseofdimensionality,thus,weapproximatetheOEDfromacombinatorialenumerationofadiscretizationofthedesignspace.InChapterIX,weprovide concludingremarksanddiscussfutureresearchopportunitiesanddirections. 5

PAGE 23

CHAPTERII LITERATUREREVIEW Itisworthnotingthattheterm optimalexperimentaldesign canmeanagreat manythingsdependingonthecontext.Thetermexperimentaldesignsometimes referredtoasdesignofexperiments,waslikelyrstpopularizedwithinthestatistical community,e.g.,see[42,43,29,41,5]andthereferencesthereinforearlierreferences onthetopic.Inthiscontext,anexperimentaldesignreferstosomeplacementofsampleswithinthepredictorspace.Anoptimalexperimentaldesignreferstoanoptimal placementofsampleswithinthepredictorspace,wheretheoptimalitycriterionmay beapplicationspecic.Theso-calledalphabeticoptimality"criteria[5,40]look tooptimizepropertiesofparameterestimatesdeningamodel,e.g., D -optimality tomaximizethedierentialShannonentropy, A -optimalitytominimizetheaverage varianceofparameterestimates,and G -optimalitytominimizethemaximumvarianceofmodelpredictions.ThesecriteriahavebeendevelopedinbothBayesianand non-Bayesiansettings[5,40,6,25,18,19].Insummary,thiseldofexperimental designresearchfocusesonoptimizingthequalityofa statistical modelrelatingthe predictorstotheresponses. II.1BayesianSimulation-BasedOptimalExperimentalDesign Althoughdevelopingappropriatestatisticalmodelsisofutmostimportancein manypredictivemodelingapplications,weemphasizethatthisdierssubstantially fromtheapproachesproposedinthisthesis.Theapproachesproposedbeloware developedspecicallyforphysics-basedcomputationallyexpensiveandpotentially nonlinearmodelswithuncertainmodelinputparameters.Givenaparameterpredictor,thecomputationalmodelprovidesthevalueoftheQoIresponses.Hence, weneednotapproximateanoptimalmodel. However,uncertaintiesinmodelinputparametersleadtouncertaintiesinmodel predictions.Ifuncertaintiesontheparameterscanbequantiedandreducedusing 6

PAGE 24

experimentaldata,thentheuncertaintiesinmodelpredictionscanalsobequantied andreduced.WeaccomplishthiswiththeformulationandsolutionofaSIP.Given aprobabilitydistributiononsomecollectionofobservableQoI,thesolutionofthe SIPisaprobabilitydistributiononthespaceofmodelinputparameters.TheprobabilitydistributionontheobservableQoImaybedeterminedbycombiningavailable informationontheexperimentaldataandobservationinstrumenterror. DierentcollectionsofQoImayprovideinformationaboutdierentdirections withintheparameterspace.Hence,inthiscontext,anoptimalexperimentaldesign OEDreferstoanoptimalcollectionofQoI,whereoptimalitycriterionareoften derivedfromaninformationtheoreticfoundation[46,65,8].ThedirectionalinformationcontainedinaparticularcollectionofQoIcanbeanalyzedbynumerical simulationsofthemodel.Thus,wemaybeabletorankdierentEDsbysomeperformancemetricbasedentirelyonsimulations.Wefollowrecentliterature[35,36,54] andrefertothistypeofexperimentaldesignas simulation-basedoptimalexperimental design SB-OEDtoemphasizethenecessityofacomputationalmodelrelatingthe uncertainparameterstotheQoI. GivenaprobabilitydistributiononthechosenspaceofobservableQoI,the BayesianparadigmisoftenusedtoformulateandsolveaSIPtodetermineaprobablitydistributiononthespaceofmodelinputparameters[59,62].Thisperspective hasleadtoaBayesianapproachtoSB-OED,whereapproximateBayesiansolutions totheSIPdeterminetheexpectedutilityofagivenED[20,8,63].Theexpected utilityofanEDisoftenwrittenasaniteratedintegraloverboththeparameterspace andthespaceofQoIdeningtheED.Hence,itiscomputationallyexpensiveto approximatemanyusefulformsoftheexpectedutility,e.g.,thecommonlyused expectedinformationgain EIG[8]derivedfromtheKullback-LeiblerKLdivergence [44,64].Therefore,muchoftheliteratureinthiseldisdirectedtowardsreducing computationalcomplexityinapproximatingtheexpectedutilityofanED. 7

PAGE 25

Locallyoptimaltechniquesreducecostbutrequireabestguess"ofthevalueof theunknownparameter[30].Thisapproachisnotsuitablewhentheposteriorparameterdistributionisbroad.In[57],MarkovchainMonteCarloMCMCmethods andLaplacianapproximationsareusedtocomputetheEIG.In[9,21],theauthorsreducecomputationalcostbyapproximatingtheposteriorswithGaussiandistributions. WhentheposteriordierssignicantlyfromaGaussianprole,suchanapproachis notapplicable.In[48,49,50],theauthorsuseLaplaceapproximationstoaccelerate thecomputationsoftheEIG.In[35],theauthorsutilizeageneralizedpolynomial chaossurrogatetoacceleratecomputationsoftheEIGandstochasticoptimization techniquestodetermineanOEDinahigh-dimensionalsetting.In[55,4],theauthors useMCMCandsimulatedannealingmethodstoacceleratethecomputationofthe expectedutility.In[1,2],theauthorsdevelopanOEDframeworkforinverseproblemswithinnitedimensionalparameterspacesresultingfromanunknowneldin thephysics-basedmodel. Allofthemethodsdescribedherehaveprovedvaluableintheappropriatecontext. However,thegeneralformulationofBayesianSB-OEDrequiresthesolutions,or approximatesolutions,ofSIPstodetermineanOED.Thus,theseapproachesrequire rstformulatingaspecictypeofSIPwithinaparticularframework,knowledgeof themethodtobeusedtosolvetheresultingSIP,and,nally,anestimateofthe uncertaintiesontheobservedQoI. II.2ThesisObjectives Theobjectiveoftheapproachesdescribedaboveistoapproximatetheexpected utilityofanEDbasedonapproximatesolutionstotheclassicalBayesianSIP[59, 62].Althoughthisapproachhasprovedimmenselyusefulinthecontextofclassical Bayesianinference,inthisthesis, wefocusspecicallyondevelopingSB-OED approachesinspiredby consistentsolutions totheSIP [16,10,12,13].The consistentBayesianSB-OEDapproach,developedinChapterIV,contributestothe 8

PAGE 26

lineofresearchregardingreductionincomputationalcostbyemployingtherecently developedconsistentBayesianformulationandsolutiontotheSIP[16]toeciently integrateoverthespaceofobserveddensitiestoapproximatetheEIG.Futurework willexplorethecouplingofthismethodwithcontinuousoptimizationtechniquesto determineOEDsinhigh-dimensionaldesignspaces. Themeasure-theoreticapproachtoSB-OED,developedinChapterV,derives acomputationallytractableformulationforSB-OEDinwhich notasingleSIPis solved and,therefore,is independent ofboththeframeworkusedtoformulatethe SIP,themethodusedtosolvetheSIP,andalsotheestimateduncertaintiesonthe observedQoI.Anadditionalbenetisthatthemeasure-theoreticmethodprovidesa quanticationofthecomputationalcomplexityinvolvedinsolvingthenalSIPusing sample-basedapproaches. InChapterVI,wepresentathoroughcomparisonbetweenthesetwoSB-OED approachesandillustratethestrengthsandweaknessesofeach.ChapterVIIcontinuestoextendthemeasure-theoreticapproachtoSB-OEDbybothscalarizinga previouslydenedmulti-objectiveoptimizationproblemandcontributingfurtherto thequanticationofthecomputationalcomplexityofaSIP.InChapterVIII,we illustrateagreedyimplementationofthemeasure-theoreticapproachtoSB-OEDin ahigh-dimensionalsetting. 9

PAGE 27

PAGE 28

thissolutiononthechoiceofobservableQoIthatcomposetheQoImap. III.1FormulatingtheStochasticInverseProblemUsingMeasureTheory Here,wesummarizetheworkof[10,12,13].Inordertoprovideacomplete descriptionofthemeasure-theoreticframeworktotheSIP,werstintroducesome necessarynotationandthencarefullydiscussataxonomyofthreeforwardproblems, withincreasinglevelsofuncertainty,alongwiththeirdirectinverseproblems.Fora morethoroughdescriptionandanalysisoftheseproblemsandsolutions,see[10,13]. Let M Y; denoteadeterministicmodelwithsolution Y thatisanimplicit functionofmodelparameters 2 R n .Theset representsthelargestphysically meaningfuldomainofparametervalues,and,forsimplicity,weassumethat is compact.Inpractice,modelersareoftenonlyconcernedwithcomputingarelatively smallsetofQoI, f Q i Y g d i =1 ,whereeach Q i isareal-valuedfunctionaldependenton themodelsolution Y .Since Y isafunctionofparameters ,soaretheQoIandwe write Q i tomakethisdependenceexplicit.GivenasetofQoI,wedenetheQoI map Q := Q 1 ; ;Q d > : !D R d where D := Q denotesthe rangeoftheQoImap. Level1:DeterministicPoint-BasedAnalysis Therstforwardproblemisthesimplestinpredictivesciencewhichistoevaluate themap Q foraxed 2 todeterminethecorrespondingoutputdatum Q = q 2D .Inotherwords,oncetheinputsofamodelarespecied,thesimplestforward problemistosolvethemodelinordertopredicttheoutputdatum.Thereisno uncertainty,saveformodelevaluationerror,inthesepointvalues. Thecorrespondinginverseproblemistodeterminetheparameters 2 that produceaparticularvalueof q 2D .Oftentimes Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 q denesasetofvaluesin eitherduetonon-linearitiesinthemap Q and/orthedimensionof beinggreater thanthedimensionof D .Thus,thesimplestinverseproblemoftenhasuncertaintyas totheparticularvalueof 2 thatproducedaxedoutputdatum.However,there 11

PAGE 29

isnouncertaintyaboutwhichset-valuedinverseproducedthexedoutputdatum. When n =2and d =1,aset-valuedinverse, Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 q ,denesacontourin thatwe arefamiliarwithfromcontourmaps,seetheleftplotinFigureIII.1.Ingeneral,we refertotheset-valuedinversesofthemap Q as generalizedcontours Assuming d n andthattheJacobianof Q hasfullrowrankeverywherein ,thegeneralizedcontoursexistaspiecewise-dened n )]TJ/F20 11.9552 Tf 11.961 0 Td [(d -dimensionalmanifolds in .TheassumptionthattheJacobianof Q hasfullrowrankisapractically importantonesinceweprefertouseQoIthataresensitivetotheparameterssince otherwiseitisdiculttoinferusefulinformationabouttheuncertainparameters. Moreover,wepreferQoItoexhibitthesesensitivitiesinuniquedirectionsin asthis allowsdierentQoItoinformaboutdierentuncertainmodelinputparameters.We formallycharacterizethepropertythattheJacobianof Q hasfullrowrankinthe followingdenition. DenitionIII.1 Thecomponentsofthe d -dimensionalvector-valuedmap Q are geometricallydistinctGD iftheJacobianof Q hasfullrowrankateverypoint in .WhenthecomponentmapsareGD,wesaythatthemap Q isGD. Henceforth,weassumethatanymap Q isGD.Thisimpliesthat d n ,i.e.,the numberofobservableQoIisnolargerthanthenumberofuncertainmodelparameters. Level2:DeterministicSet-BasedAnalysis Thesecondtypeofforwardproblemcanoftenbewrittenmathematicallyas analyzing Q B D givenaset B .Inotherwords,thereisgenerallyuncertainty astotheprecisevalue takesinaset B andsubsequentlythereisuncertaintyas totheparticulardatumtopredictintheset Q B .Thisisacommonlystudied,and wellunderstood,topicinthemathematicalandengineeringsciences.Forexample, itiscommonindynamicalsystemstostudyhowsetsofinitialconditionsevolvein time.Oftentimesweareinterestedindescribingthesizeofthesetsandhowthat 12

PAGE 30

FigureIII.1: leftTheinverseof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q isoftenset-valuedevenwhen 2 speciesaparticular Q 2D .middleTherepresentationof L asatransverse parameterization.rightAprobabilitymeasuredescribedasadensityon D ; B D mapsuniquelytoaprobabilitydensityon L ; B L .Figuresadoptedfrom[13]and [10]. sizechangesfrommodelinputstomodeloutputs.Thus,weassume ; B ; and D ; B D ; D aremeasurespaceswhere B and B D aretheBorel -algebrasinherited fromthemetrictopologieson R n and R d ,respectively,andthemeasures and D arevolumemeasures.Thevolumemeasureson and D allowustoquantitatively assesshowmodelinputschangetomodeloutputsintermsofthethemeasuresof A and Q A ,respectively. Thecorresponding deterministic inversesensitivityanalysisproblemistoanalyze Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A forsome A 2B D .While Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A 2B ,thepracticalcomputationof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A iscomplicatedbythefactthat Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 doesnotmapindividualpointsin D toindividual pointsin .Asdescribedabove,assuming d n and Q isGD, Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 mapsapointin D toageneralizedcontourdescribedasan n )]TJ/F20 11.9552 Tf 10.654 0 Td [(d -dimensionalmanifoldembeddedin .Thus, Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A denesa generalizedcontourevent ,orjustcontourevent,belonging toan inducedcontour -algebra C B wheretheinclusionisoftenproper.To observethattheinclusionisoftenproper,observethatforany B 2B B Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q B ; III.1 butinmanycases B 6 = Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q B evenwhen n = d seeFigureIII.2where n =2 13

PAGE 31

and d =1foranillustration.Inotherwords,solutionstothecorrespondinginverse problemcanbedescribedinthemeasurablespace ; C andthevolumemeasure canbeusedtoprovidequantitativeassessmentsofsolutionstothisinversesensitivity analysisproblem. FigureIII.2: Illustrationofthemapping Q : !D .Notethat Q maps B to aparticularregion Q B 2D ,andwhiletheinverseimageofthisset,givenby Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q B ,contains B ,otherpointsin mayalsomapto Q B .Figureadopted from[16]. Itisusefultoconsidersolutionstothisdeterministicinverseprobleminaspace ofequivalenceclasses.Specically,wemayusethegeneralizedcontourstodene anequivalenceclassrepresentationof wheretwopointsareconsideredequivalent iftheylieonthesamegeneralizedcontour.Welet L denotethespaceofsuch equivalenceclassesandlet L : !L denotetheprojectionmapwhere L = ` 2 L denestheequivalenceclasscorrespondingtoaparticular and )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 L ` = C ` isthe generalizedcontourin correspondingtothepoint ` 2L .Itispossibletoexplicitly represent L in bychoosingaspecicrepresentativeelementfromeachequivalence class.Asdescribedin[10,13],sucharepresentationof L canbeconstructedby piecewise d -dimensionalmanifoldsthat index the n )]TJ/F20 11.9552 Tf 13.231 0 Td [(d -dimensionalgeneralized contours.Werefertoanysuchindexingmanifoldasa transverseparameterization see themiddleplotofFigureIII.1.Givenaparticularindexingmanifoldrepresenting L Q denesabijectionbetween L and D .Themeasurespace L ; B L ; L canbe 14

PAGE 32

denedasaninducedspaceusingthebijection Q and D ; B D ; D .Then,solutions tothedeterministicinverseproblemcanbedescribedandanalyzedasmeasurable setsofpointsin L insteadofmeasurablegeneralizedcontoureventsin Level3:StochasticAnalysis Wenowassumethatthereisuncertainty,describedintermsofprobabilities,as towhichset A 2B themodelparameters belong,andthegoalistoanalyzethe probabilitiesofsetsin B D .Inthelanguageofprobabilitytheory,measurablesetsare referredtoasevents.Thus,weassumethataprobabilitymeasure P isgivenon ; B describinguncertaintyintheeventsforwhichparametersmaybelong,andthe goalistodetermine P D on D ; B D .Werefertothisasthe stochastic forwardproblem. When P resp., P D isabsolutelycontinuouswithrespecttothevolumemeasure resp., D ,thecorrespondingRadon-Nikodymderivativei.e.,theprobability densityfunction resp., D isusuallygiveninplaceoftheprobabilitymeasure. Weassumethisisthecasesothatwecanrefertoprobabilitiesofeventsintermsof themorecommonrepresentationusingintegralsanddensityfunctions,e.g., P A = Z A d ;A 2B : III.2 Solutiontothisstochasticforwardproblemisgivenbytheinducedpushforward probabilitymeasure P D denedforany A 2B D by P D A = Z A D d D = Z Q )]TJ/F19 5.9776 Tf 5.757 0 Td [(1 A d = P Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A : III.3 Thisisafamiliarprobleminuncertaintyquantication.Theapproximatesolution canbeobtainedbyaclassicMonteCarlomethod.Assumingthat Q isGD,the pushforwardvolumemeasure,bythemap Q ,isabsolutelycontinuouswithrespect totheusualLebesguemeasureon R d [13,16].Therefore,unlessotherwisenoted,we assume and D aretheusualLebesguemeasureson R n and R d ,respectively. 15

PAGE 33

ThecorrespondingSIPassumesthatwearegivenanobservedprobabilitymeasure, P obs D ,on D ; B D ,whichcanbedescribedintermsofanobservedprobability density, obs D ,i.e., obs D istheRadon-Nikodymderivativeof P obs D withrespectto D TheSIPisthendenedasdeterminingaprobabilitymeasure, P ,describedasa probabilitydensity, ,suchthatthepushforwardmeasureof P agreeswith P obs D on D ; B D .Weuse P Q P D todenotethepushforwardof P through Q ,i.e., P Q P D A = P Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A ; forall A 2B D .Usingthisnotation,aconsistentsolutiontotheSIPisdened formallyasfollows: DenitionIII.2Consistency Givenaprobabilitymeasure P obs D describedasa density obs D withrespectto D on D ; B D ,theSIPseeksaprobabilitymeasure P describedasaprobabilitydensity withrespectto on ; B suchthatthe subsequentpushforwardmeasureinducedbythemap Q satises P Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A = P Q P D A = P obs D A ; III.4 forany A 2B D .Werefertoanyprobabilitymeasure P thatsatises III.4 asa consistent solutiontotheSIP. TodeterminesuchasolutiontotheSIP,werstconsidertheSIPposedon L ; B L Since Q denesabijectionbetween L and D P obs D denesaunique P L on L ; B L seetheright-handplotinFigureIII.1and[13].Wecanthenusetheprojection map L toprovethefollowingtheoremaswasdonein[13]. TheoremIII.3 TheSIPhasauniquesolutionon ; C However,thegoalistodeneaprobabilitymeasure P onthemeasurablespace 16

PAGE 34

; B notonaspaceinvolvingcontoureventsthatarecomplicatedtodescribe.This requiresanapplicationoftheDisintegrationTheorem,whichallowsfortherigorous descriptionofconditionalprobabilitymeasuresdenedonsetsofzero -measure [24,13,32]. TheoremIII.4TheDisintegrationTheorem Assume Q : !D is B measurable, P isaprobabilitymeasureon ; B and P obs D isthepushforwardmeasureof P on D ; B D .Thereexistsa P obs D -a.e.uniquelydenedfamilyofconditional probabilitymeasures f P q g q 2D on ; B suchthatforany A 2B P q A = P q )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(A Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 q ; so P q n Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 q =0 ,andthereexiststhefollowingdisintegrationof P P A = Z D P q A dP obs D q = Z D Z A Q )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 q dP q dP obs D q ; III.5 for A 2B .Equivalently,becausethereisabijectionbetween L and D ,wemaywrite thisiteratedintegralintermsof L P A = Z L Z A )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 L ` dP ` dP L ` ; III.6 for A 2B where P L istheuniqueprobabilitymeasureinducedon L by P obs D Forproofsrelatedtotheabovedisintegrationtheoremsee[13,16].Itfollowsfrom thedisintegrationtheoremthatifaprobabilitymeasure P obs D on D ; B D isgiven, thenthespecicationofafamilyofconditionalprobabilitymeasures f P q g q 2D or, equivalently, f P ` g ` 2L on ; B canbeusedtodeneaspecicprobabiltymeasure P usingEq.III.5or,equivalently,Eq.III.6thatisconsistentinthesenseof DenitionIII.2.However,aconsistentsolutionmaynotbeunique,i.e.,theremaybe 17

PAGE 35

multiplefamiliesofconditionalprobabilitymeasureson ; B thatdenemultiple consistentprobabilitymeasures.Auniquesolutionmaybeobtainedbyimposing additionalconstraintsorstructureontheSIPthatleadstoauniquespecicationof thefamilyofconditionalprobabilitymeasureson ; B .Themethodsexploredin SectionIII.2andSectionIII.3describehowwemayuseeitherassumedorexisting measure-theoreticinformationtodenethefamiliesofconditionalprobabilities. III.2AConsistentBayesianSolutiontotheSIP Here,wesummarizetheworkof[16]. III.2.1MathematicalFormulation FollowingtheBayesianphilosophy[62],weintroduceanite prior probability measure P prior describedasaprobabilitydensity prior on ; B withrespectto Thepriorprobabilitymeasureencapsulatesanyexistingknowledgeabouttheuncertainparametersandservestodenethenecessaryfamilyofconditionalprobability measureson ; B toobtainauniquesolutiontotheSIP. Thepriorinformationcancomefromvarioussources.Forexample,engineering knowledgeofthefeasibleparameterdomainmayinformboundsonthesupportofthe priordensityandpastexperimentalresultsmayinformanappropriatedistribution fortheprioroverthisfeasibleregion.Intheabsenceofaphysicallymeaningful priormeasure,oranypreviousresultsfromwhichtobuildone,itmaybepossibleto developapriorusingacoarsesolutiontotheSIPusingthemeasure-theoreticmethod whichdoes not requireapriorprobabilitymeasure,seeSectionIII.3.The quality of apriorprobabilitymeasureandsuitablemethodsfordevelopmentofsuchaprioris acurrenttopicofscienticresearchanddebate,andthistopicisbeyondthescope ofthisexpositionoftheconsistentBayesiansolutiontotheSIP. Assumingthat Q isatleastmeasurable,thenthepriorprobabilitymeasureon ; B P prior ,andthemap, Q ,induceapushforwardmeasure P Q prior D on D ,which 18

PAGE 36

isdenedforall A 2B D P Q prior D A = P prior Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 A : III.7 Recallfromclassicalprobabilitytheorythatif P isaprobabilitymeasure, B isan eventofinterest,and A isanarbitraryevent,then P B = P B j A P A + P B j A c P A c : Furthermore,if B A ,thenthisreducesto P B = P B j A P A : III.8 Inthiscontext,theterm P B representstheconsistentprobabilitymeasureofthe event B thatweseekandwelet A := Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q B ,whichensuresthat B A .We aremotivatedtouse P A = P obs D Q A inEq.III.8sincethedenitionofthe pushforwardmeasureimpliesthattheconsistencyconditionisthensatisedbecause Q A = Q B .TheremainingterminEq.III.8isdeterminedusingtheprior measure P prior ,theclassicalBayes'theoremforevents,andthefactthat B A P B j A = P prior A j B P prior B P prior A = P prior B P prior A : Itfollowsfromthedenitionof P Q prior D that P prior A = P Q prior D Q B .Thus,we utilizethefollowingformalexpressionfor P P B := 8 > > < > > : P prior B P obs D Q B P Q prior D Q B ; if P prior B > 0 ; 0 ; otherwise ; III.9 Thefollowingpropositionandensuingtheoremhighlightthemajorstepsneededto 19

PAGE 37

provetheexistenceanduniquenessofaconsistentBayesiansolutiontotheSIPgiven byaprobablitymeasureon ; B PropositionIII.5 TheprobabilitymeasuregivenbyEq. III.9 denesaprobability measureon ; C Wenotethat,althoughPropositionIII.5saysthat P ,asdenedabove,denesa probabilitymeasureonthecontour -algebraon ; C ,itisnotnecessarilytrue thatthis P abovedenesaprobabilitymeasureon ; B ,see[16]foradiscussion onthistopic.Weuse P prior todenetheconditionalprobabilitydensitiesasRadonNikodymderivativeswithrespecttothedisintegrationofthevolumemeasure ;q TheoremIII.6utilizesthedisintegrationtheoremtoconstructaconsistentprobability measureon ; B TheoremIII.6 Theprobabilitymeasure P on ; B denedby P A = Z D Z A Q )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 q dP prior q dP obs D q ; 8 A 2B ; III.10 where dP prior q = prior Q prior D Q d ;q ; 8 q 2D isaconsistentsolutiontotheSIPinthesenseofDenitionIII.2. Wenotethat P prior isusedonlytodenethefamilyofconditionalprobabilitymeasures P prior q q 2D on ; B ,neededtoobtainauniquesolutiontotheSIP,interms ofthepriorprobabilitydensityandthemap Q .Itfollowsthat P inEquationIII.10 isgivenby P A = Z D Z A Q )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 q prior Q prior D Q d ;q dP obs D q III.11 forall A 2B .Furthermore,since dP obs D q = obs D d D q ,wecansubstitutethisinto 20

PAGE 38

Eq.III.11toobtainthedensity = prior obs D Q Q prior D Q ; 2 ; III.12 which,whenintegratedintermsoftheaboveiteratedintegral,inducesaprobability measure P on ; B thatisaconsistentsolutiontotheSIP. III.2.2NumericalApproximationofSolutions Approximating usingtheconsistentBayesianapproachrequiresanapproximationofthepushforwardofthepriorprobabilitymeasureonthemodelparameters, whichisfundamentallyaforwardpropagationofuncertainty.Weconsiderthemost basicofmethods,namelyMonteCarlosampling,tosamplefromtheprior.Weevaluatethecomputationalmodelforeachofthesamplesfromtheprioranduseastandard kerneldensityestimatorKDEtoapproximatethepushforwardoftheprior.Given theapproximationofthepushforwardoftheprior,wecanevaluate atanypoint 2 ifwehave Q .Thus,wecanconstructanapproximationof Wesummarizethestepsrequiredtonumericallyapproximatethepushforwardof thepriorinAlgorithm1,andsummarizethestepsrequiredtoapproximatethesolutiontotheSIPinAlgorithm2.Wenotethat,thetwoprominentcostsinproducing anumericalapproximationto are:theevaluationofthemodel,andthe constructionofanapproximationto Q prior D Algorithm1: ComputingthePushforwardDensityInducedbythePriorand theModel 1.Givenasetofsamplesfromthepriordensity: i i =1 ;:::;N ; 2.Evaluatethemodelateach i andcomputetheQoI: q i = Q i ; 3.UsethesetofQoIandastandardtechnique,suchasKDE,toestimate Q prior D q III.3AMeasure-TheoreticSolutiontotheSIP 21

PAGE 39

Algorithm2: NumericalApproximationoftheConsistentBayesianSolution 1.Givenapriordensityon prior ; 2.Givenanobserveddensityon D obs D ; 3.Giventhepushforwardofthepriordensityon D Q prior D ; 4.Givenasetofsamplesin : i i =1 ;:::;N ; 5.Evaluatethemodelateach i ; 6.Computethevalueof ateach i : i = prior i obs D Q i Q prior D Q i : III.13 7.Use i toconstructanapproximationto Wefollow[13,32]andadoptwhatisreferredtoasthe standardAnsatz determined bythedisintegrationofthevolumemeasure tocomputeprobabilitiesofevents insideofacontourevent,i.e.,todenethefamilyofprobabilitymeasureson ; B requiredtoobtainauniquesolutiontotheSIP.ThestandardAnsatzisgivenby P ` B = C ` B C ` C ` ; 8 B 2B C ` ; forall ` 2L where C ` isthedisintegratedvolumemeasureonthegeneralizedcontour C ` .Notethat,thisisthesameas ;q discussedabovewhere C ` = Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 q .Also, B C ` denotesthestandardBorel -algebraonthe C ` ThestandardAnsatzcanbeusedaslongas C ` isnitefora.e. ` 2L ,e.g., ashappenswhen R d iscompact, isthestandardLebesguemeasure,and Q iscontinuouslydierentiable.Assumingsuchconditionsholdtoapplythestandard Ansatz,theapproximationmethodandresultingnon-intrusivecomputationalalgorithmcanbeeasilymodiedforalmostanyotherAnsatz.See[13]formoredetails andtheoryregardinggeneralchoicesoftheAnsatz.CombininganAnsatzwiththe 22

PAGE 40

disintegrationtheoremprovesthefollowingtheorem. TheoremIII.7 Theprobabilitymeasure P on ; B denedby P A = Z L Z A )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 L ` dP ` dP L ` ; III.14 where P ` B = C ` B C ` C ` ; 8 B 2B C ` ; 8 ` 2L isaconsistentsolutiontotheSIPinthesenseofDenitionIII.2. ThestandardAnsatzresultsinaprobabilitymeasure P thatinheritskeygeometricfeaturesfromthegeneralizedcontourmap.AnyotherchoiceofAnsatzimposes someothergeometricconstraintsonthesolutiontotheinverseproblemthatarenot comingfromthemap Q .IfwechoosenottouseanyAnsatz,thenwecanalways solvetheSIPon ; C ,i.e.,wecanalwayscomputetheuniqueprobabilitymeasure whenrestrictedtocontourevents. III.3.1NumericalApproximationofSolutions Fundamentaltoapproximatingsolutions P totheSIPistheapproximationof eventsinthevarious -algebras, B D C and B .Since C B ,wecansimultaneouslyapproximateeventsinbothofthese -algebrasusingthesamesetofevents partitioning .Let fV i g N i =1 denotesuchapartitionof where V i 2B foreach i Assumethatwearegivenacollectionofsets f D k g M k =1 B D partitioning D ,seeFigureIII.3.Thebasicalgorithmicprocedureforapproximating P V i foreach i isto determinewhichofthe fV i g N i =1 approximate Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 D k ,andthentoapplytheAnsatz onthisapproximationofthecontoureventwhichhasknownprobability P D D k ,see FigureIII.4.Letting p ;i denotetheapproximationof P V i ,wecanthendenean approximationto P as P A P ;N A = N X i =1 p ;i A V i ;A 2B ; 23

PAGE 41

Wesummarizethisbasicprocedureforapproximating P inAlgorithm3. Algorithm3: NumericalApproximationoftheMeasure-TheoreticSolution 1.Let fV i g N i =1 B partition ,oftendenedbyasetofsamples i N i =1 2.Compute V i = V i for i =1 ;::;N 3.Determineanominalvalue q i forthemap Q on V i for i =1 ;::;N ,often q i = Q i 4.Chooseapartitioningof D f D k g M k =1 D 5.Compute P obs D D k for k =1 ;:::;M 6.Let C k = f i j q i 2 D k g for k =1 ;:::;M 7.Let O i = f k j q i 2 D k g for i =1 ;::;N 8.Set P V i = V i = P j 2C O i V j P obs D D O i for i =1 ;::;N InAlgorithm3, C k isusedtodeterminewhichsetsfrom fV i g N i =1 approximatethe contourevent Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 D k foreach k .Similarly, O i isusedtodeterminewhichcontour event Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 D k isassociatedto V i foreach i .TheAnsatzisappliedinthenalstep wheretheprobabilityofeach V i isdeterminedbytheprobability P obs D D O i multiplied bytheratioofthevolumeof V i tothevolumeoftheapproximatecontoureventof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 D O i Wenotethat,theprominentcostsinproducinganumericalapproximationto P are:theevaluationofthemodel,andthenearestneighborsearchesemployed inthevariousstepsofAlgorithm3,sometimesreferredtoasbinning.Dependingon theassumptionwemake,wemayreducethiscostincertainways,i.e.,givenaset ofi.i.d.randomsamplesdening fV i g N i =1 ,wemaymaketheMCassumptionthat V i = =N foreach i III.4ComparisonofMethods Abriefdiscussiononthedierenceinhoweachoftheabovemethodsdenes auniquesolutiontotheSIPiswarranted.Asnoted,thedenitionofafamilyof 24

PAGE 42

D FigureIII.3: WesummarizethestepsinAlgorithm3.toprowWedepictSteps 1-3.Wepartition with fV i g N i =1 thatareimplicitlydenedbythesetofsamples i N i =1 ,wemaketheMCassumptiontodeterminethevolumeofeach V i .Thenwe determinethenominalvalue q i = Q i .Here, N =2000.bottomrowWedepict Steps4-5.Wepartition D witharegulargridandprescribetheobservedprobablity measuretobeconstantoverthedarkenedsetof V i .Here M =100. 25

PAGE 43

D FigureIII.4: SummaryofAlgorithm3continued.toprowWedepictSteps5-6. Foreach k ,wedeterminewhichsetof i mapinto D k .Inred,weseetheunion ofthe V i correspondingtothe i thatmapinto D 1 seeninredin D .Weseethe samefor D 2 inblue.bottomrowWedepictStep8.Wecomputetheapproximate solutiontotheSIPwhereweproportionouttheprobabilityofeach D k determined bytheobservedmeasuretoitscorresponding V i .Darkercolorscorrespondtohigher probability. 26

PAGE 44

conditionalprobabilitymeasureson ; B denesauniquesolutiontotheSIP. TheconsistentBayesianmethodutilizesthedataspace D andpriorinformationon theparameterspacetodenethisfamilyofconditionalprobabilitymeasureswhich areultimatelydescribedintermsoftheircorrespondingprobabilitydensities,see Eq.III.11.Thismethodrequiresthepushforwardofthepriordensityandresults inanapproximationtothevalueof atanypoint 2 thatwehave Q .We maythenusethe i N i =1 toconstructanapproximationto .Alternatively, wemaygeneratesamplesfrom usinganaccept/rejectalgorithmwithoutactually constructing Themeasure-theoreticmethodutilizesthetransverseparameterization L ,implicitly,anddenesthefamilyofconditionalprobabilitymeasureson ; B basedon theunderlyingvolumemeasure .Thisdoes not requireanypriorinformationon theinputparameters.Thismethodrequiresvariousnearestneighborsearchesand resultsinanapproximationto P onadiscretizedparameterspace.Wemayuse thisapproximationto P toconstructanapproximationto andgeneratesamples from usinganaccept/rejectalgorithm.However,thesamplesusedtodiscretize andapproximate P neednotcomefromanunderlyingprobabilitymeasureand canbeplacedoptimallyusingonlyinformationaboutthemap Q anddataspace discretization,see[15]. III.5NumericalExample:Consistency Here,weconsideranaccessiblenonlinearsystemofequationstodisplaythe consistency ofthesolutiontotheSIP.Wenotethat,givenauniformprior,theconsistent Bayesiansolutionisidenticaltothemeasure-theoreticsolutionunderthestandard Ansatz[16].Inotherwords,thetwomethodsproducethesameconsistentsolutionto theSIPunderassumptionofauniformprior.Therefore,inthisexample,weutilize themeasure-theoreticalgorithmi.e.,Algorithm3,underthestandardAnsatz,for approximatingsolutionstotheSIP.ThefollowingconsistentsolutionstotheSIPare 27

PAGE 45

computedusingtheButler,Estep,TavenerBETPythonpackage,whichisdesigned tosolvemeasure-theoreticstochasticinverseandforwardproblems[32]. WeformulateandsolvethreeSIPsinordertobothillustratetheconsistency propertyofthesolutiontoagivenSIPandtomotivatetheremainingchapters. Considerthefollowing2-componentnonlinearsystemofequationsintroducedin[10]: 1 x 2 1 + x 2 2 =1 x 2 1 )]TJ/F20 11.9552 Tf 11.955 0 Td [( 2 x 2 2 =1 Theparameterrangesaregivenby 1 2 [0 : 79 ; 0 : 99]and 2 2 [1 )]TJ/F15 11.9552 Tf 13.182 0 Td [(4 : 5 p 0 : 1 ; 1+ 4 : 5 p 0 : 1]whicharechosenasin[10]toinduceaninterestingvariationinthesolution )]TJ/F20 11.9552 Tf 5.479 -9.683 Td [(x 1 ;x 2 .WeconsiderthreeQoImapsdenedbythesolutiontothissystem, Q = x 1 ; Q = x 2 ; Q = )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(x 1 ;x 2 : ThecorrespondingsolutionstotheSIPsaredenoted,respectively,as ; ; and Consider Q andassumethattheobserveddensityon Q isatruncatednormaldistributionwithmean1.015andstandarddeviationof0.007,seeFigureIII.5 bottom.Wegenerate1 E +5independentidenticallydistributedi.i.d.samples in ,usetheBETPythonpackagetoconstructanapproximationto P ,anda simpleaccept/rejectalgorithmtogenerateasetofsamplesfromtheresulting Wepropagatethissetofsamplesfrom throughthemodelandapproximatethe resultingpushforwarddensityusingaKDE,seeFigureIII.5.Thebottomimagein FigureIII.5displaysthe consistency propertyofthissolutiontotheSIP,i.e.,the 28

PAGE 46

pushforwardof matchestheobserveddensity. FigureIII.5: topleftApproximationof obtainedusing obs D on Q ,top rightasetofsamplesfrom ,bottomandacomparisonoftheobserveddensity obs D on Q withthepushforwardof Consider Q andassumethattheobserveddensityon Q isatruncatednormal distributionwithmean0.3andstandarddeviationof0.035.Weapproximate P usingthesame1 E +5samples,generateasetofsamplesfromtheresulting and propagatethesesamplesthroughthemodeltoapproximatethepushforwardof seeFigureIII.6.Again,weobservetheconsistencyof inthatthepushforwardof matchestheobserveddensity. Next,weconsiderusingthevector-valuedQoImap Q .Wetaketheobserved jointdensityon Q tobetheproductofthemarginaldensitieson Q and 29

PAGE 47

FigureIII.6: topleftApproximationof obtainedusing obs D on Q ,top rightasetofsamplesfrom ,bottomandacomparisonoftheobserveddensity obs D on Q withthepushforwardof 30

PAGE 48

Q asspeciedabove.Again,weapproximate P andgenerateasetofsamples fromtheresulting usingthesame1 E +5samples,seeFigureIII.7.Withthe informationfromboth x 1 and x 2 ,weseeasubstantialdecreaseinthesupport of .Intuitively,withtheassumptionsabove,weexpectthat supp supp supp : FigureIII.7: leftTheapproximationof usingthejointdensity obs D on Q rightasetofsamplesfrom Wepropagatethesetofsamplesfrom throughthemodelandcalculatethe meansandstandarddeviationsoftheresultingsetofsamplestoconstructthepushforwardof .InFigureIII.8,weshowthepushforwardof ,theobserveddensity, andtheerrorbetweenthetwoscaledbythemaximumoftheobserveddensitytodisplayarelativeerror. Inthescenarioinwhichwegatherdataleadingtoanobserveddensityonboth Q and Q ,comparedtothescenarioswhereweonlygatherdataon Q or Q butnotboth,thereisaclearreductionofuncertaintiesasisevidencedbythereduced supportin anditsoverallstructure.However,supposewecouldonlygatherdata ononeoftheseQoIinanexperimental/eldsetting.Choosingeither Q or Q 31

PAGE 49

FigureIII.8: topleftThepushforwardof ,toprighttheexactobserved density,bottomthedierencebetweenthepushforwadof andtheobserved densityscaledbythemaximumoftheobserveddensity. 32

PAGE 50

denesaseparateED.ItisnotimmediatelyclearfromthegureswhichEDismore useful. Intheremainingchapters,weseektoquantifythe utility ofanEDunderconsiderationinordertodeterminetheoptimalQoImaptobeusedinformulatingand solvingaSIP.InChapterIV,wequantifytheexpectedutilityofagivenEDusingthe KLdivergence.InChapterV,wequantifytheexpectedutilityofanEDintermsof localgeometricpropertiesofthecorrespondingQoImapthatdescribetheexpected precisionandaccuracyoftheresultingsolutiontoaSIP.InChapterVI,weprovide anindepthcomparisonbetweenthesetwoSB-OEDmethodsfocusedspecicallyona handfulofchallengingSB-OEDscenarios.InChapterVII,wecontinuetoextendthe measure-theoreticapproachtoSB-OEDbycontributingfurthertothequantication ofthecomputationalcomplexityofaSIP.InChapterVIII,weillustrateagreedy implementationofthemeasure-theoreticapproachtoSB-OEDinahigh-dimensional setting. 33

PAGE 51

CHAPTERIV CONSISTENTBAYESIANOPTIMALEXPERIMENTALDESIGN Inthischapter,wedevelopaSB-OEDapproachbasedontheconsistentBayesian methodforsolvingSIPs.WefollowtheBayesianmethodologyforSB-OEDdiscussed inChapterIIand[35,36,9].ThismethodologyquantiestheutilityofagivenED denedbyaQoImap Q basedonasetofdensitiescorrespondingtosolutionsof SIPsfor Q which,inturn,arebasedonthetypesofdatathatweexpecttoobserve intheeldforthegiven Q .SincetheconsistentBayesianmethodforsolvingSIPs hasdierentassumptionsfromtheclassicalBayesianmethod,wemodifytheclassical BayesianSB-OEDmethodologyasneeded. InSectionIV.1,weconsidertheinformationcontentofagivenEDanddenethe OEDproblem.InSectionIV.2,weaddresstheissueofinfeasibledataanddiscuss computationalconsiderations.WeprovidenumericalexamplesinSectionIV.3to displaytheecacyofthismethodontwotoyproblemsforwhichwehavesignicant intuitionregardingthelocationoftheOED. IV.1TheInformationContentofanExperimentalDesign Conceptually,anEDisconsideredinformativeiftheresultingsolutiontotheSIP issignicantlydierentfromthepriorprobabilitymeasure.Toquantifythe informationgain ofanEDweusetheKLdivergence[64],whichisameasureofthedierence betweentwoprobabilitydensities.WhiletheKLdivergenceisbynomeanstheonly waytocomparetwoprobabilitydensities,itdoesprovideareasonablemeasureofthe informationgainedinthesenseofShannoninformation[22]andiscommonlyusedin BayesianOED[35,9].Inthissection,wediscusscomputationoftheKLdivergence anddeneanOEDformulationbasedupontheexpectedinformationgainovera speciedspaceofprobableobserveddensities. IV.1.1InformationGain:Kullback-LeiblerDivergence 34

PAGE 52

Supposewearegivenadescriptionoftheuncertaintyontheobserveddatafor aQoImap Q intermsofaprobabilitymeasure P obs D thatcanberepresentedasa density obs D .Inotherwords, P obs D isabsolutelycontinuouswithrespectto D and obs D istheRadon-Nikodymderivativeof P obs D .AspresentedinChapterIII,givenprior informationabouttheparametersdescribedasaprobabilitymeasure P prior ,theSIP hasauniquesolutiongivenbyaprobabilitymeasure P thatisabsolutelycontinuous withrespectto ,andwelet denotetheprobabilitydensity.TheKLdivergence, i.e.,informationgain,of from prior ,denoted KL Q toemphasizetherole Q plays indetermining ,isgivenby KL Q prior : := Z log prior d : IV.1 Since prior isxedforagivenED, KL Q issimplyafunctionof .However,from Eq.III.12,weseethat issimplyafunctionof obs D .Therefore,tomakethis dependenceexplicit,wewrite KL Q asafunctionoftheobserveddensity, KL Q obs D := KL Q prior : : IV.2 Theobservationthat KL Q isafunctionofonly obs D iscriticaltodeningtheexpectedinformationgainonawell-denedspaceofobserveddensities,asdonein SectionIV.1.2. ExampleIV.1InformationGain Considerthefollowing2-componentnonlinearsystemofequationswithtwoparametersdiscussedinSectionIII.5, 1 x 2 1 + x 2 2 =1 ; x 2 1 )]TJ/F20 11.9552 Tf 11.956 0 Td [( 2 x 2 2 =1 : 35

PAGE 53

Recallthattheparameterrangesaregivenby 1 2 [0 : 79 ; 0 : 99] and 2 2 [1 )]TJ/F15 11.9552 Tf -422.702 -23.98 Td [(4 : 5 p 0 : 1 ; 1+4 : 5 p 0 : 1] .Asbefore,weconsiderthreeQoImapsdenedbythesolutiontothissystem, Q = x 1 ; Q = x 2 ; Q = )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(x 1 ;x 2 : ThecorrespondingsolutionstotheSIPsaredenoted,respectively,as ; ; Consider Q andassumetheobserveddensityon Q isatruncatednormal distributionwithmean0.3andstandarddeviationof0.01,seeFigureIV.1.Wegenerate40,000samplesfromtheuniformpriorandconstructanapproximationtothe pushforwardofthepriorwithAlgorithm1,seeFigureIV.1right.Then,weuse Algorithm2toconstructanapproximationto usingthesame40,000samples,see FigureIV.1left.Noticethesupportof liesinarelativelylargeregionoftheparameterspace.Theinformationgainfrom ,approximatedusingEquation IV.1 is KL Q obs D 0 : 466 Next,weconsider associatedwithsolvingtheSIPusing Q .Weassume theobserveddensityon Q isatruncatednormaldistributionwithmean1.015and standarddeviationof0.01.Weapproximatethepushforwardofthepriorand withAlgorithm1andAlgorithm2,usingthesame40,000samples,seeFigureIV.2. Althoughtheobserveddensitieson Q and Q havethesamestandarddeviation,clearly and aredierent.Visually, hasamuchlargerregionof supportwithintheparameterspacecomparedto .Thisisquantiedwiththeinformationgainfrom KL Q obs D 2 : 015 .Giventhesetwomaps, Q and Q ,andthespeciedobserveddensitiesoneachofthesedataspaces,theexperiment leadingto Q andthecorresponding obs D ismoreinformativethantheexperiment 36

PAGE 54

FigureIV.1: leftApproximationof obtainedusing obs D on Q ,whichgives KL Q obs D 0 : 466.right obs D on Q andthepushforwardoftheprior. FigureIV.2: leftApproximationof obtainedusing Q and obs D on Q whichgives KL Q obs D 2 : 015.right obs D on Q andthepushforwardofthe prior. 37

PAGE 55

leadingto Q andthecorresponding obs D Next,weconsiderusingthevector-valuedQoImap Q .Wetaketheobserved jointdensityon Q tobetheproductofthemarginaldensitieson Q and Q asspeciedabove.Again,weapproximatethepushforwardofthepriorand using thesame40,000samples,seeFigureIV.3.Withthejointinformation,weseea substantialdecreaseinthesupportof .Thisisquantiedintheinformationgain ofthis KL Q obs D 2 : 98 .Thus,weseethelargestinformationgainbyusing Q andthejointdensityobservedon Q .However,inpractice,wemaybelimited tospecifyasingleQoIforwhichwecancollectdatatoconstructanobserveddensity. Basedonthecomputedinformationgains,wemayconcludethat Q ismore informativeoftheparametersthan Q .However,duetononlinearitiesofthemaps, itisnotnecessarilytruethat Q ismoreinformativethan Q ifdierentobserved densitiesarespecied.Sincewedonotknowtheobserveddensitiespriortocollecting actualdata,wearelefttodeterminewhichoftheseQoIwe expect toproducethe mostinformative IV.1.2ExpectedInformationGain OEDmustselectanED before experimentaldatabecomesavailable.Inthe absenceofdata,weusethecomputationalmodeltoquantifythe expected information gainEIGofagivenEDdenedbyaQoImap Q .Given Q and D = Q ,let O D denotethespaceofallprobabilitydensityfunctionsover D .DeningtheEIG over O D iscomputationallyimpossiblesince O D containsuncountablyinnitelymany densities.Moreover, O D includesdensitiesthatareunlikelytobeobservedinpractice foragiven Q .Therefore,werestrict O D tobeaspacemorerepresentativeofdensities thatmaybeobservedinpracticeforaparticularEDdenedby Q Withnoexperimentaldataavailabletospecifyanobserveddensityon D fora givenQoImap Q ,weassumetheobserveddensityisaGaussianwithmeanbelonging to D andadiagonal m m covariancematrix,i.e.,weassumetheuncertaintiesin 38

PAGE 56

FigureIV.3: topleftTheapproximationofthepushforwardoftheprior.top rightTheexact obs D on Q .bottomTheapproximationof using obs D on Q whichgives KL Q obs D 2 : 98. 39

PAGE 57

eachQoIareuncorrelated.ForagivenQoImap Q ,welet O Q denotethespaceofall densitiesofthisprolewithmeanbelongingto D = Q O Q = D 2O D : D N ; ; 2D : IV.3 Thus, D isassumedtobeaGaussiandensityfunctionwithmean 2D anddiagonal covariancematrix Wemayrestrict O Q inotherwaysaswell.Forexample,wemayexpectthe uncertaintyineachQoItobedescribedbyauniformdensityofagivenwidth,in whichcasewewoulddenetherestrictionon O Q accordingly.Thischoiceofprole fortheobserveddensitieswithin O Q islargelydependentontheapplicationand specicationof Q .Theonlylimitationontheobserveddensityspacecomesfromthe pushforwardmeasureonthedataspace, P Q prior D ,asdescribedbelow. Therestrictionofpossible obs D tothisspecicspaceofdensitiesallowsustoindex eachdensitybyaunique q 2D .Basedonourpriorknowledgeoftheparametersand thesensitivitiesofthemap Q ,themodelinformsusthatsomedataaremorelikely tobeobservedthanotherdata,thisisseenintheplotof Q prior D inFigureIV.3. Therefore,weaverageover D withrespecttothepushforwardoftheprior.This utilizesboththepriorknowledgeoftheparametersandthesensitivityinformation providedbythemodel.WedenetheEIG,denoted E KL Q ,as E KL Q := Z D KL Q q Q prior D q d D : IV.4 FromEq.IV.1, KL Q itselfisdenedintermsofanintegral.Theexpandedform for E KL Q isthenaniteratedintegral, E KL Q = Z D Z ; q log ; q prior d dP Q prior D ; IV.5 40

PAGE 58

wherewemakeexplicitthat isafunctionoftheobserveddensity,and,bythe restrictionofthespaceofobserveddensitiesinEq.IV.3,isthereforeafunctionof q 2D .WeutilizeMonteCarlosamplingtoapproximatetheintegralinEq.IV.4as describedinAlgorithm4. Algorithm4: ApproximatingtheEIGofanED 1.Givenasetofsamplesfrom : i i =1 ;:::;N ; 2.Givenasetofsamplesfromthepushforwardoftheprior: q j j =1 ;:::;M ; 3.Giventhevaluesof ; q j at: i i =1 ;:::;N ; 4.For j =1 ;:::;M approximate KL Q q j : KL Q q j N N X i =1 i ; q j log i ; q j prior i 5.Approximate E KL Q : E KL Q 1 M M X j =1 KL Q q j RemarkIV.1 InStep1ofAlgorithm4,weassumewearegivenasetofsamples fromthevolumemeasure ontheparameterspace.Assumingthisvolumemeasure istheLebesguemesure,thengivenauniformprior,thissetofsamplesisequivalentto asetofsamplesfromthisprior.Inthisthesis,wefocusprimarilyonapplicationsin whichwehaveauniformpriorand,hence,usethesamplesfromthepriorinStep1of Algorithm4.Givenanapplicationwithanonuniformprior,wehavetworemedies: considerasetofsamplesfromtheposteriorinStep1andmodifytheintegralin Eq. IV.1 tobewithrespectto P ,orapproximateEq. IV.1 intermsofthevalues oftheobserveddensityandpushforwardofthepriordensitywhichallowsintegration withrespecttothepriormeasureontheparameterspace.Thistopicisbeyondthe focusofthisthesis;theinterestedreaderisreferredto[66]foracompletediscussion 41

PAGE 59

onthistopic. Algorithm4appearstobeacomputationallyexpensiveproceduresinceitrequiressolving M separateSIPsand,asnotedin[16],approximating Q prior D canbe expensive.However,weonlyneedtocomputethisapproximationonce,aseach KL Q inStep4ofAlgorithm4iscomputedusingthesamepriorandmap Q and,therefore, thesame Q prior D .Inotherwords,thefactthattheconsistentBayes'methodonly requiresapproximatingthepushforwardofthepriorimpliesthatthisinformationcan beusedtoapproximate 'sfordierentobserveddensitieswithoutrequiringadditionalmodelevaluations.Thissignicantlyimprovesthecomputationaleciencyof theconsistentBayesianapproachinthecontextofSB-OED. IV.1.3DeningtheOED Givenadeterministicmodel,priorinformationonthemodelparameters,aspace ofpotentialEDs,andagenericdescriptionoftheuncertaintiesforeachQoI,wedene theOEDasfollows. DenitionIV.2OED Let Q representthedesignspace,i.e.,thespaceofallpossibleEDs,and Q 2Q beaspecicED.Then,theOEDisthe Q 2Q thatmaximizes theEIG, Q opt :=argmax Q 2Q E KL Q : IV.6 Ingeneral,thedesignspace Q ,caneitherbeacontinuousordiscretespace.Inthis chapter,ourfocusisontheutilizationoftheconsistentBayesianmethodologywithin theOEDframework,hence,wesimplyndtheOEDoveradiscretesetofcandidate EDsthatmaycomefromasamplingofacontinuousdesignspace. IV.2InfeasibleData TheOEDprocedureproposedaboveisbaseduponconsistentBayesianinference whichrequiresthatthepushforwardmeasure,inducedbythepriorandthemodel, 42

PAGE 60

dominates theobservedmeasure,thatis,anyeventthatweobservewithnon-zero probabilitywillbepredictedusingthemodelandpriorwithnon-zeroprobability, seeSectionIII.2.Duringtheprocessofcomputing E KL Q ,itispossiblethatwe violatethisassumption.Specically,dependingonthemeanandvarianceofthe observationaldensity,wemayencounter D 2O Q suchthat R D D d D < 1,i.e.,the supportof D extendsbeyondtherangeofthemap Q ,seeFigureIV.5upperright. Werefertoobserveddatathatliesoutsidethedataspace D as infeasibledata .Here, wediscusspotentialcausesofinfeasibledataandoptionsfordealingwithinfeasible datawhenestimatinganOED. IV.2.1InfeasibleDataandConsistentBayesianInference WheninferringmodelparametersusingconsistentBayesianinference,acommon causeforinfeasibledataisthatthemodelbeingusedtoestimatetheOEDisinadequate.Thatis,thedeviationbetweenthecomputationalmodelandrealityislarge enoughtoprohibitthemodelfrompredictingalloftheobservationaldata.Thedeviationbetweenthemodelpredictionandtheobservationaldataisoftenreferredtoas modelstructureerrorandcanoftenbeamajorsourceofuncertainty.Thisisanissue formost,ifnotall,inverseparameterestimationproblems[38].Recentlytherehas beenanumberofattemptstoquantifythiserror[58].Suchapproachesarebeyond thescopeofthisthesis.Inthefollowing,weassumethatthemodelstructureerror doesnotpreventthemodelfrompredictingalltheobservationaldata. IV.2.2InfeasibleDataandOED When D isapropersubsetof R m ,thedensitiesin O Q givenbyEq.IV.3may produceinfeasibledata.Theeectofthisviolationincreasesas approachesthe boundaryof D .Toremedythisviolation,wemustmodify O Q .Wechoosetotruncate andnormalize D 2O Q sothatsupp^ D D ,where^ D isthetruncateddensity. 43

PAGE 61

Wedenethisobserveddensityspace ^ O Q accordingly, ^ O Q = ^ D = D C : D 2O Q ;C = Z D D D : IV.7 ExampleIV.2InfeasibleData Inthisexample,weusethenonlinearmodel fromSectionIII.5todemonstratethatinfeasibledatacanarisefromrelativelybenign assumptions.Supposetheobserveddensityon Q isatruncatednormaldistribution withmean0.982andstandarddeviationof0.01.Inthisone-dimensionaldataspace, thisobserveddensityisabsolutelycontinuouswithrespecttothepushforwardofthe prioron Q ,seeFigureIV.4left.Next,supposetheobserveddensityon Q isa truncatednormaldistributionwithmean0.3andstandarddeviationof0.04.Again, inthisone-dimensionaldataspace,thisobserveddensityisabsolutelycontinuouswith respecttothepushforwardoftheprioron Q ,seeFigureIV.4right.Bothofthese observedensitiesaredominatedbytheircorrespondingpushforwarddensities,i.e.,the modelcanreachalloftheobserveddata ineachcase FigureIV.4: leftThepushforwardofthepriorandobserveddensitieson Q rightThepushforwardofthepriorandobserveddensitieson .Noticethesupport ofeachobserveddensityiscontainedwithintherangeofthemodel,i.e.,theobserved densitiesareabsolutelycontinuouswithrespecttotheircorrespondingpushforward densities. 44

PAGE 62

However,considerthedataspacedenedby Q = )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(x 1 ;x 2 andthecorrespondingpushforwardandobserveddensitiesonthisspace,seeFigureIV.5.The non-rectangularshapeofthecombineddataspaceisinducedbythenonlinearityinthe modelandthecorrelationsbetween x 1 and x 2 .AsweseeinFigureIV.5,the observeddensityusingtheproductofthe1-dimensionalGaussiandensitiesisnotabsolutelycontinuouswithrespecttothepushforwarddensityon Q ,i.e.,thesupportof obs D extendsbeyondthesupportof Q prior D .ReferringtoEq. IV.7 ,wenormalizethis observeddensityover D ,seeFigureIV.5bottom.Nowthatthenewobserveddensity obeystheassumptionsneeded,wecansolvetheSIPasdescribedinSectionIII.2. IV.2.3ComputationalConsiderations Next,weaddressthecomputationalissueofnormalizing obs D over D .Fromthe plotof Q prior D inFigureIV.5topleft,itisclearthedataspacemayhaveacomplex shapeandbenon-convex.Hence,normalizing obs D ,asinFigureIV.5bottom,over D maybecomputationallyexpensive.Fortunately,theconsistentBayesianapproach providesameanstoavoidthisexpense.NotethatfromEq.III.9wehave, P = P prior P obs D Q P Q prior D Q ; IV.8 where P prior = P Q prior D Q =1whichimplies, P = P obs D Q : IV.9 Therefore,normalizing obs D over D isequivalenttosolvingtheSIPandthennormalizing~ wherethetildeindicatesthisfunctionmaynotintegrateto1over D over .Although maynotalwaysbeageneralizedrectangle,ourpreviousassumption thatwehaveadescriptionofthepriorknowledgeoftheparametersimplieswe haveacleardenitionof ,andwecanthereforeecientlyintegrate~ over and 45

PAGE 63

FigureIV.5: topleftThepushforwardofthepriorforthemap Q .topright Theobserveddensityusingthejointinformationwhichnowextendsbeyondtherange ofthemap.bottomThenormalizedobserveddensitythatdoesnotextendbeyond therangeofthemap. 46

PAGE 64

normalize~ by = ~ R ~ d : IV.10 IV.3NumericalExamples Inthissection,weconsidertwomodelsofphysicalsystems.First,weconsider astationaryconvection-diusionmodelwithasingleuncertainparameterdening themagnitudeofthesourceterm.Then,weconsideratime-dependentdiusion modelwithtwouncertainparametersdeningthethermalconductivitiesofametal rod.Ineachexample,wehaveaparameterspace ,asetofpossibleQoI,anda speciednumberofQoIforwhichwecangatherdatainanexperiment.Thisinturn denesadesignspace Q ,andwelet Q 2Q representasingleEDand D = Q the correspondingdataspace.ForeachED,welet O Q denotetheassociatedobserved densityspace. Thefollowingtwoexampleshavecontinuousdesignspaces.WechoosetoapproximatetheOEDbyselectingitfromalargesetofcandidatedesigns.Thisapproach waschosenbecauseitismuchmoreecienttoperformtheforwardpropagationof uncertaintyusingrandomsamplingonlyonceandtocomputeallofthecandidate measurementsforeachoftheserandomsamples.Alternatively,onecouldpursuea continuousoptimizationformulationwhichwouldrequireafullforwardpropagation ofuncertaintyforeachnewdesign. ExampleIV.3StationaryConvection-Diusion:UncertainSourceAmplitude Here,weconsiderastationaryconvection-diusionproblemwithasingleuncertain parametercontrollingthemagnitudeofasourceterm.ThisexampleservestodemonstratethattheOEDformulationcangiveintuitiveresultsforsimpleproblems. Problemsetup 47

PAGE 65

Considerastationaryconvection-diusionmodelonasquaredomain: 8 > > > > > > < > > > > > > : )]TJ/F20 11.9552 Tf 9.299 0 Td [(D r 2 u + r vu = S;x 2 ; r u n =0 ;x 2 )]TJ/F21 7.9701 Tf 7.314 -1.793 Td [(N @ ; u =0 ;x 2 )]TJ/F21 7.9701 Tf 7.314 -1.793 Td [(D @ ; IV.11 with S x = A exp )]TJ 13.15 8.088 Td [(jj x src )]TJ/F20 11.9552 Tf 11.956 0 Td [(x jj 2 2 h 2 ; where =[0 ; 1] 2 u istheconcentrationeld,thediusioncoecient D =0 : 01 ,the convectionvector v =[1 ; 1] ,and S isaGaussiansourcewiththefollowingparameters: x src isthelocation, A istheamplitude,and h isthewidth.Weimposehomogeneous Neumannboundaryconditionson )]TJ/F21 7.9701 Tf 7.315 -1.793 Td [(N rightandtopboundariesandhomogeneous Dirichletconditionson )]TJ/F21 7.9701 Tf 7.314 -1.794 Td [(D leftandbottomboundaries.Forthisproblem,wechose x src =[0 : 5 ; 0 : 5] ,and h =0 : 05 .Welet A beuncertainwithin [50 ; 150] .Thus,the parameterspaceforthisproblemis =[50 ; 150] Givenasourceamplitude A ,weapproximatesolutionstothePDEinEq. IV.11 usinganiteelementdiscretizationwithcontinuouspiecewisebilinearbasisfunctions denedonauniform 25 25 spatialgrid. Results Assumethatwehaveonlyonesensortoplaceinthespatialdomaintogathera singleconcentrationmeasurement.Inotherwords,thereisabijectionbetween and Q sopoints x 0 ;x 1 2 canbeusedtoindextheEDs.Thegoalistoplacethissingle sensorin tomaximizetheEIGabouttheamplitudeofthesource.Wediscretizethe continuousdesignspace Q using2,000uniformrandompointsin f x k 0 ;x k 1 g 2000 k =1 whichcorrespondsto2,000EDsdenotedby f Q k g 2000 k =1 where Q k = u x k 0 ;x k 1 .We assumetheuncertaintyinanymeasurementforeachpossibleQoIisdescribedbya truncatedGaussianprolewithaxedstandarddeviationof0.1.Thisproduces 2 ; 000 48

PAGE 66

observeddensityspaces, fO Q k g 2000 k =1 ,asdescribedinEq. IV.7 Wegenerate5,000samples, f i g 5000 i =1 ,fromtheuniformprioron andcompute f Q k i g 5000 i =1 2000 k =1 .Wecalculate f E KL Q k g 2000 k =1 usingAlgorithm4andplot E KL Q asafunctionoftheindexingsetdenedby inFigureIV.6.Noticethe EIGisgreatestwhentheEDisdenedbymeasurementstakennearthecenterofthe domainnearthelocationofthesourceandinthedirectionoftheconvectionvector awayfromthesource.Thisresultmatchesintuition,asweexpectdatagatheredin regionsofthedomainthatexhibitsensitivitytotheparameterstoproducehighEIG values. FigureIV.6: SmoothedplotoftheEIGovertheindexingsetforthedesign space Q .Noticethehighervaluesinthecenterofthedomainandtowardsthetop rightinthedirectionoftheconvectionvectorfromthelocationofthesource,this isconsistentwithourintuition. ExampleIV.4Time-DependentDiusion:UncertainDiusionCoecients Here,weconsideratime-dependentdiusionproblemwithtwouncertainparameters deningthethermalconductivitiesofametalrod. Problemsetup 49

PAGE 67

Considerthetime-dependentdiusionequation: 8 > > > > > > < > > > > > > : c @u @t = r r u + S;x 2 ;t 2 t 0 ;t f ; @u @n =0 ;x 2 @ ; u x ;0=0 ; IV.12 with S x = A exp )]TJ 13.15 8.088 Td [(jj x src )]TJ/F20 11.9552 Tf 11.955 0 Td [(x jj 2 w ; where =[0 ; 1] representsanalloyrod, =1 : 5 isthedensityoftherod, c =1 : 5 is theheatcapacity, isthethermalconductivitywhichisuncertaininthisproblem, and S isaGaussiansourcewiththefollowingsourceparameters: x src =[0 : 5] isthe location, A =50 istheamplitude,and w =0 : 05 isthewidth.Welet t 0 =0 t f =1 andimposehomogeneousNeumannboundaryconditionsoneachendoftherod. Supposetherodismanufacturedbyweldingtogethertworodsofequallengthand ofsimilaralloytype.However,duetothemanufacturingprocess,theactualalloy compositionsmaydierslightlyleadingtouncertainthermalconductivityproperties ontheleft-halfandright-halfoftherod,denotedby 1 and 2 ,respectively.Specifically,welet =[0 : 01 ; 0 : 2] 2 Giventhethermalconductivityvaluesforeachsideoftherod,weapproximate solutionstothePDEinEq. IV.12 usingthestate-of-the-artopensourceniteelementsoftwareFEniCS[3,47]witha 40 40 triangularmesh,piecewiselinearnite elements,andaCrank-Nicolsontimesteppingscheme. Results Supposetheexperimentsinvolveplacingtwocontactthermometersalongthefull rodthatcanrecordtwoseparatetemperaturemeasurementsat t f .Thus,thereisa bijectionbetween and Q sopoints x 0 ;x 1 2 canbeusedtoindextheEDs. Thegoalistoplacethetwosensorsin tomaximizetheEIGabouttheuncertain 50

PAGE 68

thermalconductivities.Wediscretizethecontinuousdesignspace Q using20,000 uniformrandompointsin f x k 0 ;x k 1 g 20000 k =1 ,whichcorrespondsto20,000EDs denotedby f Q k g 20000 k =1 ,where Q k = )]TJ/F20 11.9552 Tf 5.479 -9.683 Td [(u x k 0 ;t f ;u x k 1 ;t f foreach k .Weassume theuncertaintyinanymeasurementforeachpossibleQoIisdescribedbyatruncated Gaussianprolewithaxedstandarddeviationof0.1.Thisproducesobserveddensity spaces, fO Q k g 20000 k =1 ,asdescribedinEq. IV.7 Wegenerate5,000samples, f i g 5000 i =1 ,fromtheuniformprioron andcompute f Q k i g 5000 i =1 20000 k =1 .Wecalculate f E KL Q k g 20000 k =1 usingAlgorithm4andplot E KL Q asafunctionoftheindexingset for Q inFigureIV.7.Wenotethat, duetothesymmetrywithin alongtheline x 0 = x 1 ,weonlyshowtheEIG overhalfoftheindexingset becausetheotherhalfcorrespondstoarenaming ofcontactthermometeroneascontactthermometertwoandviceversa.Moreover, E KL Q isclearlynonconvexandsubstantiallymorecomplicatedthantheresults observedinExampleIV.3.WelabelthefournotablelocalmaximaoftheEIGas R1,R2,R3,andR4.Placingthecontactthermometerssymmetricallyawayfromthe midpoint{neartheendsoftherodR1{canleadtooptimaldesigns,whichmatches physicalintuitionseethelowerrightcornerofFigureIV.7.Inthetableintheright FigureIV.7,weshowthelocalOEDlocationsandEIGvaluesforeachofthefour regionslabeledinFigureIV.7.TheglobalOEDislocatedinR1. Inthefollowingchapter,weproposeanOEDformulationbasedsolelyonlocallinearapproximationsofQoImaps.Inthisway,weprovideadescriptionof theexpectedinformationgain"ofanEDindependentoftheuncertaintiesonthe observeddata.Moreover,themethoddescribedinthefollowingchapterallowsfor theconsiderationofoptimizationobjectivesotherthanexpectedinformationgain". Inparticular,weareinterestedintheaccuracyofthesolutiontoagivenSIP.We consideramulti-objectiveoptimizationprobleminwhichweoptimizeboththeexpectedinformationgain"andtheexpectedaccuracyoveradesignspace.Weprovide 51

PAGE 69

RegionOptimalEIG R14.909 R24.769 R34.729 R44.269 FigureIV.7: leftSmoothedplotoftheEIGovertheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof .This locationcorrespondstoplacingthermometersnearseparateendsoftherod.However, weobservemultiplelocalmaximawithinthedesignspace,denotedR1,R2,R3and R4.rightTheEIGforthelocalOEDineachofthefourregions. athoroughcomparisonofthetwoSB-OEDmethodsinChapterVI. 52

PAGE 70

CHAPTERV MEASURE-THEORETICOPTIMALEXPERIMENTALDESIGN Inthischapter,wefocusontwomajorgoals:aquanticationofhowamap Q impactstheaccuracyofthesolutiontoagivenSIP[14],andadenitionofOEDindependentoftheuncertaintiesonthedataspace.Therefore,wedonotsolveanySIPs duringthesectionsleadingtothedenitionoftheOED,andwedonotspecifyuncertaintiesonthedataspace.Weconsiderlocallinearapproximationsofagivenmap Q deninganED,andquantifytwodrivinggeometricpropertiesofinverseimages:the expected skewness andexpected scalingeect [17].Theexpectedskewnessdescribes theexpectedaccuracyofthesolutiontoagivenSIP,andtheexpectedscalingeect describestheexpectedprecisionofthesolutiontoagivenSIP.Thisexpectedscaling eectcanbethoughtofasatypeofquanticationoftheexpectedinformationgained bysolutionofagivenSIP.Furthercomparisonbetweentheexpectedscalingeect andtheEIGcanbefoundinChapterVI. WeintroducenotationinSectionV.1.InSectionV.2,weintroducelinearalgebra resultsneededtoecientlyapproximatethelocalskewnessandlocalscalingeect ofagivenQoImap.InSectionsV.3andV.4,wedeneandprovideameansfor approximatingtheexpectedskewnessandtheexpectedscalingeect,respectively.We denetheOEDinSectionV.5andprovideseveralnumericalexamplesinSectionV.6. RemarkV.1 In[14],thecomputationalcomplexityofagivenSIPwascharacterized intermsoflocallinearpropertiesoftheQoImapunderconsideration.However, thisworkdidnotprovideacomputationallyecientmethodforapproximatingthe expectedcomputationalcomplexitynorprovideanoptimalityobjectivefordescribing theexpectedprecisionofthesolutiontoaSIP. V.1Notation Welet Q denotethespaceofallpiecewisedierentiableQoImaps,i.e.,thespace ofEDs,underconsiderationbythemodelerforcollectingdatatosolveastochastic 53

PAGE 71

inverseproblem.Forexample,supposethemodelerisconsideringtwodierentEDs thatleadtoQoImaps, Q a : !D a ; or Q b : !D b : Then, Q = Q a ;Q b : Foranymap Q 2Q ,weareinterestedinhowthismapaectsthelocalgeometric propertiesrelatedtotheshapespecicallytheskewnessandscalingofthevolume measureoftheinverseofanoutputevent E 2B D .Conceptually,theskewness of Q describesageneralincreaseingeometriccomplexityof E under Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,which quantitativelyisrelatedtothenumberofsamplesin requiredtoaccuratelyestimate Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Thelocalscalingeectof E bythemap Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 describestheprecision ofusingthemap Q toidentifyparametersthatmapto E .InSectionsV.3andV.4, weshowthatboththelocalskewnessof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andthelocalscalingof E bythe map Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 canbedescribedintermsofsingularvaluesofaJacobianof Q .Welet J ;Q 2 R d n denotetheJacobianof Q atapoint 2 ,andwhentheQoImapbeing consideredisclearfromcontext,wesimplyuse J Forsimplicityindescribingsets,weignoreanyboundariesof or D .However,in practice,weassumethat iscompact,which,bytheassumedsmoothnessof Q ,gives that D isalsocompact.Inotherwords,weassumethat and D D arenite, whichisoftenthecaseinpracticebytheintroductionofknownorassumedbounds ofparameteranddatavalues.Second,sincetheskewnessandscalingpropertiesof Q mayvarysignicantlythroughout when Q isnonlinear,wemustaccountforthis variabilityindeterminingoptimalQoI. V.2 d -dimensionalParallelepipeds Determiningthesizeof d -dimensionalparallelepipedsembeddedin n -dimensional 54

PAGE 72

spaceisfundamentaltoecientlyapproximatinglocalskewnessandscalingeectsof agivenmap Q .Weareinterestedintwocases: d -dimensionalparallelepipedsdened bytherowsofagivenmatrix J ,and d -dimensionalparallelepipedsdeterminedby thecrosssectionsof n -dimensionalcylindersthataredenedbythepre-imageofa d dimensionalunitcubeunder J .Thefollowingtechnicallemmaandensuingcorollary describeamethodfordeterminingthesizesoftheseobjectsbasedonthesingular valuesofJacobiansof Q LemmaV.2 Let J beafullrank d n matrixwith d n ,and Pa J denotethe d -dimensionalparallelepipeddenedbythe d rowsof J .TheLebesguemeasure d in R d of Pa J isgivenbytheproductofthe d singularvalues f k g d k =1 of J ,i.e., d Pa J = d Y k =1 k : V.1 Proof: Thesingularvaluesof J areequaltothesingularvaluesof J > .Consider thereducedQRfactorizationof J > J > = ~ QR; V.2 where ~ Q is n d and R is d d .BythepropertiesoftheQRfactorization,weknow thesingularvaluesof R arethesameasthesingularvaluesof J > .Let x 2 R d ,then jj ~ Q x jj 2 = ~ Q x > ~ Q x = x > ~ Q > ~ Q x = x > x = jj x jj 2 ; V.3 so ~ Q isanisometry.ThisimpliestheLebesguemeasureoftheparallelepipeddened bytherowsof R isequaltotheLebesguemeasureoftheparallelepipeddenedby thecolumnsof J > ,ortherowsof J d Pa J = d Pa R = d Y k =1 k = d Y k =1 k ; V.4 55

PAGE 73

where f k g 1 k d arethesingularvaluesof R and f k g 1 k d arethesingularvalues of J Wenowturnourattentiontothesecondcaseofdescribingthesizeofa d dimensionalparallelepipeddeterminedbythecrosssectionofan n -dimensionalcylinderdenedbythepre-imageofa d -dimensionalunitcubeunder J .Inthiscase,we considerthepseudo-inverseof J J + = J > JJ > )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .Asisevidentfromtheformula ofthepseudo-inverse,therangeof J + isequaltotherowspaceof J .Thisimplies thatthe d -measureofthecross-sectionofthepre-imageofaunitcubeunder J is equaltothethe d -measureoftheparallelepipeddenedbythecolumnsof J + CorollaryV.3 Let J beafullrank d n matrixwith d n .Then Pa J + > isa d dimensionalparallelepipeddeningacross-sectionofthepre-imageofa d -dimensional unitcubeunder J anditsLebesguemeasure d isgivenbytheinverseoftheproduct ofthe d singularvalues f k g d k =1 of J ,i.e., d Pa J + > = d Y k =1 k )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 : V.5 Proof: Considerthepseudo-inverseof J J + = J > JJ > )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 : Fromthisequation,itisclearthatthecolumnspaceof J + isequaltotherowspace of J .Therowspaceof J denesasubspaceorthogonaltothe n -dimensionalcylinder thatisthepre-imageofaunitcubeunder J .Therefore,thecolumnspaceof J + is orthogonaltothepre-imagecylinderand Pa J + > isa d -dimensionalparallelepiped deninganorthogonalcross-sectionofthiscylinder. Frombasicresultsinlinearalgebra,thesingularvaluesof J + > areequalto thoseof J + .Then,frompropertiesofthepseudo-inverse,thesingularvaluesof J + 56

PAGE 74

aretheinverseofthesingularvaluesof J .Finally,fromLemmaV.2,itfollowsthat d Pa J + > = d Y k =1 k )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ; V.6 where f k g d k =1 arethesingularvaluesof J V.3SkewnessandAccuracy In[14],itwasshownthatthenumberofsamplesdeningregulargridsandthus alsouniformi.i.d.setsofsamplesin requiredtoobtainaccurateapproximationsin -measureof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isproportionaltothemaximum localskewness ofthemap Q Below,wedenethelocalskewnessofaQoImap,andprovideameansforcomputing theexpectedskewness. First,assumethat Q islinearand E 2B D ageneralizedrectangle.When d = n Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isa d -dimensionalparallelepipedin ignoringanyaectsfrompossible intersectionswiththeboundaryof thatisin1-to-1correspondencewith E .If d
PAGE 75

J 1 denotesthesubmatrixof J withtherstrowremoved, Pa J 1 isthe d )]TJ/F15 11.9552 Tf 10.994 0 Td [(1 -dimensionalparallelepipeddenedbythe d )]TJ/F15 11.9552 Tf 10.994 0 Td [(1 rowvectors of J 1 ,and d and d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 representthe d -and d )]TJ/F15 11.9552 Tf 12.549 0 Td [(1 -dimensionalLebesguemeasures,respectively. Wenowrelatethisfundamentaldecompositionresulttoinversesets,where,for axed Q 2Q ,welet J k; denotethesubmatrixof J withthe k throwremoved.For simplicity,weassumethat Q islinearand1
PAGE 76

Then,wedenethe localskewness ofthemap Q 2Q atapoint as S Q =max 1 k d S Q J ;Q ; j k : V.8 Conceptually, S Q J ; j k describestheamountof redundant informationpresent inthe k thcomponentoftheQoImapcomparedtowhatispresentintheother d )]TJ/F15 11.9552 Tf 11.346 0 Td [(1 componentswheninvertingnearthepoint 2 .Thesmallestvalue S Q canbe isone.Thereisnolargestvaluesincethereexistsmaps Q thathaveGDcomponent maps,buttheconditionoftheJacobianmaybearbitrarilylarge.IftheJacobianwere toeverfailtobefullrank,then S Q wouldbeinnite.However,theassumptionof GDQoIpreventsthisfromoccurring. ThefundamentaldecompositionofTheoremV.4alongwithLemmaV.2provides aconvenientmethodfordeterminingtheskewnessintermsofthe d -dimensional parallelepipedsdescribedby Q CorollaryV.6 Forany Q 2Q S Q canbecompletelydeterminedbythethe normsof n -dimensionalvectorsandproductsofsingularvaluesoftheJacobianof QoImapsofdimensions d )]TJ/F15 11.9552 Tf 11.955 0 Td [(1 and d S Q =max 1 k d j j k j d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 Pa J k; d Pa J : V.9 Proof: S Q =max 1 k d S Q J ; j k =max 1 k d j j k j j j ? k j =max 1 k d j j k j d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Pa J k; d Pa J ; V.10 thenapplyingLemmaV.2wehave max 1 k d j j k j d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Pa J k; d Pa J =max 1 k d j j k j Q d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 r =1 k;r Q d r =1 r : V.11 59

PAGE 77

where f r g 1 r d arethesingularvaluesof J and f k;r g 1 r d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 arethesingularvalues of J k; CorollaryV.6impliesthatwecanexploitecientsingularvaluedecompositions toalgorithmicallyapproximate S Q atanypoint 2 .Since S Q mayvary substantiallyover ,wemustquantifythisvariabilityinordertooptimallychoose Q 2Q .Althoughwehaveassumedthat Q 2Q isGD,werealizethat,inpractice, thismaynotbetrue.Inordertoaccommodatethisscenario,andtoreducethe impactofoutliers,wedenetheexpectedvalueof S Q inspiredbytheharmonicmean ofanitesetofpositivenumbers,i.e.,if isarandomvariableand f > 0,then theharmonicmeanof f isgivenby H f = 1 Z 1 f d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 : Thismotivatesthefollowing DenitionV.7 Forany Q 2Q ,wedenethe expectedskewnessESK as H S Q = 1 Z 1 S Q d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ; V.12 Generally,weapproximate H S Q usingMonteCarlotechniquestogeneratea setofindependentidenticallydistributedi.i.d.samples i N i =1 andcompute H S Q H N S Q := 1 N N X i =1 1 S Q i )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 : V.13 RemarkV.8 Noticethedescriptionsofskewnessareindependentofthegeneralized rectangle E 2B D .Inotherwords,skewnessisapropertyinherenttothemap Q itself anddescribesthewayinwhichthegeometryof E 2B D ischangedbyapplying Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 V.4ScalingandPrecision 60

PAGE 78

Tomotivatewhatfollows,considerthesimpleproblemwherewemustchoose betweentwodierentEDsleadingtotwodistinctQoImaps Q a and Q b sothat Q = Q a ;Q b .Let E a and E b representthesetofallprobableobservations fromusingeither Q a or Q b ,respectively.Then,dependingonwhichexperiment weobserve,wewouldconcludethateithertheparametersbelongto )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(Q a )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E a or )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(Q b )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E b almostsurely.Suppose )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(Q a )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E a )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(Q b )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E b ; thenwegenerallyexpectthatensemblesofparametersamplesgeneratedfromresults basedontheexperimentleadingto Q a willhavesmallervariancewhichisadescriptionofprecisioninstatisticaltermsthanthosebasedontheexperimentleading to Q b .Thismotivatesageneralmeasure-theoreticgoalfordesigningexperiments whereeventsofhighprobabilityonadataspacearemadesmallinvolume"onthe parameterspacebyinvertingtheQoImap. Webeginwithasimplifyingassumptionthat Q 2Q islinearwithGDcomponent maps.Then,thereexistsa d n matrix J ,suchthat Q = J .If d = n and= R n itiseasilyshownfromstandardresultsinmeasuretheoryandlinearalgebrathat Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E = D E det J )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 = D E d Y k =1 k )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ; V.14 where f k g 1 k d arethesingularvaluesof J .Notethatif R n isproper,then theaboveequationisnotnecessarilytrueas Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E mayintersecttheboundaryof .Weneglectsuchboundaryeectsinthecomputations,andsimplynotethatin certaincasestheymayplayanimportantrolealthoughthisisnotthetypicalcase inourexperience.If d
PAGE 79

skewnessof Q ,inordertoreducetheimpactofoutliers,wedenetheexpectedvalue of M Q bytheharmonicmeanofarandomvariable.Thismotivatesthefollowing DenitionV.9 Forany Q 2Q and 2 ,wedenethe local -measure scalingeect of Q as M Q = d Y k =1 k )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ; V.15 where f k g 1 k d arethesingularvaluesoftheJacobian J ;Q .The expected measurescalingeectESE isgivenby H M Q = 1 Z 1 M Q d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ; V.16 Aswiththeexpectedskewnessof Q ,wegenerallyapproximate H M Q usingasetof i.i.d.randomsamples i N i =1 andcomputing H M Q H N M Q := 1 N N X i =1 1 M Q i )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 : V.17 Wesummarizetheaboveresultsintothefollowing CorollaryV.10 Forany Q 2Q ,thelocalskewness,localscalingeect,average skewness,andaveragescalingeectof Q canbecomputedusingnormsofrow-vectors andsingularvaluesoftheJacobian J ;Q RemarkV.11 Analternativeto H M Q thataccountsforpossiblydierent D E istousesimplemultiplicationof D E assuggestedbyEq. V.14 ThegeometricinterpretationoftheESEoeredabovedescribeshowthesizeof setsarescaledastheyareinvertedwiththemap Q .Thatis,givenaQoImapwith anESEof10,weexpectthecrosssectionoftheinverseimageofanarbitraryset E 2B D to increase insizebyafactorof10.Likewise,givenaQoImapwithan ESEvalueof0.1,weexpectthecrosssectionoftheinverseimageofanarbitrayset 62

PAGE 80

E 2B D to decrease insizebyafactorof10.AnESEvalueof1suggeststhesizeof thecrosssectionoftheinverseimageof E 2B D willremainunchangedfromitssize in D .ThisisillustratedintheschematicinFigureV.1. WeoeranalternativegeometricinterpretationoftheESEthatshowsclearly howtheESEisrelatedtothe reduction ofuncertainty,i.e.,theprecisionofsolving aSIPwithaparticularQoImap.ConsidertwoQoImaps Q a and Q b suchthat theESEof Q a is0.1andtheESEof Q b is0.01.Weexpect Q a )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 maps E 2B D with D E =10toasetthesizeofaunitcubein ,andweexpect Q b )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 maps E 2B D with D E =100toasetthesizeofaunitcubein .Thatis,weexpect themap Q b )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 tomaplargersetsin D intoaregionthesizeoftheunitcubein ThisisillustratedintheschematicinFigureV.2. FigureV.1: TheESEdescribeshowuncertaintiesofaxedsizeonthedataspace arereducedundertheinverseoftheQoImap. V.5AMulti-ObjectiveOptimizationProblem Welet S R denotethesetofallpossiblevaluesof H S Q and M R denote thesetofallpossiblevaluesof H M Q forall Q belongingtoaspecied Q .Clearly, S isboundedbelowby1,whichrepresentsthecaseofoptimalglobalskewness"in aQoImap Q .Similarly, M isboundedbelowby0,whichrepresentsthecasewhere 63

PAGE 81

FigureV.2: TheESEdescribeshowmuchuncertaintyonthedataspaceisreduced tothesizeoftheunitcube. asetofoutputdataexactlyidentiesaparticulargeneralizedcontourofparameters responsibleforthedata. For Q ,wecanclearlyorderall Q 2Q accordingtoeitherthevaluesof H S Q or H M Q .Inotherwords,thevaluesof H S Q and H M Q separatelydescribean orderingindexonthespaceofallpossibleQoIgivenby Q .However,themapping Q 7! H S Q ;H M Q describesadoubleindexingwithnonaturalordering.We denemetricsinordertoquantifythedistancetotheoptimalpointof ; 0inthe Cartesianproductspacedescribedbythepairs H S Q ;H M Q .Whilethereare manyoptionsfordeningmetrics,athoroughinvestigationontheeectofdierent metricsisbeyondthescopeofthisthesiswhichisfocusedontheexpositionofthis generalapproach.Below,wechooseaparticularformforthemetricson S and M thatminimizestheeectofthepossiblydisparaterangesofvalueswemayobserve in S and M onthesolutiontothemulti-objectiveoptimizationproblem. Wedene S ;d S and M ;d M usingthemetrics d S x;y = j x )]TJ/F20 11.9552 Tf 11.955 0 Td [(y j 1+ j x )]TJ/F20 11.9552 Tf 11.955 0 Td [(y j forall x;y 2S ; V.18 64

PAGE 82

and d M x;y = j x )]TJ/F20 11.9552 Tf 11.955 0 Td [(y j 1+ j x )]TJ/F20 11.9552 Tf 11.956 0 Td [(y j forall x;y 2M : V.19 Choose 2 ; 1andlet Y denotetheCartesianproductspace SM ,withmetric denedby d Y x;y = !d S x 1 ;y 1 + )]TJ/F20 11.9552 Tf 11.956 0 Td [(! d M x 2 ;y 2 forall x;y 2 Y : V.20 Notethat determinestherelativeimportanceweplaceoneitherprecisionoraccuracy.Choosing =0implieswedisregardskewnessintheobjectivewhereaschoosing =1implieswedisregardthescalingeect. DenitionV.12 Given Q and 2 ; 1 ,the OEDproblem isdenedbythe multi-objectiveoptimizationproblem min Q 2Q d Y p;y ; V.21 where p = ; 0 isthe idealpoint and y = )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(H S Q ;H M Q for Q 2Q V.6NumericalExamples Inthissection,weagainconsidertwomodelsofphysicalsystemsoriginallyshown inChapterIV.First,weconsiderastationaryconvection-diusionmodelwithasingle uncertainparameterdeningthemagnitudeofthesourceterm.Inthisexample,we considerchoosingasingleQoItodeneanOED.Hence,weoptimizeonlyforthe ESE,asskewnessisnotdenedforone-dimensionaldataspaces.Next,weconsidera time-dependentdiusionmodelwithtwouncertainparametersdeningthethermal conductivitiesofametalrod. InDenitionV.12,wedenetheOEDintermsofa minimization problem.In thissection,weconsider maximizingtheinverse oftheoptimizationobjectivefortwo reasons:tocompareresultstothoseinChapterIVinamoreobviousway,and, 65

PAGE 83

byconsideringtheinverseof d Y p;y andtheESEandESKweaccentuatethe smallervaluesoftheoptimizationobjectivesinsuchawaythatmakestheOEDmore obviousinplotsoverindexingspaces. ExampleV.1StationaryConvection-Diusion:UncertainSourceAmplitude Here,weconsiderastationaryconvection-diusionproblemwithasingleuncertain parameterdeningthemagnitudeofasourceterm.ThisexampleservestodemonstratethattheOEDformulationgivesintuitiveresultsforsimpleproblems.SeeExampleIV.3foradescriptionofthemodelandproblemsetup. Results Wediscretizethecontinuousdesignspace Q fromExampleIV.3using2,000uniformrandompointsin f x k 0 ;x k 1 g 2000 k =1 ,whichcorrespondsto2,000EDsdenoted by f Q k g 2000 k =1 Wegenerate5,000uniformsamplesin andcompute f Q k i g 5000 i =1 2000 k =1 .We approximate f J i ;Q k g 5000 i =1 2000 k =1 usingaradialbasisfunctionRBFinterpolation method.Essentially,thismethodconstructsalocalsurrogateof Q k about i and dierentiatesthesurrogatetoobtainanapproximationto J i ;Q k ,seeAppendixA fordetails.Wecalculate f H N M Q k g 5000 k =1 usingEq. V.17 andplottheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 as afunctionoftheindexingsetdenedby inFigureV.3.Weseeresultssimilarto thoseinExampleIV.3,i.e.,weobservehighervaluesnearthecenterofthedomain nearthelocationofthesourceandinthedirectionoftheconvectionvectoraway fromthesource. ExampleV.2TimeDependentDiusion:UncertainDiusionCoecients Here,weconsideratime-dependentdiusionproblemwithtwouncertainparameters deningthethermalconductivitiesofametalrod.Wechoosethedataspacesunderconsiderationtobetwo-dimensionsaswell.WeconsideroptimizingtheESE, ESK,andsolvingamulti-objectiveoptimizationproblem.SeeExampleIV.4fora descriptionofthemodelandproblemsetup. 66

PAGE 84

FigureV.3: SmoothedplotoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingsetforthedesign space Q .Noticethehighervaluesinthecenterofthedomainandtowardsthetop rightinthedirectionoftheconvectionvectorfromthelocationofthesource,this isconsistentwithbothintuitionandtheresultsfromExampleIV.3. Results Wediscretizethecontinuousdesignspace Q fromExampleIV.3using20,000 uniformrandompointsin f x k 0 ;x k 1 g 20000 k =1 ,whichcorrespondsto20,000EDs denotedby f Q k g 20000 k =1 ,where Q k = )]TJ/F20 11.9552 Tf 5.48 -9.683 Td [(u x k 0 ;t f ;u x k 1 ;t f foreach k .Wegenerate5,000uniformsamples, f i g 5000 i =1 ,in andcompute f Q k i g 5000 i =1 20000 k =1 Weapproximate f J i ;Q k g 5000 i =1 20000 k =1 usinganRBFinterpolationmethod,seeAppendixAfordetails.Wecalculate f H N M Q k g 20000 k =1 and f H N S Q k g 20000 k =1 using Equations V.17 and V.13 ,respectively. InFiguresV.4-V.6,weshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ,ESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,and )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(d Y p;y )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 with =0 : 5 overtheindexingset forthedesignspace Q .Wenotethat,duetothesymmetry within alongtheline x 0 = x 1 ,weonlyshoweachoptimizationobjectiveoverhalf oftheindexingset becausetheotherhalfcorrespondstoarenamingofcontact thermometeroneascontactthermometertwoandviceversa.Eachoftheoptimization objectivesproducenonconvexfunctionsovertheindexingsetandaresubstantially morecomplicatedthantheresultsobservedinExampleV.1.Foreachobjective,we 67

PAGE 85

labelthenotablelocalmaximawithin .FortheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andESK )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ,weobserve fournotablelocalmaxima,andfor d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Y weobserveonenotablelocalmaxima.However, ineachcase,placingthecontactthermometerssymmetricallyawayfromthemidpoint i.e.,neartheendsoftherodR1canleadtooptimaldesigns,whichmatches physicalintuitionseethelowerrightcorneroftheimagesinFiguresV.4-V.6.The tablesintherightofFiguresV.4{V.6showthelocalOEDoptimizationobjective valuesforeachoftheregionslabeledintheircorrespondingplotsovertheindexingset InFigureV.4,theESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthelocalOEDsinR2andR3arerelativelysimilar. Thisisexpected,asthesymmetryofthephysicaldomainandthelocationofthesource suggestthatthesetwoEDlocationswouldyieldasimilarreductioninuncertainties ofthemodelinputparameters.TheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthesetwolocalOEDarenotexactly thesameduetovarioussourcesofapproximationerrors:thediscretizationofthe domain,thesamplingofthe ,andthesamplingof RegionOptimalESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 R16 : 54 E +02 R27 : 00 E +02 R36 : 45 E +02 R46 : 06 E +02 FigureV.4: leftSmoothedplotoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 overtheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof .This locationcorrespondstoplacingthermometersneareachendoftherod.However, weobservefourprominantlocalmaxima,denotedR1,R2,R3andR4.rightThe ESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthelocalOEDineachofthefourregions. 68

PAGE 86

RegionOptimalESK )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 R19 : 95 E )]TJ/F15 11.9552 Tf 11.955 0 Td [(01 R29 : 42 E )]TJ/F15 11.9552 Tf 11.955 0 Td [(01 R39 : 61 E )]TJ/F15 11.9552 Tf 11.955 0 Td [(01 R46 : 13 E )]TJ/F15 11.9552 Tf 11.955 0 Td [(01 FigureV.5: leftSmoothedplotoftheESK )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof .This locationcorrespondstoplacingthermometersneareachendoftherod.However, weobservefourprominantlocalmaxima,denotedR1,R2,R3andR4.rightThe ESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthelocalOEDineachofthefourregions. RegionOptimal d Y p;y )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 R12 : 17 E +02 FigureV.6: leftSmoothedplotof d Y p;y )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,with =0 : 5,overtheindexing set forthedesignspace Q .Noticethehighvaluesinthebottomrightcorner of .Thislocationcorrespondstoplacingthermometersneareachendofthe rod.Here,weobserveonlyoneprominantlocalmaxima.right d Y p;y )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,with =0 : 5,forthelocalOEDinthesingleregionofinterest. 69

PAGE 87

V.7Summary Inthischapter,wedevelopedaSB-OEDapproachbasedentirelyonlocallinear approximationsofQoImaps.Thisapproachoptimizestwogeometricpropertiesof inverseimageswhicharerelatedtotheaccuracyandprecisionofthesolutiontoa givenSIP.Thenoveltyofthisapproachlieswithintwokeyfeatures:theapproach denesanOEDindependentoftheexpectedandoftenunknownuncertaintieson theobserveddata,and,providesaquanticationoftheexpectedcomputational limitationsinaccuratelyapproximatingsolutionstoagivenSIP.Inthefollowingchapter,wemakedirectcomparisonsbetweenthisapproachandtheconsistentBayesian approachtoSB-OEDdevelopedinChapterIV. Themulti-objectiveoptimizationproblemdenedinEq.V.21issomewhatunsatisfying.Themetricsdenedon S and M arechosentoreducethechancesthat oneoptimizationobjectivedominatesthedecisionbecausethevaluesofthatobjective arelarger.Then ischosentodeterminea weighting ofthepriorityofoptimizing eachobjective.Determininganoptimal tosatisfytheneedsofaspecicproblem isanopenquestion.InChapterVII,weaddressthisquestion,inasomewhatunconventionalway,todenea singleobjective OEDproblemthatisimplicitlyafunction oftheESE,thenumberofmodelsolvesavailabletoapproximatethesolutiontoa givenSIP,andthedimensionoftheparameterspace. 70

PAGE 88

CHAPTERVI EXPECTEDSCALINGEFFECTVSEXPECTEDINFORMATION GAIN Inthischapter,weprovideanin-depthexplorationandcomparisonofthetwo previouslydevelopedSB-OEDapproaches:theconsistentBayesianapproach,describedinChapterIV,andthemeasure-theoreticapproach,describedinChapterV. TheconsistentBayesianapproachisbasedontheclassicalBayesianmethodologyfor SB-OED.ThisapproachquantiestheutilityofagivenEDusingtheEIG,whichis denedintermsofsolutionstoasetofprobableSIPsofinterest.Thus,thisapproach requiresknowledgeoftheproleoftheobserveddensityoneachQoIinordertosolve theSIPandthenapproximatetheOED. Themeasure-theoreticapproachquantiestheutilityofagivenEDintermsof theESE 1 ,whichdescribeshowmeasurablesetsinthedataspaceare scaled asthey areinvertedintotheparameterspacebyagivenQoImap.Thisapproachisbased entirelyonlocallinearapproximationsoftheQoImapunderconsideration.Thus, computationoftheESErequiresgradientapproximations.However,thisapproach does not requirepriorknowledgeoftheproleoftheobserveddensityoneachQoI. Thegeneralityofthisapproach,basedonpropertiesofagivenQoImap not on propertiesofsolutionstoasetofSIPs,provesvaluableinrobustlyapproximating OEDswithlimitednumbersofmodelsolvesavailableandunknownprolesofthe observeddensityoneachQoI. InSectionVI.1,weintroducetwotechnicallemmasusedintheanalysisofthe resultspresentedinthischapter.InSectionVI.2,wediscusscomputationalconsiderationsforeachapproach.Wefocusonthecomputationalcostasafunctionof thenumberofmodelsolves.InSectionVI.3,weprovideanexampletoillustrate theabilityofeachmethodtodistinguishbetweenhighlysensitiveQoIgivenalim1 Themeasure-theoreticapproachalsoconsiderstheESK,however,forsimplicity,wefocusonly ontheESEinthischapter. 71

PAGE 89

itednumberofmodelsolves.InthecontextofOED,weoftenmustprescribean expectedutilitysystematicallytoabroadsetofEDs.ThissetofEDsmayinclude EDswheretheproleoftheobserveddensityislargerelativetotherangeofthedata space.Moreover,itmayincludeEDscomposedofQoIprovidingredundantdata, i.e.,thecomponentsmayfailtobelinearlyindependentinregionsof,orthroughout,theparameterspace.InSectionVI.4,weconsiderthestabilityofeachOED methodwithrespecttovaryingprolesoftheobserveddensity,withaspecicfocus onobserveddensitieswhosesupportextendswellbeyondtherangeofthedataspace. InSectionVI.5,weexploretheabilityofeachOEDmethodtodistinguishoptimal propertiesamongstEDsdenedbyQoImapswithJacobiansthatarenotlocally,or globally,fullrankin .InSectionVI.6,weprovidesummarizingremarksforthe chapter. VI.1TechnicalLemmas Inthissection,weintroducetwotechnicallemmasusedlaterinthischapter.The rstlemmastatesthat,underacertainassumption,theMCapproximationtothe KL-divergenceisboundedfromabovebylog N ,where N isthenumberofsamples usedintheMCapproximation.ThesecondlemmastatesthattheKL-divergenceof from prior isequaltotheKL-divergenceof obs D from Q prior D LemmaVI.1 Suppose Q : !D is B -measurable, prior isuniformover ,and i N i =1 isthesetofi.i.d.sampleswithrespectto prior in usedtoapproximate both and KL Q prior : .ThentheMCapproximationto KL Q prior : denoted KL Q prior : V ,isboundedby log N KL Q prior : V log N : Proof: Theuniformpriorisgivenby prior = 1 8 2 : VI.1 72

PAGE 90

Weobservethat KL Q prior : V ismaximumwhenthereisonlyone i 2 supp Inthiscase,usingthestandardMCapproximationthat V i =N implies thatthecomputed isgivenby = 8 > > < > > : N ; 2V i ; 0 ;= 2V i ; VI.2 where V i istheVoronoicellassociatedwith i 2 supp .Then, KL Q prior : V = N N X i =1 i log i prior i VI.3 N N log N= 1 = VI.4 =log N whereEq.VI.3comesfromtheMCestimateoftheintegralandEq.VI.4results fromthesubstitutionof prior and aboveintoEq.VI.3. LemmaVI.2 Assume Q : !D is B -measurable.Then,theKL-divergenceof from prior isequaltotheKL-divergenceof obs D from Q prior D KL Q prior : = KL Q Q prior D : obs D : 73

PAGE 91

Proof: UsingthedenitionoftheKL-divergencegivenbyEq.IV.1,wehave KL Q prior : = Z log prior d = Z prior obs D Q Q prior D Q log obs D Q Q prior D Q d = Z obs D Q Q prior D Q log obs D Q Q prior D Q dP prior = Z D obs D q Q prior D q log obs D q Q prior D q dP Q prior D q = Z D Q prior D q obs D q Q prior D q log obs D q Q prior D q d D q = Z D obs D q log obs D q Q prior D q d D q = KL Q Q prior D : obs D ; whereweusethedenitionof giveninEq.III.12andthefactthat Z f Q dP prior = Z D f q dP Q prior D q : VI.2ComputationalConsiderations Inthissection,wediscussthecomputationalrequirementofeachoftheOED approachesinthecontextofseveralrelevantcomputationalmodelingscenarios.The variousscenariosdonotlargelyimpactourabilitytocomputeapproximationsto OEDsusingtheconsistentBayesianapproach.TheconsistentBayesianapproach requiresapproximatesolutionstoSIPs,andthissimplynecessitatesaforwardprogagationofuncertaintythroughthecomputationalmodelinordertocomputethe pushforwardofthepriordensity.Asdiscussedpreviously,weemploystandardKDE techniquestoconstructanon-parametricapproximationofthepushforwardofthe 74

PAGE 92

PAGE 93

todetermineanOED 3 .Inthisscenario,wechoosetoutilizeradialbasisfunctions RBFstoapproximatelocalresponsesurfaces[70,26].Essentially,thesemethods canoperateonanunstructuredsetofsamplestoconstructalocalsurrogateabouta given 2 anddierentiatethesurrogatetoobtainanapproximationtothegradient,seeAppendixAfordetails.Thisis,inmanyways,themostchallengingscenario forimplementationofthemeasure-theoreticapproachtoOED.Hence,intheensuing examples,weassumewedonothaveanadjointandthatwehave N uniformrandom samplesin ,withcorrespondingdata,forwhichwecanusetodetermineanOED. VI.3HighlySensitiveQoI Inthissection,weexploretheabilityofeachOEDmethodtodistinguishbetween EDscomposedofQoIthatarehighlysensitivetochangesinparameters.Webegin withanexampletoillustrateourndings,andfollowwithanexplanation. RemarkVI.3 Wenotethat,inExampleVI.1below,theproleoftheobserveddensityoneachQoIdierssubstantiallyfromthatchosenforthesamemodelinExampleIV.4.ThischoiceismadetohighlightaspeciclimitationoftheBayesian approachtoOED,specically,thatthemethodhasdicultydistinguishingbetween twohighlysensitiveEDswhenlimitedmodelsolvesareavailable. ExampleVI.1Time-DependentDiusion:LimitedModelSolves Weconsideratime-dependentdiusionproblemwithtwouncertainparametersdeningthe thermalconductivitiesofametalrod.SeeExampleIV.4foracompletedescription ofthemodelandproblemsetup. Recall,theexperimentsinvolveplacingtwocontactthermometersalongthefull rodthatcanrecordtwoseparatetemperaturemeasurementsat t f .Thus,thereisa bijectionbetween and Q sopoints x 0 ;x 1 2 canbeusedtoindexthe EDs.FortheconsistentBayesianapproach,welettheuncertaintyineachQoIbe 3 Or,evenworse,tousethese N samplestobothdetermineanOEDandthensolvetheSIPwith thesamesetof N samples. 76

PAGE 94

describedbyatruncatedGaussianprolewithaxedstandarddeviationof 1 E )]TJ/F15 11.9552 Tf 11.88 0 Td [(04 Thischoiceofuncertaintiessuggeststhemeasurementinstrumenthasahighdegree ofaccuracyrelativetotherangeofthedatathatweexpecttoobservebasedonthe priorinformation,i.e., D supp obs D << D D Here,wefocusontheabilityofeachOEDapproachtocorrectlyclassifytwo specicgroupsofEDs:oneoptimalgroupandonesub-optimalgroup.Inparticular, weareinterestedintheabilitiesofeachmethodasafunctionofthenumberofmodel solvesavailable.WechoosetheoptimalgroupofEDstobe10pointschosenrandomly fromasquarewithsidelength 0 : 04 centeredattheindexingpoint : 97 ; 0 : 03 2 andthesub-optimalgrouptobe10pointschosenrandomlyfromasquarewithside length 0 : 04 centeredattheindexingpoint : 9 ; 0 : 7 2 ,seeFigureVI.1. FigureVI.1: ThetwogroupsofEDsshownintheindexingset .Thegroupin thebottomrightoveraredregionistheoptimalgroupofEDsandthegroupnear theupperrightoverablueregionisthesub-optimalgroup.Forreference,weshow thesetwoclustersplottedontopoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 computedusingall5,000available samplesin asdescribedabove. Results 77

PAGE 95

ForbothOEDmethods,weconsiderapproximatingtheESEandEIGvaluesfor increasingnumbersofmodelsolvesupto N =5000 ,whereeachsetofsamplesis drawnfromtheuniformprior.Weassumethatwedonothaveanadjointproblem availabletoapproximate f J i ;Q j g N i =1 20 j =1 ,therefore,weemployaRBFmethodto obtaintherequiredgradientinformation,seeAppendixAfordetails. Foragivensetof N samples,weapproximatetheESEandEIGvaluesforeach ofthe20EDs.Recall,ourgoalistocorrectlyclassifyeachEDasbelongingtoeither theoptimalorsub-optimalgroup.InFigureVI.2,weshowtheapproximateESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 andEIGvalues,foreachofthe20EDs,asafunctionofthenumberofmodelsolves available.NoticethecleardierenceinESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 betweentheoptimalandsub-optimal clustersofEDsforall N .However,fromthisgure,theEIGvaluesofeachofthe 20EDsappeartobethesameforall N InTableVI.1,wechooseoneoptimalandonesub-optimalEDandshowtheEIG valuesforincreasing N .WeseethattheEIGvaluesforthechosenoptimalandsuboptimalEDsarethesame,outtotwelvesignicantdigits,for N 1000 .Thisposes aconcern,astheESEinformsusthattheoptimalclusterofEDsisapproximatelyten timesmoresensitivetoinformativeoftheparametersthanthesub-optimalcluster, i.e.,weexpectanEDfromtheoptimalclustertoscaleasetinthedataspaceof D measureapproximately650unitstoasetin of -measureoneunitandanEDfrom thesub-optimalclustertoscaleasetinthedataspaceof D -measureapproximately 70unitstoasetin of -measureoneunit,seeFigureV.2inChapterVforan illustrationofthisgeometricinterpretationoftheESE.Thus,usinganEDfromthe optimalclustertosolveaSIPreducesuncertainsetsinthedataspacebyanorder ofmagnitudemorein -measurecomparedtousingoneofthesuboptimalEDs. However,theEIGdoesnotdierentiatebetweenthetwogroupsuntilweusemore than1000modelsolves.Evenfor 1000 N 5000 ,theEIGvaluesamongstthetwo clustersarerelativelysimilar.Consider N =5000 ,therelativedierencebetweenthe 78

PAGE 96

twoEIGvaluesisapproximately 0 : 0041% FigureVI.2: leftTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthe20EDsunderconsideration.Noticetheclear dierenceinESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 valuesbetweentheoptimalgroupredandthesub-optimalgroup blue.Theoptimalgrouplookstobejustasingleline,however,thereareactually10 linesplottedhereandtheysimplyliesoclosetogetherthattheyareindistinguishable. rightTheEIGforthe20EDsunderconsideration.Inthisimage,wedonotseea cleardistinctionbetweenthetwogroupsofEDs.Again,thereare10redlinesand10 blueslineshere,theyarejustnotdistinguishable.SeeTableVI.1foraquantitative descriptionoftheseEIGvalues. LemmaVI.1explainswhytheEIGvaluesfortheoptimalandsub-optimalEDs inExampleVI.1arethesamefor N 1000.Consider N =1000,foranyofthe20 EDs,each producedinAlgorithm4containsonlyasinglesampleandtherefore thecorresponding KL Q computedisequaltologandthenEIG=log 6 : 908.Hence,asweincrease N ,weincreasethemaximumvaluetheEIGcantake anditsabilitytodierentiatebetweenhighlysensitiveEDs.Thissuggeststhatina scenarioinwhichmorethanoneEDhasanEIGofapproximatelylog N ,weshould bewaryoftheEIG'sabilitytodierentiateeectivelybetweenEDs.FigureVI.2 showsthattheESEdoesnotsuerfromthissamelimitation. VI.4UnknownProleoftheObservedDensity SomeapplicationslooktodetermineanOEDtosubsequentlyreduceuncertainties inmodelinputparametersfromuncertainobservationaldata without priorknowledge oftheprolesoftheobserveddensityoneachQoI.Forexample,consideracontami79

PAGE 97

N EIGforoptimalEDEIGforsub-optimalED 102.302585092992.30258509299 503.912023005433.91202300543 1004.605170185994.60517018599 3005.703782474665.70378247466 5006.214608098426.21460809842 10006.907755278986.90775527898 20007.600902459547.60073425065 30008.006367567658.00620455542 40008.29404964018.29361794088 50008.517193191428.51684558669 TableVI.1: TheEIGforasingleEDfromtheoptimalclusterandasingleEDfrom thesub-optimalcluster.TheEIGvaluesareidenticaltotwelvesignicantguresfor N 1000.However,fromtheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 valuesinFigureVI.2,weknowthattheoptimal EDisapproximately10timesmoresensitivetoinformativeoftheinputparameters thanthesub-optimalED. nanttransportmodelinwhichweseektoreduceuncertaintiesinthediusivitypropertiesofthedomainbasedonconcentrationmeasurementatsomespeciedpointsin space-time.SupposewemaymeasureeachQoImultipletimesandusethesemultiplemeasurementstodenetheproleoftheobserveddensityoneachQoI.Priorto gatheringthesemultiplemeasurements,itisunlikelythattheproleoftheobserved densityisknown.ThispresentsapotentialissuefortheconsistentBayesianapproach asthismethodrequiresthepriorknowledgeoftheprolesoftheobserveddensityon eachQoIinordertodetermineanOED. Alternatively,withinthecontextofOED,weoftenmustprescribeanexpected utilitysystematicallytoabroadsetofEDs.ThissetofEDsmayincludeEDswhere thestandarddeviationoftheproleoftheobserveddensityislargerelativetothe rangeofthedataspace.Inthefollowingexample,weconsiderbothofthesescenarios: thedependenceoftheEIGontheprolesoftheobserveddensityoneachQoI,and theimpactofhighlyuncertainobserveddensitiesrelativetothe D -measureofthe dataspace.Aftertheexample,weprovideanexplanationofourndings. ExampleVI.2Time-DependentDiusion:UnknownObservedDensityProle Weconsideratime-dependentdiusionproblemwithtwouncertainparametersden80

PAGE 98

ingthethermalconductivitiesofametalrod.SeeExampleIV.4foracompletedescriptionofthemodelandproblemsetup. Recall,theexperimentsinvolveplacingtwocontactthermometersalongthefull rodthatcanrecordtwoseparatetemperaturemeasurementsat t f .Thus,thereisa bijectionbetween and Q sopoints x 0 ;x 1 2 canbeusedtoindextheEDs. Forthisexample,weconsiderthescenarioinwhichwedonotknowtheuncertainties ineachQoI,i.e., inEq. IV.3 isunknown.Moreover,weassumewearegiven limitedresourcestodetermineanOED,specically,wehavejust N =100 model solvesatasetofi.i.drandomsamplesin Asinthelastexample,wespecifytwogroupsofEDs:oneoptimalgroupandone sub-optimalgroup.Wedeneoptimalandsub-optimalintermsoftheESEvalues approximatedusingthesame N =100 i.i.d.randomsamplesthatareavailablefor theconsistentBayesianapproach.Inparticular,weareinterestedintherobustnessof theconsistentBayesianapproachtovaryingprolesoftheobserveddensityoneach QoI.WechoosetheoptimalgroupofEDstobe50pointschosenrandomlyfroma squarewithsidelength 0 : 04 centeredattheindexingpoint : 97 ; 0 : 03 2 ,and thesub-optimalgrouptobe50pointschosenrandomlyfromasquarewithsidelength 0 : 04 centeredattheindexingpoint : 73 ; 0 : 26 2 ,seeFigureVI.3. Results ForbothOEDmethods,weconsiderapproximatingtheESEandEIGvaluesfor N =100 samples.Weassumethatwedonothaveanadjointproblemavailableto approximate f J i ;Q j g N i =1 100 j =1 ,therefore,weemployaRBFmethodtoobtainthe requiredgradientinformation,seeAppendixA. WeapproximatetheESEandEIGvaluesforeachofthe100EDs.Recall,our goalistocorrectlyclassifyeachEDasbelongingtoeithertheoptimalorsub-optimal group.InFigureVI.4,weshowtheapproximateESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 andEIGasafunctionof thestandarddeviation oftheGaussianuncertaintydenedoneachQoI.Noticethe 81

PAGE 99

FigureVI.3: ThetwogroupsofEDsshownintheindexingset .Thegroup inthebottomrightoveraredregionistheoptimalgroupofEDsandtheother groupoverablueregionisthesub-optimalgroup.Forreference,weshowthesetwo clustersplottedontopoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 computedusing N =100availablesamplesin asdescribedabove. 82

PAGE 100

cleardierenceinESEbetweentheoptimalandsub-optimalclustersofEDsforall .ThisisbecausetheESEis not afunctionoftheproleoftheobserveddensityon eachQoI. However,theEIGvaluesareclearlydependenton .Forsmall ,theEIGvalues forallofthe100EDsareindistinguishableinFigureVI.4.Then,thismethodcorrectly classieseachEDaseithersub-optimaloroptimalfor 1 E 03 1 : 5 .Finally,for > 1 : 5 ,theEIGvaluesfortheoptimalgroupofEDslieinthemiddleoftheEIG valuesforthesub-optimalgroupofEDs. InTableVI.2,wechooseoneoptimalandonesub-optimalEDandshowtheEIG valuesforincreasing .Weseethatthesub-optimalEDhasthesameEIGvalueas theoptimalEDfor 1 E 05 1 E 04 ,asmallerEIGvaluefor 1 E 04 < 1 : 5 andalargerEIGvaluefor > 1 : 5 .Moreover,noticethattheEIGvalueforthe sub-optimalEDactuallybeginstoincreaseas increasesbeyond 1 E + 00 .Given verynoisyobservationaldataand,hence,alargeuncertaintyontheQoIdening anED,theEIGpotentiallymisclassiesoptimalandsub-optimalEDsandproduces non-intuitiveresultsregardingtheincreaseoftheEIGastheuncertaintiesonthedata increase. Toprovideamoreglobalillustrationoftheimpactsthat hasontheEIGvalues fortheentiredesignspace,inFigureVI.5,weshowtheEIGovertheindexingset forvariousvaluesof andscatterplotthetop 100 OEDs.Itisclearfrom theseimagesthattheOEDasdenedbytheEIGisafunctionof .Thetop 100 OEDlocationsbeginas,apparently,uniformrandompointsin ,thenbeginto clustertowardsthebottomcenterof ,thenclusterinthebottomrightof andnallysettleintwoseparateregionsnearthecenterof as becomeslarge. AsdepictedinExampleVI.2,theEIGproducesinconsistentOEDresultsasthe proleoftheobserveddensityoneachQoIvaries.Moreover,insomecases,the EIGvalueincreasesasthestandarddeviationoftheproleoftheobserveddensity 83

PAGE 101

FigureVI.4: leftWeshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 asafunctionof toemphsizethatthe ESEisdenedindependentoftheexpectedproleoftheobserveddensityoneach QoI.rightTheEIGforthe100EDsunderconsiderationasafunctionof .The optimalgroupisshowninredandthesub-optimalgroupisshowninblue.Notethat, as increase,theEIGfortheoptimalclusterbecomessmallerthantheEIGforthe sub-optimalgroup. EIGforoptimalEDEIGforsub-optimalED 1E-054.605170185994.60517018599 1E-044.605170185994.60517018599 5E-04 4.60517018599 4.60514348489 1E-03 4.60517018599 4.59878890151 1E-02 4.58943082345 4.39595820507 1E-01 4.15175649436 2.15023400619 5E-01 1.96082600602 0.463244860459 1E+00 0.942029897225 0.359941128198 1.5E+00 0.46543957244 0.377398140826 2.0E+000.241747553646 0.391255373739 3.0E+000.114709270539 0.404324508399 5.0E+000.0925786430422 0.412147506656 1.0E+010.0976835734117 0.415690505556 1.0E+020.101585686731 0.416892047918 TableVI.2: TheEIGforasingleEDfromtheoptimalgroupandasingleEDfrom thesub-optimalgroup.InboldfontwehighlightlargerEIGvalueineachcolumn. FromtheESEvaluesinFigureVI.4,weknowthattheoptimalEDisapproximately 10timesmoresensitivetoinformativeoftheinputparametersthanthesub-optimal ED.However,theEIGvalueforthesub-optimalEDisthesameforsmall ,smaller for inthemiddleoftherange,andlargerfor > 2 : 0 E +00. 84

PAGE 102

FigureVI.5: TheEIGovertheindexingset andthetop100OEDsfor increasing .Forverysmall =1 E )]TJ/F15 11.9552 Tf 12.13 0 Td [(05,weseethatthetop100OEDsappearto berandompointsin .For0 : 1 1 : 0,thetop100OEDsareallclusteredin thebottomrightof .Then,assigmaincreasebeyond1 : 0,thetop100OEDs moveawayfromthebottomrightof andsettleintworegionsnearthecenter ofthespace. 85

PAGE 103

increases.Inotherwords,increasingtheuncertaintyinthedataspacecanleadusto concludethesameQoIhaveimprovedinformationgains,whichiscounter-intuitive. Weillustratethecomplexitiesofthisissuewiththenonlinearsystemintroducedin SectionIII.5: 1 x 2 1 + x 2 2 =1 x 2 1 )]TJ/F20 11.9552 Tf 11.955 0 Td [( 2 x 2 2 =1 Recall,theparameterrangesaregivenby 1 2 [0 : 79 ; 0 : 99]and 2 2 [1 )]TJ/F15 11.9552 Tf 11.371 0 Td [(4 : 5 p 0 : 1 ; 1+ 4 : 5 p 0 : 1]whicharechosenasin[10]toinduceaninterestingvariationintheQoI.We considertwoEDsdenedbythethefollowingQoI: Q = x 1 ; Q = x 2 : Forclarity,ratherthanconsidertheEIGforeachED,weconsiderthe KL Q value forasingle obs D 2O Q chosensothatthecenteroftheparameterspacemapstothe centerofthe obs D foreachED.Weconsiderthe KL Q valuesforeachEDforvarying InFiguresVI.6{VI.13,weshowtheobserveddensitiesonthedataspacedenedby eachED,thecorrespondingpushforwardoftheprior,andtheassociatedposteriorsfor variousvaluesof .For =1E-07,thesupportof obs D containsjustasinglesamplefor eachED.Therefore,thecorrespondingposteriorsalsohavesupportsthatcontainjust asinglesample.Thisyieldsaninformationgainvalue KL Q log0000 10 : 5966. As increaseto0.03,weseereasonableresults,i.e.,astheuncertaintiesonthe observeddensityincreasetheinformationgainvaluesdecreaseforeachED.However, as goesfrom0.03to0.05,weseethe KL Q valuefor Q increase .Thisisnot intuitive,weexpectthatastheuncertaintiesontheobserveddataincreasethatour 86

PAGE 104

informationgainshoulddecrease.Weseethesamepropertyfor Q as increases from0.1to1.0. Toshedlightonthissituation,werefertoLemmaVI.2,whichstatesthat KL Q prior : = KL Q Q prior D : obs D : Astheuncertaintyproleontheobserveddensityincreases,theobserveddensity approachesauniformdensityover D .Thus,theKLdivergenceof obs D from Q prior D quantieshowdierent Q prior D isfromauniformdensity.ThisisobservedinFigureVI.13,whereboththeobserveddensitieson Q and Q arenearlyuniformand Q prior D isclosertouniformover D Q than Q prior D over D Q FigureVI.6: topleftApproximationof using obs D on Q with =1E-07, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E-07, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q VI.5RedundantData 87

PAGE 105

FigureVI.7: topleftApproximationof using obs D on Q with =1E-03, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E-03, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 88

PAGE 106

FigureVI.8: topleftApproximationof using obs D on Q with =1E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 89

PAGE 107

FigureVI.9: topleftApproximationof using obs D on Q with =2E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =2E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 90

PAGE 108

FigureVI.10: topleftApproximationof using obs D on Q with =3E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =3E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 91

PAGE 109

FigureVI.11: topleftApproximationof using obs D on Q with =5E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =5E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 92

PAGE 110

FigureVI.12: topleftApproximationof using obs D on Q with =1E-01, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E-01, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 93

PAGE 111

FigureVI.13: topleftApproximationof using obs D on Q with =1E+00, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E+00, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 94

PAGE 112

Inthissection,weconsidertheabilityofeachOEDapproachtodierentiate betweenEDsthatarecomposedofQoIcontainingredundantQoIcomponents,i.e., theQoIcomponentsareeithernotGDorverynearlynotGD.ConsiderthetimedependentdiusionproblemdiscussedindetailinChaptersIVandV.InFigureVI.14,weshowtheplotsoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andtheEIGovertheindexingset ofthedesignspacefor N =5000and =0 : 1. FigureVI.14: SmoothedplotsoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andtheEIGovertheindexingset forthedesignspace Q IntheimagesofFigureVI.14,theline x 0 = x 1 representstheEDsdenedby placingbothsensorsattheexactsamelocationin.Althoughtheremaybeother combinationsofsensorlocationsthatyieldlocallyorgloballyredundantQoI,wefocus onthisregionofthedesignspaceasitisthemostobviousregionofEDsproducing redundantQoI.FortheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,itappearsthatalloftheEDsneartheline x 0 = x 1 havesimilarESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 valuesandarealldeterminedtobeequallysuboptimal.However, theEIGdierentiatesbetweenregionsontheline x 0 = x 1 ,itsuggestsplacingboth sensorsnearthecenterorbothsensorsnearoneoftheendsoftherodismoreoptimal thatplacingthembothaboutone-thirdofthewayfromeitherendoftherod. TheESEcannotbeusedtodierentiatebetweenEDscomposedofredundant QoIbecausewhenthelocallinearapproximationsarelinearlydependent,theESE isinnitesoESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 iszero.TheEIGontheotherhand,distinguishestheutilityof 95

PAGE 113

thesensorlocationsalongtheline x 0 = x 1 becausetheEIGisbasedonthesolutions toSIPs,and,besidestheadditionalcomputationalcomplexityinvolved,solvingthe SIPwithredundantQoIproducesasimilarsolutionassolvingitwithjusttheGD componentsoftheQoImap. InFigureVI.15,weshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andtheEIGgivenjustasinglesensorto placewithintogatherdata.Weseeinthiscase,bothoptimizationobjectivesshow cleardierencesinutilityamongthedierentlocationsoftherod.TheEIGproduces asimilardescriptionoftheone-dimensionaldesignspacetothatoftheline x 0 = x 1 inthetwo-dimensionaldesignspace.Thisisaniceproperty,asitmaybedicultto eliminateanyEDswithredundantQoIpriortodetermininganOED. FigureVI.15: SmoothedplotsoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 andtheEIGovertheindexingset forthedesignspace Q TheoptimalapplicationoftheESEtoproblemsinwhichweexpectEDsto becomposedofQoIwithredundantQoIcomponentsisatopicofcurrentresearch. TheESKisapossibleremedytothisproblem,asitdescribestheamountofnew" informationprovidedbyeachQoIcomposinganED.Givenanentiredesignspace thatproducesEDswithredundantQoI,alloftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 valueswouldbeunusually small,ifnotzero,thus,suggestingthatweshouldconsiderEDsdenedbylowerdimensionaldataspaces.Thiscanbeconsideredbothastrengthandaweakness foreachapproach.AlthoughtheESEcannotdierentiatebetweenredundantQoI,it 96

PAGE 114

doestelltheresearcherthattheyhaveconsideredasetofEDsthatcontainredundant QoI,andtheresearchershouldconsiderusinglessQoIcomponentstodenetheEDs. However,inourexperience,theEIGdierentiatesbetweenredundantdataquitewell, butitdoesnotinformifredundantQoIcomponentsdenetheEDs.Thisisatopic ofcurrentresearchandfurtherpursuitofaremedyisbeyondthescopeofthisthesis. VI.6Summary Thisthischapter,weexaminedtheabilitiesofthetwoSB-OEDapproachesproposedinthreecriticalscenarios:limitedmodelsolvesavailabletodeterminean OEDamongasetofEDscomposedofhighlysensitiveQoI,unknownoruncertainknowledgeoftheprolesoftheobserveddensityoneachQoIcomposingtheEDs underconsiderationandprolesoftheobserveddensitywithsupportsthatextend wellbeyondtherangeoftheQoImap,andEDscomposedofQoIcontaining redundantcomponents. AlthoughweillustratedtheinadequacyoftheEIGindierentiatingbetweentwo highlysensitiveEDswhenrestrictedtofewmodelsolves,thereareapproachesthat mayremedythisissue.Forexample,surrogatemodelscanbeusedtogreatlyreduce thesamplingerrorinUQproblemsandarealikelysolutiontotheinadequaciesof theEIGgivenfewmodelsolves. Whentheprolesoftheobserveddensitiesaredenedappropriately,theEIG producesasimilardescriptionofthedesignspacetotheESE.Insomesense,wemust tunetheproleoftheobserveddensityforeachEDsothattheresultingEIGvalues describeanamountof relative informationgainedcomparedtotheotherEDsunder consideration.Wemaythinkofthisastuning todisplaytherelativesensitivitiesof eachEDasopposedtotheutilityofeachEDgivenaspeciedprolefortheobserved densities.Theissueofthesupportoftheobserveddensityextendingwellbeyondthe rangeoftheQoImapunderconsiderationmightbeaddressedpriortoapproximating theEIG,therebyeliminatingEDsthatshowlittlesensitivitytochangesinthemodel 97

PAGE 115

inputparametersbeforeaSB-OEDapproachisemployed. TheissueofredundantQoImaybethemostpromisingpathforfuturework stemmingfromthischapter.Oftentimes,applicationsstrivetogathermultitudesof dataforsubsequentuseininferringmodelinputparameters.Insomecases,this abundanceofdataislledwithredundantinformation,i.e.,multipleQoIthatare sensitivetothesamedirectionswithintheparameterspace.AlthoughtheEIGappearstodierentiatebetweentheutilitiesofEDscontainingredundantinformation, usingahigherdimensionaldataspacethannecessaryincreasescomputationalcosts andincreasestheriskthatwegathersetsofinfeasibledata. Inthischapter,wehaveprovidedstrongargumentsforsupportingtherobust natureofthemeasure-theoreticapproachtoSB-OED.Thisapproachquantiesnot onlytheESEdiscussedhere,butalsotheESKdiscussedinChapterV.TheESK describesthecomputationalcomplexityofagivenSIPandbyminimizingtheESK weminimizetheexpectederrorsinourapproximationtoagivenSIP.However,we donot eliminate theerrorinourapproximation.AnexactsolutiontoaSIPinvolving solutionstoPDEsis,ingeneral,unattainable,however,wemayimprovethesolution toaspecicgroupofSIPs.Inthefollowingchapter,weproposeareformulationof theSIPforapplicationswithepistemicuncertaintiesinmodelinputparameters. 98

PAGE 116

CHAPTERVII THECOMPUTATIONALCOMPLEXITYOFTHESIP InChapterV,weintroducedtheESKofaQoImapanddiscusseditsrelationship tothecomputationalcomplexityinaccuratelyapproximatingthesolutiontoaSIP [14].AlthoughaQoImapwithsmallESKhasoptimalpropertiesforsetapproximation,ingeneral,thereisstillerrorinanyapproximationtothesolutiontoaSIP.In thischapter,wecontinuetheexplorationofthecomputationalcomplexityofagiven SIP.InSectionVII.1,weconsidertheimpactthiserrorhasinthescenarioofparameteridenticationunderuncertainty.Intheseproblems,thereexistsatrueparameter responsibleforobserveddata,however,duetomeasurementerror,modelinadequacy, limiteddata,etc.,wearelimitedtodescribingsetsof probable parametersresponsible fortheobserveddata.TheaccuracyofthesolutiontoagivenSIPimpactstheability tocorrectlyidentifythistrueparameterwithinaneventofnonzeroprobabilityin WeconsideranalternativeapproachforformulatingandsolvingtheSIPsuchthat, undercertainassumptions,weguaranteethatthetrueparameteriscontainedwithin thesupportofanapproximatedsolutiontotheSIP, V ,i.e., P )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [( true 2 supp V =1 : FornonlinearQoImaps,thisapproachislimitedinitsabilitytoproduceatight coveringoftheinverseimage,however,thedevelopmentofthisapproachprovides insightintotwocriticalareasofthemeasure-theoreticmethodforSB-OED.InSectionVII.2,weprovideacleardenitionofa single-objective optimizationproblem thatconsidersboththeprecisionandaccuracypropertiesofaSIP.Thisobjectiveis, implicitly,afunctionofthesensitivitiesoftheQoImap,thenumberofmodelsolves available,andthedimensionoftheparameterspace.InSectionVII.3,wedescribean alternativecharacterizationofthecomputationalcomplexityofagivenSIPinterms ofaboundontheerroroftheapproximationtoaninverseimage. 99

PAGE 117

VII.1AReformulationoftheSIPforEpistemicUncertainty AsdescribedinChapterIII,oneapproachforapproximatingthesolutiontoaSIP istoutilizeAlgorithm3.Essentially,thisalgorithmdiscretizes withofteni.i.d. randomsamples i N i =1 thatimplicitlydenetheVoronoitessellation fV i g N i =1 Then,probabilitiesonthediscretizeddataspace,determinedbytheobservedprobabilitymeasure,areproportionedouttotheappropriate V i .Weconsiderthesimple caseofalinearQoImapandauniformobserveddensitydenedonsomegeneralized rectangle E 2B D .Inthiscase,theapproximationtothesolutiontotheSIP,asdeterminedbyAlgorithm3,issimplyaprobabilitymeasurethatisconstantoversome unionofVoronoicellsapproximatingtheinverseimageof E ,andzeroeverywhere else.Inotherwords,theSIPreducestoaccuratelyapproximatingthesetdening thesupportoftheinverseimage. Thenitesamplingof maycausethisapproximationtobeapoorapproximationoftheactualinverseimage,seeFigureVII.1left.Suchanapproximationmay notincludethetrueparameterresponsiblefortheobserveddata.Therefore,wedevelopamethodtoproduceacoveroftheexactinverseimage,implicitly,byinating theuncertaintiesoneachobservedQoItoreectthediscretizationerrorexpectedin theapproximatesolutiontotheSIP.Thatis,wedenean ~ E 2B D suchthatthe approximation of Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E covers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E and,thus,isguaranteedtocontainthetrue parameter. VII.1.1CoveringtheExactInverseImage SupposewearegivenalinearQoImap,aVoronoitessellationoftheparameter space fV i g N i =1 denedimplicitlybyasetofi.i.d.randomsamples i N i =1 ,and anapproximationto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E forsomegeneralizedrectangle E 2B D .Asseenin FigureVII.1,thisapproximationmayexcluderegionsoftheexactinverseimage. 100

PAGE 118

Thus,thetrueparameterresponsiblefortheobserveddatamaynotbecontained withinthesupportoftheapproximateinverseimage. FigureVII.1: leftIngreen,weshowtheapproximateinverseimagewiththe givenVoronoitesselationof .Theexactinverseimageisshowninblue.Noticethe trueparameter true inrediscontainedwithintheexactinverseimage,however,itis notcontainedwithintheapproximation.rightInblue,weshowtheevent E 2B D thatdenes Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,thetruedatuminred,andtheobserveddatuminblackthatis responsibleforthedenitionof E 2B D Thisapproximateinverseimage,denoted Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,isdeterminedbyAlgorithm3. Inthiscase,thealgorithmsimplymapseach i N i =1 to D with Q and,if Q i 2 E then V i ispartoftheinverseimage,otherwise V i isnotpartoftheinverseimage. Theunder-approximatedregionsof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E resultfromthecoarsediscretizationof by fV i g N i =1 Tomodify Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E sothatitcoverstheexactinverseimage,wemustensurethat any V i suchthat V i Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E 6 = ; ispartoftheapproximateinverseimage.One approachtoobtainthisgoalistoextendtheboundariesof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E outward,bysome appropriatedistance,sothattheVoronoiapproximationtothisinatedparallelogram covers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,seeFigureVII.2. Thedistanceinwhichweextendtheboundariesof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isdeterminedbythe 101

PAGE 119

maximumradiusoftheVoronoicellsinthetessellation R N =max i =1 ;:::;N radius V i ; VII.1 wheretheradiusofaVoronoicellisdenedbytheradiusoftheminimumcircumscribingballcenteredat i radius V i :=sup 2V i d i ; : Wedenethis inated inverseimageimplicitlybasedonaninationof E 2B D FigureVII.2: Inpurpleweshow Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,inyellowweshowtheinatedinverse imageresultingfromextendingtheboundariesoutward,andingreenweshowthe approximationtotheinatedinverseimagethatcovers Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Notethat,this inationisdeterminedbyextendingtheboundariesoutwardeachdirectionbya distanceof R N ,themaximumradiusoftheVoronoicellsinthetesselation. VII.1.2Inating E 2B D toImplicitlyDeneaCover Theinationof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ischosensothattheapproximationtothisinationcovers theoriginal Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Here,wediscussanapproachtoimplicitlydenethisination basedoninating E 2B D to ~ E 2B D sothat Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E denesthisexactinationand, thus,theapproximationto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E ,denotedby Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E ,covers Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Recall,a Voronoicellintheapproximationofaninverseimagebelongstotheapproximation 102

PAGE 120

becauseitscenter i ismappedintothegeneralizedrectangleofuncertainty E 2B D i.e., Q i 2 E .Thus,wemayimplicitlydenethisinationbyinatingthe generalizedrectangleofuncertaintyinthedataspace E to ~ E ,sothattheboundaries of Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E areeachmovedoutwardby R N fromtheboundariesof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .We determinehowmuchtoinate E ineachcoordinatedirectionusingtheJacobianof themap Q Recall,weassumethat Q isalinearmapsothereexistsa d n matrix J ,such that Q = J .Letthe i throwof J bedenotedby J i ,then k J i k informsofusthe amount Q i changesas movesinthedirectionofthe n -dimensionalvectordenedby thisrow,moreover,this n -dimensionalvectorisorthogonaltothetwoboundariesof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E denedby Q i .Toextendeachoftheseboundariesoutwardby R N ,weextend thecorrespondingboundariesof E outwardby k J i k R N .Wedothisforeachofthe d componentsoftheQoImaptoobtain ~ E 2B D suchthat Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E covers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,see FigureVII.3. Thisapproachdoesnotproduceaminimumcoverof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E withthegiven Voronoitesselation fV i g N i =1 .IntermsofnonlinearQoImaps,wehavefoundthis methodgenerallyproducesalargecoverof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,solargethat,insomecases,it isnotsubstantiallymoreusefulthansimplypropogatingtheoriginaluncertaintyof theparametersforwardtomakeaprediction.However,thereareotherapproaches availablethatlookpromisinginregardstoproducingamuchtighterandmoreuseful coverof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E fornonlinear Q .Forexample,convolutionscanbeusedinsignal processingtosmoothnoisydata.Inthissameway,itappearsaconvolutioncanbe usedtosmooth Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E carefullysothattheresultingsmoothedinverseimagecovers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E .However,theresultsfromthissectionhavestillprovidedvaluableinsight intothedenitionofasingle-objectiveoptimizationproblemseeSectionVII.2denedintermsoftheQoImap,numberofsamples,anddimensionof .Also,the resultsinthissectionhaveinspiredanalternativedescriptionofthecomputational 103

PAGE 121

FigureVII.3: topleftTheinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E andtheapproximation.top rightThe E 2B D deningtheuncertainitesontheobserveddata.bottomright Theinated ~ E 2B D determinedbytheJacobianof Q and R N .bottomleftThe exactinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,theinatedinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E ,andtheresulting approximationto Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E thatcovers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E 104

PAGE 122

complexityofagivenSIPintermsofaboundonthesetapproximationerrorofthe resultinginverseimageseeSectionVII.3.Weleavetheexplorationofconvolutions toproducetightcoversof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E fornonlinearmapsandarbitrary E 2B D tofuture work. VII.2Single-ObjectiveOptimizationProblem InChapterV,wedenedthemeasure-theoreticSB-OEDproblemwithascalarizedmulti-objectiveoptimizationproblemwhereweminimizeaweightedsumofthe ESEandESKtodetermineanOED.However,wedidnotprovideareliablemethod fordetermininganoptimalweight inDenitionV.12.Intuitively,thisoptimal weightshouldbeafunctionofthenumberofsamplesavailableandthedimension oftheparameterspace,asthesequantitieswilllargelyimpactourabilitytoaccuratelyapproximateagiveninverseimage.Here,weusetheapproachintroduced inSectionVII.1toimplicitlyscalarizethemulti-objectiveproblem.Theresulting scalarizationisafunctionoftheQoImap,thenumberofsamplesavailable,andthe dimensionoftheparameterspace. Essentially,weutilizetheobservationsofSectionVII.1toquantifytheutility ofagivenEDintermsofthemeasureoftheinatedinverseimage Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E whose approximationyieldsacoverof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .WenotethatweperformthisSB-OED methodwiththeanticipationofsolvingtheoptimalSIPinthestandardway.In otherwords,oncewedetermineanOED,wedonotinatetheuncertaintieson theproleofthegivenobserveddensitytoensureacoverofthesolutiontotheSIP. Instead,weutilizethisapproachtoquantifylocalgeometricpropertiesofapossibly nonlinearQoImap.ThiscomputationissimilartocomputingtheESE.Weassume that E 2B D isgivenbyageneralizedrectangle,however,wequantifythe ination eect usingtheinated ~ E 2B D introducedabove.Thus,theresultingformforthe localinationeectofagiven Q nowhasa D ~ E inthenumerator,andthis ~ E is determinedbytheJacobianof Q ,andthusdependsonthelocallinearapproximation 105

PAGE 123

oftheQoImapunderconsideration. VII.2.1Ination AssumewearegivenanEDdenedbytheQoImap Q .GiventheJacobianof Q atsome 2 ,denoted J ;Q ,thelengthsofthesidesofthecorresponding ~ E denoted~ w k; ,aredeterminedby ~ w k; = w k +2 R N k J k; k ; wherethe w k arethewidthsoftheoriginalgeneralizedrectangleofuncertainty E 2 B D ,and J k; isthe k throwof J ;Q .TheLebesguemeasureof ~ E isgivenby D ~ E = d Y k =1 ~ w k; = d Y k =1 w k +2 R N k J k; k : VII.2 Notethat, D ~ E isafunctionoftheQoImapunderconsideration, 2 ,and R N denedinEq.VII.1. R N isafunctionofthenumberofsamplesusedtoconstruct thetessellation,andisimplicitlyafunctionofdim ,asweexpectatessellation usingthesamenumberofsamplesinahigherdimensionalspaceto,ingeneral,have alargermaximumradiusoftheVoronoicells.Theabovemotivatesthefollowing DenitionVII.1 Forany Q 2Q and 2 ,wedenethe localinationeect of Q as I Q = D ~ E Q d k =1 k ; where f k g 1 k d arethesingularvaluesoftheJacobian J ;Q and ~ E istheinated generalizedrectangledescribedabove.The expectedinationeectEIE isgiven by H I Q = 1 Z 1 I Q d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 : AswiththeESEandESK,wegenerallyapproximatetheEIEusingasetofi.i.d. 106

PAGE 124

randomsamples i N i =1 andcomputing H I Q H N I Q := 1 N N X i =1 1 I Q i )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 : VII.3 Wenotethat,thissingle-objectivedoesrelatetotheESEinanotableway.As thenumberofsamplesincreases,themaximumradiusofthevononoicellsgoesto zeroandtheEIEreducestotheESEscaledby D E .Formally TheoremVII.2 Assume Q : !D is B -measurableandGD.Let i N i =1 bea setofi.i.d.samplesinthecompact drawnfromaprobabilitydensity f suchthat f > 0 forall 2 .Then,forany 2 lim N !1 I Q = D E M Q : VII.4 Proof: In[61,39]itisshownthatlim N !1 R N =0.Thus, lim N !1 I Q =lim N !1 D ~ E Q d k =1 k =lim N !1 Q d k =1 w k +2 R N k J k; k Q d k =1 k = Q d k =1 w k Q d k =1 k = D E Q d k =1 k = D E M Q : Thisfollowsintuition,givenascenarioinwhichwecanaordarbitrarilymany modelsolves,wesimplywishtominimizetheESEanddonotcareaboutcomputationalcomplexitybecausewehavemanymodelsolvesavailabletoapproximatethe solutiontotheSIP.InExampleVII.2,weshowtheEIEforvarious N andnotethat, 107

PAGE 125

forsmall N ,therearesimilaritiesbetweentheEIEandtheESK. Wenotethat,unliketheESE,theshapeandmeasureofthegeneralizedrectangle E 2B D doimpacttheEIE.Specically,theproductofthesidelengthsof ~ E implies thenumeratorisnolongeraconstantforeachEDevenwhenthe E 2B D ineach dataspacehasthesameshapeandmeasure.Thismeansthat,inordertousethe EIEasanoptimizationcriterion,wemustspecifytheshapeof E 2B D ,i.e.,wemust havepriorknowledgeoftheproleoftheobserveddensityoneachQoI. ExampleVII.1TwoLinearEDs Inthisexample,weillustratetheinationeffectwithtwolinearmaps Q a and Q b denedby Q a = 2 6 4 10 01 3 7 5 ;Q b = 2 6 4 0 : 5051 : 178 1 : 1780 : 505 3 7 5 ; whereweassumethat E 2B D isgivenbyasquarewithsidelengths0.5.Notethat, inthiscase,theQoImapsarelinear,thus,weneednotapproximateanygradients oraveragethescalingeect,skewness,orinationeectovertheparameterspace. TheseQoImapswerechosensothatthescalingeectof Q b islessthanthescaling eectof Q a ,specically, 0 : 883 M Q b
PAGE 126

weshowrepresentativeinverseimagesofachosen E 2B D undereachoftheQoI maps. InFigureVII.5,weshowtheinverseoftheinationeectofeachQoImapasa functionofthenumberofsamplesavailable.Withfewsamplesavailable,theination eectdetermines Q a ismoreoptimalthan Q b ,despitethefactthat M Q b 400 samples. FigureVII.4: leftArepresentativeinverseimageresultingfrominverting E 2B D usingthemap Q a .rightArepresentativeinverseimageresultingfrominverting E 2B D usingthemap Q b .Although Q b producesasmallerinverseimageusing thesamesized E ,thecorrelationamongstthetwoQoIcomposing Q b producesan inverseimagethatismorediculttoapproximateaccuratelywithnitesampling. ExampleVII.2TimeDependentDiusion:UncertainDiusionCoecients Here,weconsideratime-dependentdiusionproblemwithtwouncertainparameters deningthethermalconductivitiesofametalrod.Wechoosethedataspacesunder considerationtobetwo-dimensionsaswell.WeconsideroptimizingtheEIEandillustratethedependenceofitsdescriptionofthedesignspaceonthenumberofmodel solvesavailabletoapproximatethesolutiontoaSIPresultingfromtheOED.See 109

PAGE 127

FigureVII.5: Theinverseoftheinationeect,forboth Q a and Q b ,asa functionofthenumberofsamplesavailable.Although M Q b 400, thechoicechangesand Q b istheOED. ExampleIV.4foradescriptionofthemodelandproblemsetup. Results Supposetheexperimentsinvolveplacingtwocontactthermometersalongthefull rodthatcanrecordtwoseparatetemperaturemeasurementsat t f .Thus,thereisa bijectionbetween and Q sopoints x 0 ;x 1 2 canbeusedtoindextheEDs. Wediscretizethecontinuousdesignspace Q using10,000uniformrandompointsin f x k 0 ;x k 1 g 10000 k =1 ,whichcorrespondsto10,000EDsdenotedby f Q k g 10000 k =1 where Q k = )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(u x k 0 ;t f ;u x k 1 ;t f foreach k .Weassumetheproleoftheobserved densitytobeuniformoverasquarewithsidelength0.2,thus, w 1 = w 2 =0 : 2 in Eq. VII.2 AsinExampleVII.1,weareinterestedinhowthenumberofmodelsolves, N availableforsolvingtheSIPimpactstheEIEoverthedesignspace.However,the QoIweconsiderinthisexamplearenownonlinear,andweestimateandsamplethe JacobiansinordertocomputetheEIE.TocontrolforsamplingerrorsinthecomputationoftheEIE,wex ^ N =5000 andusethissamesetofsamplesincomputing H ^ N I Q asgiveninEq. VII.3 .Inotherwords,theonlytermthatchangesinthis 110

PAGE 128

computationfordierent N isthevalueof D ~ E whichisgivenbyEq. VII.2 using adierentsetof N samplesfromthoseusedinthecomputationsoftheJacobians.In otherwords,foragiven N ,wegenerate N samplesandapproximate R N .Then,we usethexedsetof ^ N samplesandcompute f Q k i g ^ N i =1 10000 k =1 .Weapproximate f J i ;Q k g ^ N i =1 10000 k =1 usingaRBFinperpolationmethod,seeAppendixAfordetails. Wecalculate f H ^ N I Q k g 10000 k =1 usingEquation VII.3 InFigureVII.6,weshowtheresultingdescriptionsofthedesignspacefor N = 1 E + k for k =0 ; 1 ;:::; 5 .For N =1 E +0 ,theEIE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 resemblestheESK )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 seen inFigureV.5.Wedonotsuggestthatas N 0 theEIEconvergestotheESK becausetheESKisindependentofthemeasureoftheinverseimageandEIEisnot, however,thesimilaritiesarenotable.Asweincreasethenumberofsamplesavailable, theEIE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overthedesignspacechangesnoticeably.For N =1 E +5 ,theEIE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 looksverysimilartotheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,however,itdeterminestheOEDtobeinadierent locationthanthatdeterminedbytheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .Thisillusratesthepointmadeabovethat as N !1 ,theEIEconvergestotheESEmultipliedby D E VII.3ABoundontheErrorofSetApproximations Here,webuildontheideasfromSectionVII.1.Wemayconsiderthecover describedabovetobeanoutersetapproximationoftheregioninwhichsetapproximationerrormayexist.Inthesameway,wemay deate theinverseimageand deneaninnersetapproximationoftheregioninwhichsetapproximationerrormay exist.Thisinspiresthetopicofthissection:determiningaboundontheerrorofan approximationtoaninverseimage. Wefocusentirelyonsetapproximation,i.e.,wedenetheerrorofthesolution toaSIPtobethesymmetricdierencebetweenthesupportoftheexactinverse imageandthesupportoftheapproximateinverseimage.Thus,weassumethe supportoftheexactinverseimagedoesnotspantheentireparameterspace,i.e., 111

PAGE 129

FigureVII.6: TheEIE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingset forthedesignspace Q for variousnumbersofsamples N .Inthetopleft,theEIE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 looksverysimilartothe ESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 showninFigureV.5.Asweincreasethenumberofsamples,thelocationof theOEDdoesnotchange,however,thegeneraldescriptionof Q changesnoticably. WebegintoseesimilarfeaturestothatoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 seeninFigureV.4. 112

PAGE 130

Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isproperwhichalsosuggests E D isproper.Wedenetheerrorin anapproximationofasetusingthesymmetricdierence. DenitionVII.3SetApproximationError The error inanapproximation, ^ A ,to A 2B isgivenby Error ^ A = A 4 ^ A ; where istheLebesguemeasureon ,and 4 isthesymmetricdierenceoperator. VII.3.1InnerandOuterErrorBounds Asdescribedabove,assuming Q isalinearmapand E 2B D ageneralizerectangle,wemayinate E 2B D to ~ E 2B D sothatweguarantee Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E seeEq.VII.2.Wemaythinkofthisinatedimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E asanoutersetapproximationoftheregionof inwhichsetapproximationerrorto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E mayexist. Similarly,wemay defalate E 2B D to E ~ 2B D sothat Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ isaninnersetapproximationoftheregionof inwhichsetapproximationerrorto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E maylive. Thus, Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,andaboundontheerrorisgivenby Error Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E )]TJ/F20 11.9552 Tf 11.956 0 Td [( Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ where Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E istheapproximationto Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E istheinatedinverseimage, and Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ~ inthedeatedinverseimage.Formally, TheoremVII.4 Let R n and D R d ,with d n .Let ; B ; and D ; B D ; D bemeasurespacesusingtheBorel -algebra,where and D arethe Lebesguemeasuresdenedon and D ,respectively.Let Q : !D belinearand E 2B D ageneralizedrectangle.Let i N i =1 beasetofsamplesin implicitly deningtheVoronoitessellation fV i g N i =1 .Then,usingthediscretization fV i g N i =1 and Algorithm3,theerrorintheapproximation Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E to Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isboundedby Error Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E )]TJ/F20 11.9552 Tf 11.955 0 Td [( Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ : VII.5 113

PAGE 131

Proof: Theboundfollowsfromtheobservationthat Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ~ Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E whichimplies Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E 4 Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E n Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ whichimplies Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E )]TJ/F20 11.9552 Tf 11.955 0 Td [( Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ~ : WeillustratethebounddescribedinTheoremVII.4intheexamplebelow. ExampleVII.3Linear ConsiderthelinearQoImap Q denedby Q = 2 6 4 0 : 2490 : 249 0 : 249 )]TJ/F15 11.9552 Tf 9.299 0 Td [(0 : 0415 3 7 5 : First,wefocusontheerrorinapproximationsmadetotheinverseimageofasquare withsidelengths0.1withauniformgridof N samples.InFigureVII.7,weshow theapproximateinverseimage,theexactinverseimage,theinnerandouterset approximationsoftheerror,andtheerror.Intheleftcolumn,theexactinverse imageisshowninblueanditsapproximationingreen.Intherightcolumn,the errorisshowninred,theoutersetapproximationinorange,andtheinnersetapproximationinbeige.Weseethatas N increases,boththeerrorandthebound ontheerrorbecomeconnedtoasmallerregionaroundtheboundaryoftheexact inverseimage.InTableVII.1,weshowtheerrorandtheboundontheerrorfor N =100 ; 192 ; 400 ; 784 ; 1600 samplesonaregulargrid. Next,weconsidertheerrorboundfor N =100 ; 192 ; 400 ; 784 ; 1600 uniformrandomsamples.InFigureVII.8,weshowtheapproximateinverseimage,theexact 114

PAGE 132

NError Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Errorbound 1000.04560.234 1920.03190.168 4000.02380.117 7840.01330.0839 16000.01150.0587 TableVII.1: TheMCapproximationto Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheerrorboundgiven inEq.VII.5usingaregulargridofsamples. inverseimage,theinnerandoutersetapproximationsontheerror,andtheerror. Intheleftcolumn,theexactinverseimageisshowninblueanditsapproximationin green.Intherightcolumn,theerrorisshowninred,theoutersetapproximationin orange,andtheinnersetapproximationinbeige.Weseethatas N increases,both theerrorandtheboundontheerrorbecomeconnedtoasmallerregionaroundthe boundaryoftheexactinverseimage.InTableVII.1,weshowtheerrorandthebound ontheerrorfor N =100 ; 192 ; 400 ; 784 ; 1600 uniformrandomsamples. NError Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Errorbound 1000.04770.517 1920.03290.388 4000.03090.263 7840.02140.1971 16000.01360.146 TableVII.2: TheMCapproximationto Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheerrorboundgiven inEq.VII.5usinguniformrandomsamples.. VII.4Summary Weconcludethissectionwithseveralpromisingdirectionsforfuturework.First, theinabilityofthecoverdescribedinSectionVII.1tobetightaroundtheboundaryof theinverseimagefornonlinearQoImapssuggestsanalternativeapproachtocovering theseimages.AsdiscussedattheendofSectionVII.1,convolutionsappeartobean appropriatetooltoresolvethisissue.Ineachoftheabovesections,theresultsare 115

PAGE 133

intimatelytiedtotheboundaryoftheinverseimage.Forexample,inSectionVII.3, weobservetheregionthatboundstheerrortoappeartobestronglydetermined bythepropertiesoftheboundaryoftheinverseimage.Thissuggeststheexpected errorinsetapproximationsmaybedeterminedasafunctionofthemeasureofthe boundaryofthesetasalower-dimensionalobject.Thepursuitofthistopicisbeyond thescopeofthisthesis,however,wenotethatinitialexplorationoftheboundaries ofparallelepipedsproducesasimilarscalarizationtothemulti-objectiveoptimization problemdescribedinSectionVII.2. 116

PAGE 134

FigureVII.7: TheMCapproximationto Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheouterandinner setapproximationusingaregulargridofsamplesfor N =192 ; 400 ; 784 ; 1600from toptobottom 117

PAGE 135

FigureVII.8: TheMCapproximationto Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheouterandinner setapproximationusinguniformrandomsamplesfor N =192 ; 400 ; 784 ; 1600from toptobottom. 118

PAGE 136

CHAPTERVIII AGREEDYIMPLEMENTATIONFORSB-OED Oftentimes,applicationsyieldhigh-dimensionalparameter,data,anddesign spaces.Thesehigh-dimensionalspacesmakeuncertaintyquanticationdicultin manyways,e.g.,accurateapproximationofasolutiontotheSIP,thepushforward oftheprior,andtheOEDbecomecomputationallyexpensive.Here,wefocusonthe OED.Insomecases,itmaybeinfeasibletoexhaustivelysearchthedesignspaceto determinetheOED.Forexample,givenanapplicationwithabudgetofgathering nineQoImeasurementsfromasetof100possibleQoIyields )]TJ/F18 7.9701 Tf 5.479 -4.379 Td [(100 9 =1 : 9 E +12possibleEDs.Ratherthanacombinatorialsearchofthedesignspace,weproposeagreedy algorithmtoavoidthiscomputationalexpense. VIII.1AGreedyAlgorithm WedescribeagreedyalgorithmforSB-OED.WechoosetofocusontheESE, however,thisalgorithmisappropriateforanyoftheoptimizationcriteriadiscussed above.First,wecomputetheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 foreachoftheEDsdenedbyasingleQoI.The QoIthatdenestheEDwiththegreatestESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 isthentheQoIdeningtheOED forasingleQoI. Given thisOEDforasingleQoI,wethencomputetheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthe EDsdenedbyatwo-dimensionalvector-valuedQoIwhereoneofthecomponents istheOEDforasingleQoI.Again,thegreedyoptimalpairofQoIisgivenbythe EDfromthisdesignspacewiththegreatestESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .Thealgorithmcontinuesin thiswayuntilthedimensionofthevector-valuedQoImapmatchesthedimensionof thedesireddataspace D thatisspeciedatthebeginningofthealgorithm.This issummarizedinAlgorithm5wherewelet Q d denoteadesignspacedenedby vector-valuedQoImapsofdimension d wheretherst d )]TJ/F15 11.9552 Tf 12.041 0 Td [(1componentsarechosen bythepreviousstepsinthealgorithm. SomegreedySB-OEDmethodsgatherthedataforthecurrentED,solvean inverseproblem,andthenusetheresultingsolutiontotheinverseproblemtoinform 119

PAGE 137

theexpectationdeningtheESEforthenextstep.Wenotethat,thealgorithm proposedheredoes not requirethesolutionofaSIPbeforedeterminingthenext optimalQoI.Ateachstep,wecomputetheexpecationintheESEusingthesameset ofi.i.d.samplesin Algorithm5: AGreedyAlgorithmforSB-OED 1.Specifyadesireddim D 2.Let Q denoteallpossibleEDscomposedofasingleQoI. for d =1 ;:::; dim D do Q opt =argmax Q 2Q d ESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q if d > > > > > < > > > > > > : c @u @t = r r u + S;x 2 ;t 2 t 0 ;t f ; @u @n =0 ;x 2 @ ; u x ;0=0 ; VIII.1 with S x = A exp )]TJ 13.15 8.088 Td [(jj x src )]TJ/F20 11.9552 Tf 11.955 0 Td [(x jj 2 w ; 120

PAGE 138

where=[ )]TJ/F15 11.9552 Tf 9.299 0 Td [(0 : 5 ; 0 : 5] 2 representsanalloyplate, =1 : 5isthedensityoftheplate, c =1 : 5istheheatcapacity, isthethermalconductivity,and S isaGaussian sourcewiththefollowingsourceparameters: x src =[0 ; 0]isthelocation, A =50 istheamplitude,and w =0 : 05isthewidth.Welet t 0 =0, t f =2,andimpose homogeneousNeumannboundaryconditionsoneachboundaryoftheplate. Supposetheplateismanufacturedbyweldingtogetherninesquareplatesof thesameshapeandofsimilaralloytype,seeFigureVIII.1.However,duetothe manufacturingprocess,theactualalloycompositionsmaydierslightlyleadingto uncertainthermalconductivityproperties ineachofthenineregionsoftheplate, denotedby 1 ;:::; 9 .Thatis,welet =[0 : 01 ; 0 : 2] 9 Giventhethermalconductivityvaluesforeachofthenineregionsoftheplate, weapproximatesolutionstothePDEinEq.VIII.1usingthestate-of-the-artopen sourceniteelementsoftwareFEniCS[3,47]witha40 40triangularmesh,piecewise linearniteelements,andaCrank-Nicolsontimesteppingscheme. FigureVIII.1: Thedomainisconstructedbyweldingtogetherninesquareplates, eachofwhichhasconstant,butuncertain,thermalconductivity VIII.2.2ChooseOneSensor Supposetheexperimentsinvolveplacingninecontactthermometersontheplate thatcanrecordnineseparatetemperaturemeasurementsat t f .Wediscretizeusing 121

PAGE 139

10,000uniformrandompoints, f x k 0 ;x k 1 g 10000 k =1 .Thissetofpointsservesastheset ofpossiblesensorlocationsQoIin,fromwhichwemustchoosenine.TheOED isgivenbytheEDthatmaximizestheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,however,giventhisdiscretizationof ,thereare )]TJ/F18 7.9701 Tf 5.48 -4.378 Td [(10 ; 000 9 =2 : 7 E +30possibleEDs.Inordertoavoidthiscombinatorial computationalexpense,weuseAlgorithm5toapproximatetheOEDfromthisset ofEDs. First,weconsider Q ,thespaceofEDsdenedbyplacingasinglecontact thermometerontheplatethatcanrecordthetemperatureat t f .Thus,thereisa bijectionbetweenand Q sopoints x 0 ;x 1 2 canbeusedtoindextheEDs.We let Q k = u )]TJ/F15 11.9552 Tf 5.479 -9.684 Td [( x k 0 ;x k 1 ; t f foreach k .These Q k denoteallofthepossiblecomponents thatmakeuptheQoImapsthatdenetheEDsforthisrstsensorlocationandall subsequentsensorlocations.Thus,foreach Q k 2Q Q k = Q k .Wegenerate 1,000samples, f i g 5000 i =1 ,in andapproximate f J i ;Q k g 1000 i =1 10000 k =1 usinganRBF interpolationmethod,seeAppendixAfordetails.Wenotethat,theseJacobiansare actuallygradientsanddonotneedtobecomputedagainafterthis.Wecalculate theESEforeach Q z 2Q usingEq.V.17. InFigureVIII.2,weshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingsetforthecurrent designspace Q .WeobservethattheOEDislocatedatthecenterorthedomain, nearthelocationofthesource.Thechosenmanufacturingofthemetalplateby weldingtogetherninesquareplatesisvisiblewiththissimulateddata.Thus,wehave determinedtheoptimallocationforasinglesensorandwexthisdecisionthroughout therestofthenextsection.DenotethisEDas Q opt 2Q VIII.2.3ChooseKSensors Now,weconsider Q ,thespaceofEDsdenedbyplacingtwocontactthermometersontheplatethatcanrecordthetemperatureat t f ,wheretherstcontactthermometerisplacedattheoptimalsinglesensorlocationgivenaboveand thesecondcontactthermometerisplacedatanyofthe10,000possiblesensorlo122

PAGE 140

FigureVIII.2: TheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingsetforthedesignspace.Noticethe OEDisshowninwhite,nearthecenterofthedomainwherethesourceislocated. 123

PAGE 141

cations.Thus,thereisagainabijectionbetweenand Q .Welet Q = Q opt 2Q ;Q z : Q z 2Q .WecalculatetheESEforeach Q z 2Q using Eq.V.17. InFigureVIII.3top,weshowtheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 overtheindexingsetforthecurrent designspace Q .WeobservethattheOEDisapairofsensors,onedenedby Q opt 2Q ,theotherlocatedtotheleftofthecenterofthedomainnearthe boundaryoftwooftheplatesthatwereweldedtogether.Wenotethat,itappears therearefourlocationsthatwouldproducesimilarESEvalues:totheleft,top,right, andbottomoflocationofthesource.Thisisnotsurprisingduetothesymmetryof theproblem.ItisinterestingthattherearelocalminimumintheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 aswemove fromthecenterofthedomainoutwardtowardstheboundaries.Wehavedetermined thelocationfortwosensorsandwexthisdecisionthroughouttherestofthissection. DenotethisEDas Q opt 2Q Inthisway,wecontinuetoplacethenextoptimalsensorin,seeFiguresVIII.3 {VIII.6.Ineachgure,thenextoptimalsensorlocationisshowninwhite,and thepreviouslychosensensorlocationsareshowninblack.Noticethatafterasensor inplacedinaregion,inthesubsequentgures,theregionaroundthatsensorhas relativelylowESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 values. 124

PAGE 142

FigureVIII.3: topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensors areshowninblack,andtheoptimallocationforthenextsensorisshowninwhite atthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthe previouslyplacedsensorsareshowninblack,andtheoptimallocationforthenext sensorisshowninwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 125

PAGE 143

FigureVIII.4: topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensors areshowninblack,andtheoptimallocationforthenextsensorisshowninwhite atthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthe previouslyplacedsensorsareshowninblack,andtheoptimallocationforthenext sensorisshowninwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 126

PAGE 144

FigureVIII.5: topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensors areshowninblack,andtheoptimallocationforthenextsensorisshowninwhite atthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthe previouslyplacedsensorsareshowninblack,andtheoptimallocationforthenext sensorisshowninwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 127

PAGE 145

FigureVIII.6: topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensors areshowninblack,andtheoptimallocationforthenextsensorisshowninwhite atthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthe previouslyplacedsensorsareshowninblack,andtheoptimallocationforthenext sensorisshowninwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 128

PAGE 146

CHAPTERIX CONCLUSIONS IX.1Summary Thisthesishasdevelopedabroadsetoftoolstoemploytoecientlydetermine anOEDinthecontextofSB-OED.Webeganwithacontributiontothecurrentstate ofBayesianSB-OEDinwhichweemploytherecentlydevelopedconsistentBayesian approachtosolvingSIPstoecientlyintegrateovertheobservableQoIspaceduring computationoftheEIGofagivenED.Thedevelopmentofthismethodrequiredthe adaptationofanobserveddensityspacetoconformtotheassumptionrequiredinthe consistentBayesianformulation,anissuethatisavoidedintheclassicalBayesianSBOEDformulation.WefollowedthiswithanovelperspectiveonSB-OEDinspiredby theset-basedframeworkofthemeasure-theoreticapproachtoSIPs.Thisperspective proposedadenitionoftheutilityofagivenEDbasednotonsolutionstoSIPS, butonlocalgeometricpropertiesinherenttotheQoImapdenedbytheED.This perspectiveallowsforaquanticationoftheutilityofagivenEDthatisindependent ofboththemethodologyemployedtosolvethechosenSIPandthespecicationofa probabilitymeasureonthespaceofobservableQoI. ThesimilaritiesoftheresultsfromthesetwoSB-OEDapproachesencouraged anextensivecomparisonbetweenthetwoapproachesfocusedonahandfulofchallengingSB-OEDscenarios.Fromthiscomparison,weconcludedthat,ingeneral, themeasure-theoreticapproachallowsforamorerobustandsystematicapproachto SB-OEDinwhichthedescriptionsofagivendesignspacearesubstantiallymoreconsistentinthissetofchallengingSB-OEDscenarios.ThedependenceofthemeasuretheoreticapproachtotheinherentlocalpropertiesofagivenQoImapprovidethis robustinformationinthesedicultsettings,whereasthedependenceoftheconsistentBayesianapproachonactualsolutionstoSIPsprovedtoobscureresultsin challengingscenarios. 129

PAGE 147

Althoughwedeterminedthemeasure-theoreticapproachtoprovideamoresystematicmethodforSB-OED,weneglectedtoexploretheimpactoftheESKon comparisonswiththeEIG.Weconcludedthisthesiswithaseriesofresultsregarding approximationerrorsandcomputationalcomplexityofagivenSIP.Thisexploration providedacleardenitionofasingle-objectiveoptimizationproblemdenedimplicitlyintermsofthesensitivitiesoftheQoImap,thedimensionoftheparameterspace, andthenumberofmodelsolvesavailabletoultimatelyapproximatethesolutiontoa chosenSIP.Thisnalresultsprovidesaclearpathforimplementationofthismethod toalargeclassofapplicationsinwhichmodelsolvesareexpensive,andtherefore limited,resultingincoarseapproximationstothesolutionofagivenSIP. IX.2FutureWorkandOpenProblems Throughoutthisthesis,weeludedtoseveralripeopportunitiesforfuturework. Below,weconsolidatethoseideas. IX.2.1ContinuousOptimization Firstandforemost,thepursuitofacomputationallyecientapproachforcouplingtheSB-OEDmethodsdevelopedwithinthisthesiswithcontinuousoptimization techniquesisanintriguingtopicforfuturework.Wehaverestrictedourselvestorelativelylowdimensionaldesignspacesinthisthesis,theextensionofthisworkto high-dimensionaldesignspaceswoulddemandecientmethodsincontinuousoptimizationasanexhaustivesearchofthedesignspacewouldbecomeintractable. IX.2.2SurrogatestoAccelerateandImproveApproximationoftheEIG WeillustratedadependenceoftheEIGonthenumberofmodelsolvesavailable tocomputerelevantapproximations.Thepursuitofsurrogatemodelingtoboth acceleratethecomputationoftheEIGandincreasetheaccuracyinthescenarioof fewmodelsolvesavailableisaninterestingtopicoffutureresearch. IX.2.3OptimalDesignSpace Thedeterminationofthe optimal designspacetoconsiderinanygivenapplicationisofgreatinterest.Asdiscussedseveraltimesthroughoutthisthesis,notall 130

PAGE 148

PAGE 149

beusedtosmoothagivenapproximateinverseimagesothattheresultingsmoothed inverseimagecoversthesoughtaftersolution. IX.2.6ImproveSystematicSB-OEDFormulation ThecontinuedimprovementoftheSB-OEDmethodsdescribedinthisthesisis anobviousprojecttopursue.Asweencountermoreapplicationsandmoredicult SB-OEDscenarios,wewillundoubtedlybeencouragedtomodifyandimproveupon thecurrentmethodswithinthisthesis.Theultimategoalofacompletelyhands-o approachtoSB-OEDapplicabletonearlyallmodelingscenariosofinterestislikely unrealistic,however,themoresystematicandrobustthecurrentsmethodsbecomeis ofgreatusetothescienticmodelingcommunity. 132

PAGE 150

REFERENCES [1]AlenAlexanderian,NoemiPetra,GeorgStadler,andOmarGhattas.A-optimal designofexperimentsforinnite-dimensionalbayesianlinearinverseproblems withregularized l 0 -sparsication. SIAMJournalonScienticComputing 36:A2122{A2148,2014. [2]AlenAlexandrian,NoemiPetra,GeorgStadler,andOmarGhattas.Afast andscalablemethodfora-optimaldesignofexperimentsforinnite-dimensional bayesiannonlinearinverseproblems. SIAMJournalonScienticComputing 38:A243{A272,2016. [3]MartinAlns,JanBlechta,JohanHake,AugustJohansson,BenjaminKehlet, AndersLogg,ChrisRichardson,JohannesRing,MarieRognes,andGarthWells. TheFEniCSProjectVersion1.5. ArchiveofNumericalSoftware ,3,2015. [4]BillyAmzal,FrdricYBois,EricParent,andChristianPRobert.Bayesianoptimaldesignviainteractingparticlesystems. JournaloftheAmericanStatisticalAssociation ,101:773{785,2006. [5]A.Atkinson,A.Donev,andR.Tobias. OptimumExperimentalDesigns,with SAS .OxfordUniversityPress,2007. [6]AnthonyCAtkinsonandAlexanderNDonev.Theconstructionofexactdoptimumexperimentaldesignswithapplicationtoblockingresponsesurfacedesigns. Biometrika ,pages515{526,1989. [7]VictorBayona,MiguelMoscoso,ManuelCarretero,andManuelKindelan.Rbffdformulasandconvergenceproperties. JournalofComputationalPhysics 229:8281{8295,2010. [8]JosM.Bernardo.Expectedinformationasexpectedutility. TheAnnalsof Statistics ,7:686{690,1979. [9]F.Bisetti,D.Kim,O.Knio,Q.Long,andR.Tempone.Optimalbayesian experimentaldesignforpriorsofcompactsupportwithapplicationtoshock-tube experimentsforcombustionkinetics. Int.J.NumericalMethodsinEngineering 108:136{155,2016. [10]J.Breidt,T.Butler,andD.Estep.AMeasure-TheoreticComputationalMethod forInverseSensitivityProblemsI:MethodandAnalysis. SIAMJournalon NumericalAnalysis ,49:1836{1859,2011. [11]MartinDBuhmann. Radialbasisfunctions:theoryandimplementations ,volume12.Cambridgeuniversitypress,2003. [12]T.Butler,D.Estep,andJ.Sandelin.AComputationalMeasureTheoretic ApproachtoInverseSensitivityProblemsII:APosterioriErrorAnalysis. SIAM JournalonNumericalAnalysis ,50:22{45,2012. 133

PAGE 151

[13]T.Butler,D.Estep,S.Tavener,C.Dawson,andJ.J.Westerink.AMeasureTheoreticComputationalMethodForInverseSensitivityProblemsIII:Multiple QuantitiesofInterest. SIAMJournalonUncertaintyQuantication ,pages1{27, 2014. [14]T.Butler,L.Graham,D.Estep,C.Dawson,andJ.J.Westerink.Denitionand solutionofastochasticinverseproblemforthemanning'snparametereldin hydrodynamicmodels. AdvancesinWaterResources ,78:60{79,2015. [15]T.Butler,L.Graham,S.Mattis,andS.Walsh.Ameasure-theoreticinterpretationofsamplebasednumericalintegrationwithapplicationstoinverseand predictionproblemsunderuncertainty.SubmittedtoSIAMJ.Sci.Comput., 2016. [16]T.Butler,J.Jakeman,andT.Wildey.Aconsistentbayesianformulationfor stochasticinverseproblemsbasedonpush-forwardmeasures.Submittedto SIAMJ.Sci.Comput.,2016. [17]T.Butler,M.Pilosov,andS.Walsh.Simulation-basedoptimalexperimental design:Ameasure-theoreticperspective.InReview,2016. [18]KathrynChaloner.Anoteonoptimalbayesiandesignfornonlinearproblems. JournalofStatisticalPlanningandInference ,37:229{235,1993. [19]KathrynChalonerandKinleyLarntz.Optimalbayesiandesignappliedtologistic regressionexperiments. JournalofStatisticalPlanningandInference ,21:191{ 208,1989. [20]KathrynChalonerandIsabellaVerdinelli.Bayesianexperimentaldesign:A review. StatisticalScience ,10:273{304,1995. [21]MerliseAClyde. Bayesianoptimaldesignsforapproximatenormality .PhD thesis,UniversityofMinnesota,1993. [22]T.A.CoverandJ.A.Thomas. ElementsofInformationTheory .JohnWiley& Sons,2006. [23]M.CrampinandF.A.E.Pirani. ApplicableDierentialGeometry .Cambridge UniversityPress,1987. [24]C.DellacherieandP.A.Meyer. ProbabilitiesandPotential .North-HollandPublishingCo.,Amsterdam,1978. [25]HolgerDetteandLindaMHaines.E-optimaldesignsforlinearandnonlinear modelswithtwoparameters. Biometrika ,81:739{754,1994. [26]GregoryFasshauerandMichaelMcCourt. Kernel-basedapproximationmethods usingMatlab ,volume19.WorldScienticPublishingCoInc,2015. 134

PAGE 152

PAGE 153

[41]AndreIKhuriandSiuliMukhopadhyay.Responsesurfacemethodology. Wiley InterdisciplinaryReviews:ComputationalStatistics ,2:128{149,2010. [42]J.Kiefer.Optimumexperimentaldesigns. JournaloftheRoyalStatisticalSociety.SeriesBMethodological ,21:272{319,1959. [43]JackKieferandJacobWolfowitz.Optimumdesignsinregressionproblems. The AnnalsofMathematicalStatistics ,pages271{294,1959. [44]S.KullbackandR.A.Leibler.Oninformationandsuciency. TheAnnalsof MathematicalStatistics ,22:79{86,1951. [45]ElisabethLarssonandBengtFornberg.Anumericalstudyofsomeradialbasis functionbasedsolutionmethodsforellipticpdes. Computers&Mathematics withApplications ,46:891{902,2003. [46]D.V.Lindley.Onameasureoftheinformationprovidedbyanexperiment. The AnnalsofMathematicalStatistics ,27:986{1005,1956. [47]AndersLogg,Kent-AndreMardal,andGarthWells,editors. AutomatedSolutionofDierentialEquationsbytheFiniteElementMethod .SpringerBerlin Heidelberg,2012. [48]QuanLong,MohammadMotamed,andRaulTempone.Alaplacemethodfor under-ddeterminedbayesianoptimalexperimentaldesigns. ComputerMethods inAppliedMechanicsandEngineering ,285:849{876,2015. [49]QuanLong,MarcoScavino,RaulTempone,andSuojinWang.Fastestimationofexpectedinformationgainsforbayesianexperimentaldesignsbasedon laplaceapproximations. ComputerMethodsinAppliedMechanicsandEngineering ,259:24{39,2013. [50]QuanLong,MarcoScavino,RaulTempone,andSuojinWang.Fastbayesian optimalexperimentaldesignforseismicsourceinversion. ComputerMethodsin AppliedMechanicsandEngineering ,291:123{145,2015. [51]RichardALuettichJr,JohannesJWesterink,andNormanWSchener.Adcirc: Anadvancedthree-dimensionalcirculationmodelforshelves,coasts,andestuaries.report1.theoryandmethodologyofadcirc-2ddiandadcirc-3dl.Technical report,DTICDocument,1992. [52]CharlesAMicchelli.Interpolationofscattereddata:distancematricesand conditionallypositivedenitefunctions.In Approximationtheoryandspline functions ,pages143{145.Springer,1984. [53]AnnaM.MichalakandPeterK.Kitanidis.Estimationofhistoricalgroundwater contaminantdistributionusingtheadjointstatemethodappliedtogeostatistical inversemodeling. WaterResourcesResearch ,40:n/a{n/a,2004.W08302. 136

PAGE 154

[54]PeterMuller,DonA.Berry,AndyP.Grieve,MichaelSmith,andMichaelKrams. Simulation-basedsequentialbayesiandesign. JournalofStatisticalPlanningand Inference ,137:3140{3150,2007. [55]PeterMller,BrunoSans,andMariaDeIorio.Optimalbayesiandesignbyinhomogeneousmarkovchainsimulation. JournaloftheAmericanStatisticalAssociation ,99:788{798,2004. [56]R.-E.Plessix.Areviewoftheadjoint-statemethodforcomputingthegradient ofafunctionalwithgeophysicalapplications. GeophysicalJournalInternational 167:495,2006. [57]KennethJ.Ryan.Estimatingexpectedinformationgainsforexperimentaldesignswithapplicationstotherandomfatigue-limitmodel. JournalofComputationalandGraphicalStatistics ,12:585{603,2003. [58]K.Sargsyan,H.N.Najm,andR.Ghanem.Onthestatisticalcalibrationof physicalmodels. InternationalJournalofChemicalKinetics ,47:246{276, 2015. [59]A.M.Stuart.Inverseproblems:ABayesianperspective. ActaNumerica 19:451{559,2010. [60]Ne-ZhengSunandWilliamW.-G.Yeh.Coupledinverseproblemsingroundwater modeling:1.sensitivityanalysisandparameteridentication. WaterResources Research ,26:2507{2525,1990. [61]S.TavenerT.WildeyC.DawsonT.Butler,D.EstepandL.Graham.Solving stochasticinverseproblemsusingsigma-algebrasoncontourmaps. [62]AlbertTarantola. InverseProblemTheoryandMethodsforModelParameter Estimation .siam,2005. [63]JojannekevanDenBerg,AndrewCurtis,andJeannotTrampert.Optimalnonlinearbayesianexperimentaldesign:anapplicationtoamplitudeversusoset experiments. GeophysicalJournalInternational ,155:411,2003. [64]T.vanErvenandP.Harremoes.Renyidivergenceandkullback-leiblerdivergence. IEEETransactionsonInformationTheory ,60:3797{3820,2014. [65]IsabellaVerdinelli.Advancesinbayesianexperimentaldesign. BayesianStatistics ,4:467{481,1992. [66]S.Walsh,T.Wildey,andJ.Jakeman.Optimalexperimentaldesignusinga consistentbayesianapproach.SubmittedtoASCE-ASMEJournalofRiskand UncertaintyinEngineeringSystems,PartB:MechanicalEngineering,2017. [67]MPWandandMCJones.Multivariateplug-inbandwidthselection. ComputationalStatistics ,9:97{116,1994. 137

PAGE 155

PAGE 156

APPENDIXA.RADIALBASISFUNCTIONSFORDERIVIATIVE APPROXIMATIONS TheuseofradialbasisfunctionsRBFstointerpolatedataonanunstructured setofsamplesisawell-studiedtopic[70,26,34,52,27].Thesemethodshavefound awiderangeofapplicationsfromapproximatingsolutionstoPDEs[45,37,72,11] tomachinelearningalgorithms[71,26]toapproximatinggradients[31,7].In[7], theconvergenceoftheRBFmethod,withrespecttoapproximatingdierentialopperators,isstudiedforagivensetofequispacedsamples.Here,weconsiderthe convergenceoftheRBFmethod,withrespecttoapproximatinggradients,onan arbitrarysetofuniformrandomsamples. InSectionA.1,wereviewthewell-knownRBFmethodforinterpolatingdataon anunstructuredsetofsamples.InSectionA.2,wedisplaytheconvergenceofthe methodforapproximatinggradientsontwooscillatorytestfunctions. A.1UsingRBFstoInterpolateUnstructuredData Supposewearegivenasetoffunctionevaluations f f ;:::;f N g ,where ;:::; N 2 aredistinct,forsomefunction f : R n R .Then,an interpolantisgivenby[70], s N = N X i =1 i )]TJ 7.472 0.479 Td [( )]TJ/F20 11.9552 Tf 11.955 0 Td [( i ; where issomeRBFandthe i aredeterminedbysolvingthelinearsystem A = f where a j;k = )]TJ 7.472 0.478 Td [( j )]TJ/F20 11.9552 Tf 11.955 0 Td [( k = 1 ;:::; N ,and f = )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(f ;:::;f N .This linearsystemenforcestheinterpolationconditions s N i = f i for i =1 :::N Whenweareinterestedinapproximatingthegradientof f atsomepoint 2 thenwemaychoosetoconstructonlyalocalinterpolantwith K
PAGE 157

Oftentimes,thepartialderivativesof ,denoted @ j for j =1 ;:::;n =dim ,are easilydeterminedanalytically.Inwhichcase,thepartialderivatives @ j s K of s K are givenby, @ j s K = K X k =1 i @ j )]TJ 7.472 0.478 Td [( )]TJ/F20 11.9552 Tf 11.956 0 Td [( k for j =1 ;:::;n: InAlgorithm6,weillustratethestepsinvolvedinapproximating r f using thisRBFmethod.Computationally,themostsignicantexpensescomefromthe nearestneighborsearchandsolutionofthelinearsystemtodeterminethecoecients k Algorithm6: RBFGradientApproximation 1.Givenasetofsamples: i 2 ;i =1 ;:::;N ; 2.Given f i ;i =1 ;:::;N ; 3.Given = i forsome i 2f 1 ;:::;N g ; 4.Findthe K nearestneighborsof : k ;k =1 ;:::;K ; 5.Determinetheinterpolationweights k bysolvingthelinearsystemdenedby theinterpolationconditions s K k = f k for k =1 ;:::;K ; 6.For j =1 ;:::;n =dim ,derivethepartialderivativesof s K @ j s K = K X k =1 k @ j )]TJ 7.472 0.479 Td [( )]TJ/F20 11.9552 Tf 11.955 0 Td [( k 7.For j =1 ;:::;n =dim ,evaluate @ j s K 8.Construct r s K r s K = @ 1 s K ;:::;@ n s K Therearemanypossiblechoicesfor [70,26].Here,wechoose tobeamultivariateGaussianoftheform, r = e )]TJ/F21 7.9701 Tf 6.587 0 Td [(r ; 140

PAGE 158

where istheso-calledshapeparameterand r isthedistancefromthecenterofthe Gaussianto .Wenotethat,acurrenttopicofresearchisthecarefuldeterminiation ofanoptimal [7,68,28],however,adiscussionofthistopicisbeyondthescopeof thiswork.Thechoiceoftheoptimalnumberofnearestneighborstoconsider K is alsoatunableparameter.Forsimplicity,inallofthecomputationsthatfollow,we choose =1and K =10 3 dim )]TJ/F18 7.9701 Tf 6.586 0 Td [(2 A.2ConvergenceoftheExpectedRelativeError Here,weconsidertheconvergenceofthismethodasafunctionofthenumber ofuniformrandomsamplesavailablein .First,weintroduceanoscillatorytest function f : =[0 ; 1] 2 R denedby, f =sin 1 sin 2 ; r f = )]TJ/F15 11.9552 Tf 5.48 -9.684 Td [(2 cos 1 sin 2 ; 2 sin 1 cos 2 : A.1 Weconsiderapproximationsof r f atthepoint = : 200 ; 0 : 249 2 ,where r f : 942 ; 0 : 038.For N =20 ; 40 ; 80 ;:::; 20480,weapproximate r f usingAlgorithm6with K =10,i.e.,weusethe10nearestneighborstoapproximate thegradient.Wecomputeeachapproximation1,000timesusingdierentsetsof N uniformrandomsamplesandcomputetheexpectedrelativeerror.InFigureA.1, weshowtheconvergenceoftheexpectedrelativeerroroftheapproximationofeach componentof r f andthe O 1 N convegencerateforreference.Although,for each N ,theexpectedrelativeerrorfor @ 2 f isordersofmagnitudelargerthan thatof @ 1 f ,bothconvergeatarateofapproximately O 1 N .Notethat,we expectthismethod,oranymethod,tohavedicultyapproximating @ 2 f because thiscomponentisveryclosetozerowhichcausesrelativeerrorstobeamplied. 141

PAGE 159

FigureA.1: Theconvergenceoftheexpectedrelativeerrorforeachcomponentof Eq.A.1.Noticeeachcomponentof r f convergesatapproximatelytherate O 1 N Next,weconsidertheoscillatoryfunction f : =[0 ; 1] 4 R denedby, f =sin 1 sin 2 sin 3 sin 4 ; r f = )]TJ/F15 11.9552 Tf 5.479 -9.683 Td [(2 cos 1 sin 2 sin 3 sin 4 ; A.2 2 sin 1 cos 2 sin 3 sin 4 ; 2 sin 1 sin 2 cos 3 sin 4 ; 2 sin 1 sin 2 sin 3 cos 4 : Weconsiderapproximationsof r f atthepoint = : 200 ; 0 : 249 ; 0 : 100 ; 0 : 150 2 where r f : 923 ; 0 : 018 ; 3 : 911 ; 2 : 065.For N =160 ; 320 ; 640 ;:::; 20480,we approximate r f usingAlgorithm6with K =90,i.e.,weusethe90nearest neighborstoapproximatethegradient.Wecomputeeachapproximation1,000times usingdierentsetsof N uniformrandomsamplesandcomputetheexpectedrelative error.InFigureA.2,weshowtheconvergenceoftheexpectedrelativeerrorofthe 142

PAGE 160

approximationofeachcomponentof r f andthe O 1 N convegencerateforreference.Although,foreach N ,theexpectedrelativeerrorfor @ 2 f isordersof magnitudelargerthanthatoftheotherpartials,allconvergeatarateofapproximately O 1 N .Notethat,weexpectthismethod,oranymethod,tohavediculty approximating @ 2 f becausethiscomponentisveryclosetozerowhichcauses relativeerrorstobeamplied. FigureA.2: Theconvergenceoftheexpectedrelativeerrorforeachcomponentof Eq.A.2.Noticeeachconvergesatapproximatelytherate O 1 N RemarkA.1 InChaptersV,VIandVII,weapproximategradientsofQoImapsdenedbylinearfunctionalsofthesolutiontoastationaryconvection-difussionproblem withanuncertainsourceamplitude,andatime-dependentdiusionproblemwithuncertaindiusioncoecients.ThoseapproximationsarecomputedusingAlgorithm6 with K =10 InChapterVIII,weapproximategradientsofQoImapsdenedbylinearfunctionalsofthesolutiontoatime-dependentdiusionproblemwithuncertaindiusion 143

PAGE 161

coecients.ThoseapproximationsarecomputedusingAlgorithm6with K =20 144

PAGE 162

Glossary BET Butler,Estep,Tavener.27 ED ExperimentalDesign.2 EIE ExpectedInationEect.106 EIG ExpectedInformationGain.7 ESE ExpectedScalingEect.62 ESK ExpectedSkewness.60 GD GeometricallyDistinct.12 i.i.d. independentidenticallydistributed.28 KDE KernelDensityEstimator.21 KL Kullback-Leibler.7 MCMC MarkovchainMonteCarlo.7 OED OptimalExperimentalDesign.3,7 QoI QuantityofInterest.1 SB-OED Simulation-BasedOptimalExperimentalDesign.3,7 SIP StochasticInverseProblem.1 145