Citation |

- Permanent Link:
- http://digital.auraria.edu/AA00006563/00001
## Material Information- Title:
- Simulation-based optimal experimental design theories, algorithms, and practical considerations
- Creator:
- Walsh, Scott N. (
*author*) - Place of Publication:
- Denver, Colo.
- Publisher:
- University of Colorado Denver
- Publication Date:
- 2017
- Language:
- English
- Physical Description:
- 1 electronic file (162 pages) : ;
## Thesis/Dissertation Information- Degree:
- Doctorate ( Doctor of philosophy)
- Degree Grantor:
- University of Colorado Denver
- Degree Divisions:
- Department of Mathematical and Statistical Sciences, CU Denver
- Degree Disciplines:
- Applied mathematics
## Subjects- Subjects / Keywords:
- Uncertainty -- Mathematical models ( lcsh )
Probabilities ( lcsh ) Inverse problems (Differential equations) ( lcsh ) - Genre:
- bibliography ( marcgt )
theses ( marcgt ) non-fiction ( marcgt )
## Notes- Review:
- Several new approaches for simulation-based optimal experimental design are developed based on a measure-theoretic formulation for a stochastic inverse problem (SIP). Here, an experimental design (ED) refers to an experiment defining a particular map from the space of model inputs to the space of observable model outputs. The term optimal experimental design (OED) then refers to the choice of a map from model inputs to observable model outputs with optimal properties. We develop a consistent Bayesian approach to OED, which follows the classical Bayesian approach to OED, and determines the OED based on properties of approximate solutions to a set of representative SIPs. ( , )
- Review:
- Subsequently, we develop an OED approach based entirely on quantifiable geometric properties of the maps defining the EDs. We prove efficient computable approximations to these quantities based on singular value decompositions of sampled Jacobian matrices of the proposed maps. We examine the similarities and differences among these two approaches on a set of challenging OED scenarios. A new description of the computational complexity is introduced and, under certain assumptions, shown to provide a bound on the set approximation error of an inverse image. Furthermore, a greedy algorithm is described for implementation of these OED approaches in computationally intensive modeling scenarios. Several numerical examples illustrate the various concepts throughout this thesis.
- Bibliography:
- Includes bibliographic resource.
- System Details:
- System requirements: Adobe Reader.
- Statement of Responsibility:
- by Scott N., Walsh.
## Record Information- Source Institution:
- University of Colorado Denver
- Holding Location:
- Auraria Library
- Rights Management:
- All applicable rights reserved by the source institution and holding location.
- Resource Identifier:
- on10204 ( NOTIS )
1020495212 ( OCLC ) on1020495212 - Classification:
- LD1193.L622d W35 ( lcc )
## Auraria Membership |

Downloads |

## This item has the following downloads: |

Full Text |

SIMULATION-BASED OPTIMAL EXPERIMENTAL DESIGN: THEORIES
ALGORITHMS, AND PRACTICAL CONSIDERATIONS by SCOTT N. WALSH B.S., Mathematics, Cleveland State University, 2009 B.A., Physics, Cleveland State University, 2009 M.S., Applied Mathematics, University of Colorado Denver, 2016 A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Philosophy Applied Mathematics 2017 This thesis for the Doctor of Philosophy degree by Scott N. Walsh has been approved for the Applied Mathematics Program by Stephen Billups, Chair Troy Butler, Advisor Julien Langou Varis Carey Yail Jimmy Kim July 29, 2017 n Walsh, Scott N. (Ph.D., Applied Mathematics) Simulation-Based Optimal Experimental Design: Theories, Algorithms, and Practical Considerations Thesis directed by Assistant Professor Troy D. Butler ABSTRACT Several new approaches for simulation-based optimal experimental design are developed based on a measure-theoretic formulation for a stochastic inverse problem (SIP). Here, an experimental design (ED) refers to an experiment defining a particular map from the space of model inputs to the space of observable model outputs. The term optimal experimental design (OED) then refers to the choice of a map from model inputs to observable model outputs with optimal properties. We develop a consistent Bayesian approach to OED, which follows the classical Bayesian approach to OED, and determines the OED based on properties of approximate solutions to a set of representative SIPs. Subsequently, we develop an OED approach based entirely on quantifiable geometric properties of the maps defining the EDs. We prove efficient computable approximations to these quantities based on singular value decompositions of sampled Jacobian matrices of the proposed maps. We examine the similarities and differences among these two approaches on a set of challenging OED scenarios. A new description of the computational complexity is introduced and, under certain assumptions, shown to provide a bound on the set approximation error of an inverse image. Furthermore, a greedy algorithm is described for implementation of these OED approaches in computationally intensive modeling scenarios. Several numerical examples illustrate the various concepts throughout this thesis. The form and content of this abstract are approved. I recommend its publication. Approved: Troy D. Butler iii ACKNOWLEDGMENTS First and foremost, I would like to thank my advisor Troy Butler for his dedication and support throughout my graduate career. I am incredibly fortunate to have crossed paths with Troy during his first years as a faculty member at CU Denver. During a time when the future of my graduate career was uncertain, Troy introduced me to a fascinating held of research which encouraged me to continue to pursue this degree. He provided me with opportunity after opportunity to contribute to a fascinating held of research while also building collaborations with researchers at various institutions and laboratories. His willingness to mentor me and provide me with the skills and knowledge required to contribute to this held of research is the reason this thesis is complete today. I would also like to thank my committee members: Stephen Billups, Julien Langou, Varis Carey, and Yail Jimmy Kim. I have benehted greatly from their support and advice in the making of this thesis. I would like to thank Lindley Graham, Steve Mattis and Clint Dawson for their guidance and support during my summer spent at the Institute for Computational and Engineering Sciences at UT Austin. The discussions we had, and perspectives that they provided, were invaluable to my success. I would like to thank Tim Wildey and John Jakeman for their thoughtful discussions during my summer spent at San-dia National Laboratories. Their open minds and willingness to entertain my ideas enabled me to grow as a researcher, both during that summer and throughout my graduate career. There are many friends and fellow graduate students who helped me through graduate school in a variety ways. I would like to thank Abulitibu Tuguluke for spending his Saturday mornings studying for the analysis prelim with me. I would like to thank Devon Sigler for his insightful questions and discussions on the patio of The Market. Without a doubt, these conversations contributed greatly to the quality IV and clarity of parts of this thesis. I cannot list all of my friends and fellow graduate students who have helped me through this process, but thank you Emileigh Willems, Megan Sorenson, Michael Pilosov, Jordan Hall, Lucas Ortiz, Takao Miller, and Trever Hallock. Last, but not least, I would like to thank my parents. My Mom, for her encouragement and support during the difficult times of graduate school. My Dad, for his time spent both helping me to overcome the challenges of the prelims and discussing my obscure analysis questions. His ability to explain challenging concepts in various ways provided me with the deep understanding required to complete the prelims and ultimately contribute to the held of applied mathematics. v TABLE OF CONTENTS CHAPTER I. INTRODUCTION................................................ 1 1.1 Stochastic Inverse Problems............................... 1 1.2 Optimal Experimental Design............................... 2 1.3 Overview.................................................. 3 IP LITERATURE REVIEW........................................... 6 II. 1 Bayesian Simulation-Based Optimal Experimental Design .... 6 11.2 Thesis Objectives......................................... 8 III. A MEASURE-THEORETIC FRAMEWORK FOR FORMULATING STOCHASTIC INVERSE PROBLEMS ........................................... 10 III. 1 Formulating the Stochastic Inverse Problem Using Measure Theory 11 111.2 A Consistent Bayesian Solution to the SIP................ 18 111.3 A Measure-Theoretic Solution to the SIP.................. 22 111.4 Comparison of Methods ................................... 24 111.5 Numerical Example: Consistency........................... 27 IV. CONSISTENT BAYESIAN OPTIMAL EXPERIMENTAL DESIGN . . 34 IV. 1 The Information Content of an Experimental Design........ 34 IVY Infeasible Data ......................................... 42 IV. 3 Numerical Examples ...................................... 47 V. MEASURE-THEORETIC OPTIMAL EXPERIMENTAL DESIGN ... 53 V. l Notation................................................. 53 V.2 d-dimensional Parallelepipeds ........................... 54 V.3 Skewness and Accuracy.................................... 57 V.4 Scaling and Precision.................................... 61 V.5 A Multi-Objective Optimization Problem................... 63 V.6 Numerical Examples ...................................... 65 vi V. 7 Summary .............................................. 70 VI. EXPECTED SCALING EFFECT VS EXPECTED INFORMATION GAIN 71 VI. 1 Technical Lemmas...................................... 72 VI. 2 Computational Considerations.......................... 74 VI. 3 Highly Sensitive Qol.................................. 76 VI.4 Unknown Profile of the Observed Density............... 79 VI.5 Redundant Data ....................................... 95 VI. 6 Summary .............................................. 97 VII. THE COMPUTATIONAL COMPLEXITY OF THE SIP................... 99 VII. 1 A Reformulation of the SIP for Epistemic Uncertainty. 100 VII.2 Single-Objective Optimization Problem................ 105 VII.3 A Bound on the Error of Set Approximations .......... Ill VII. 4 Summary ............................................. 115 VIII. A GREEDY IMPLEMENTATION FOR SB-OED........................... 119 VIII. 1 A Greedy Algorithm................................... 119 VIII. 2 Numerical Example.................................... 120 IX. CONCLUSIONS.................................................. 129 IX. 1 Summary ............................................. 129 IX. 2 Future Work and Open Problems........................ 130 REFERENCES......................................................... 133 APPENDIX A. RADIAL BASIS FUNCTIONS FOR DERIVIATIVE APPROXIMATIONS 139 A.l Using RBFs to Interpolate Unstructured Data.......... 139 A.2 Convergence of the Expected Relative Error........... 141 vii FIGURES III. 1 (left) The inverse of Q-1(Q(A)) is often set-valued even when A G A specifies a particular Q(A) G V. (middle) The representation of C, as a transverse parameterization, (right) A probability measure described as a density on (V,Bv) maps uniquely to a probability density on (Â£,Bc). Figures adopted from [13] and [10].......................................... 111.2 Illustration of the mapping Q : A > V. Note that Q maps B to a particular region Q(B) G V, and while the inverse image of this set, given by Q~1(Q(B)), contains B, other points in A may also map to Q(B). (Figure adopted from [16])............................................ 111.3 We summarize the steps in Algorithm 3. (top row) We depict Steps 1- 3. We partition A with {Vi}^=l that are implicitly defined by the set of samples {A^}^ we make the MC assumption to determine the volume of each Vj. Then we determine the nominal value qW = Q(A^). Here, N = 2000. (bottom row) We depict Steps 4-5. We partition V with a regular grid and prescribe the observed probablity measure to be constant over the darkened set of V%. Here M = 100............................. 111.4 Summary of Algorithm 3 continued, (top row) We depict Steps 5-6. For each k, we determine which set of A^ map into Dk In red, we see the union of the Vi corresponding to the \ that map into D\ seen in red in V. We see the same for D2 in blue, (bottom row) We depict Step 8. We compute the approximate solution to the SIP where we proportion out the probability of each Dk (determined by the observed measure) to its corresponding Vi. Darker colors correspond to higher probability...... 111.5 (top left) Approximation of p^ obtained using p^s on Q-1-, (top right) a set of samples from p^, (bottom) and a comparison of the observed density p^s on with the pushforward of p^............................. 111.6 (top left) Approximation of p^ obtained using p^s on Q(top right) a set of samples from p^\ (bottom) and a comparison of the observed density p^s on Q^ with the pushforward of p^..................... 30 111.7 (left) The approximation of p^ using the joint density p^s on Q^, (right) a set of samples from p^........................................... 31 /ox 111.8 (top left) The pushforward of p\_ (top right) the exact observed density, (bottom) the difference between the pushforwad of p\ and the observed density scaled by the maximum of the observed density............ 32 IV. 1 (left) Approximation of p^ obtained using p^s on Q^l\ which gives KLq(i)(pvs) ~ 0.466. (right) p^s on and the pushforward of the prior................................................................. 37 IV.2 (left) Approximation of p^ obtained using Q^ and p^s on Q(2\ which gives KLq(2)(p$s) 2 .015. (right) pps on Q^ and the pushforward of the prior............................................................. 37 IV.3 (top left) The approximation of the pushforward of the prior, (top right) The exact p^s on Q(3d (bottom) The approximation of p^ using p^s on Q(3\ which gives KLq(3)(p^s) & 2.98................................... 39 IV.4 (left) The pushforward of the prior and observed densities on Q(right) The pushforward of the prior and observed densities on (-2\ Notice the support of each observed density is contained within the range of the model, i.e., the observed densities are absolutely continuous with respect to their corresponding pushforward densities.......................... 44 IV.5 (top left) The pushforward of the prior for the map Q^\ (top right) The observed density using the joint information which now extends beyond the range of the map. (bottom) The normalized observed density that does not extend beyond the range of the map.............................. 46 IX IV.6 Smoothed plot of the EIG over the indexing set Q for the design space Q. Notice the higher values in the center of the domain and towards the top right (in the direction of the convection vector from the location of the source), this is consistent with our intuition....................... 49 IV.7 (left) Smoothed plot of the EIG over the indexing set Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q. This location corresponds to placing thermometers near separate ends of the rod. However, we observe multiple local maxima within the design space, denoted Rl, R2, R3 and R4. (right) The EIG for the local OED in each of the four regions.................................................... 52 V.l The ESE describes how uncertainties of a fixed size on the data space are reduced under the inverse of the Qol map............................... 63 V.2 The ESE describes how much uncertainty on the data space is reduced to the size of the unit cube.............................................. 64 V.3 Smoothed plot of the ESE-1 over the indexing set Q for the design space Q. Notice the higher values in the center of the domain and towards the top right (in the direction of the convection vector from the location of the source), this is consistent with both intuition and the results from Example IV.3..................................................... 67 V.4 (left) Smoothed plot of the ESE-1 over the indexing set Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q. This location corresponds to placing thermometers near each end of the rod. However, we observe four prominant local maxima, denoted Rl, R2, R3 and R4. (right) The ESE-1 for the local OED in each of the four regions.......................................................... 68 V.5 (left) Smoothed plot of the ESK-1 over the indexing set Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q. This location corresponds to placing thermometers near each end of the rod. However, we observe four prominant local maxima, denoted Rl, R2, R3 and R4. (right) The ESK-1 for the local OED in each of the four regions. 69 V. 6 (left) Smoothed plot of (dy^ip, y))_1, with oj = 0.5, over the indexing set Q x Q for the design space Q. Notice the high values in the bottom right corner of Q x Q. This location corresponds to placing thermometers near each end of the rod. Here, we observe only one prominant local maxima, (right) (dy^ij), y))-1, with oj = 0.5, for the local OED in the single region of interest......................................................... 69 VI. 1 The two groups of EDs shown in the indexing set Q x Q. The group in the bottom right (over a red region) is the optimal group of EDs and the group near the upper right (over a blue region) is the sub-optimal group. For reference, we show these two clusters plotted on top of the ESE-1 (computed using all 5,000 available samples in A as described above). . 77 VI.2 (left) The ESE-1 for the 20 EDs under consideration. Notice the clear difference in ESE-1 values between the optimal group (red) and the sub-optimal group (blue). The optimal group looks to be just a single line, however, there are actually 10 lines plotted here and they simply he so close together that they are indistinguishable, (right) The EIG for the 20 EDs under consideration. In this image, we do not see a clear distinction between the two groups of EDs. Again, there are 10 red lines and 10 blues lines here, they are just not distinguishable. See Table VI. 1 for a quantitative description of these EIG values.............................. 79 VI.3 The two groups of EDs shown in the indexing set Q x Q. The group in the bottom right (over a red region) is the optimal group of EDs and the other group (over a blue region) is the sub-optimal group. For reference, we show these two clusters plotted on top of the ESE-1 (computed using N = 100 available samples in A as described above)................... VI.4 (left) We show the ESE-1 as a function of a to emphsize that the ESE is defined independent of the expected profile of the observed density on each Qol. (right) The EIG for the 100 EDs under consideration as a function of a. The optimal group is shown in red and the sub-optimal group is shown in blue. Note that, as a increase, the EIG for the optimal cluster becomes smaller than the EIG for the sub-optimal group............... VI.5 The EIG over the indexing set Q x Q and the top 100 OEDs for increasing a. For very small a = IE 05, we see that the top 100 OEDs appear to be random points in Q x Q. For 0.1 < a < 1.0, the top 100 OEDs are all clustered in the bottom right of Q x Q. Then, as sigma increase beyond 1.0, the top 100 OEDs move away from the bottom right of Q x Q and settle in two regions near the center of the space................... VI.6 (top left) Approximation of p^ using p^s on with a =lE-07, (top right) a comparison of the observed density p^s on with the pushfor-ward of the prior on (top left) Approximation of p^ using p^s on Q^ with a =lE-07, (top right) a comparison of the observed density p^s on Qwith the pushforward of the prior on Q^.......................... VI.7 (top left) Approximation of p^ using p^s on with a =lE-03, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on Q^ with a =lE-03, (top right) a comparison of the observed density p^s on Qwith the pushforward of the prior on Q^.......................... VI.8 (top left) Approximation of p^ using p^s on with a =lE-02, (top right) a comparison of the observed density p^s on with the pushfor-ward of the prior on (top left) Approximation of p^ using p^s on with a =lE-02, (top right) a comparison of the observed density p^s on Qwith the pushforward of the prior on Q^...................... 89 VI.9 (top left) Approximation of p^ using p^s on with a =2E-02, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on Q^ with a =2E-02, (top right) a comparison of the observed density p^s on Qwith the pushforward of the prior on Q^...................... 90 VI.10(top left) Approximation of p^ using ppbs on with a =3E-02, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on Q^ with a =3E-02, (top right) a comparison of the observed density p^s on Qwith the pushforward of the prior on Q^...................... 91 VI. 11 (top left) Approximation of p^ using p^s on with a =5E-02, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on Q^ with a =5E-02, (top right) a comparison of the observed density p^s on Qwith the pushforward of the prior on Q^...................... 92 VI.12(top left) Approximation of p^ using p^s on with a =1E-01, (top right) a comparison of the observed density p^s on with the pushforward of the prior on (top left) Approximation of p^ using p^s on Q^ with a =1E-01, (top right) a comparison of the observed density p^s on Q^ with the pushforward of the prior on Q^.................... 93 VI. 13(top left) Approximation of p^ using p^s on with a =lE+00, (top right) a comparison of the observed density p^s on with the push-forward of the prior on (top left) Approximation of p^ using p^s on with a =lE+00, (top right) a comparison of the observed density Pps on Qwith the pushforward of the prior on Q^.................... VI.14Smoothed plots of the ESE-1 and the EIG over the indexing set Q x Q for the design space Q.................................................. VI. 15Smoothed plots of the ESE-1 and the EIG over the indexing set Q for the design space Q..................................................... VII. 1 (left) In green, we show the approximate inverse image with the given Voronoi tesselation of A. The exact inverse image is shown in blue. Notice the true parameter A*e in red is contained within the exact inverse image, however, it is not contained within the approximation, (right) In blue, we show the event E e that defines Q~1(E), the true datum in red, and the observed datum in black that is responsible for the definition of E e VII.2 In purple we show Q~1(E), in yellow we show the inflated inverse image resulting from extending the boundaries outward, and in green we show the approximation to the inflated inverse image that covers Q~1(E). Note that, this inflation is determined by extending the boundaries outward each direction by a distance of Rn, the maximum radius of the Voronoi cells in the tesselation........................................... VII.3 (top left) The inverse image Q~1(E) and the approximation, (top right) The E e Bv defining the uncertainites on the observed data, (bottom right) The inflated E E determined by the Jacobian of Q and R^. (bottom left) The exact inverse image Q~1(E), the inflated inverse image Q~1(E), and the resulting approximation to Q~l(E) that covers Q~1(E). 94 95 96 101 102 104 xiv VII.4 (left) A representative inverse image resulting from inverting E G Â£>p using the map Q^a\ (right) A representative inverse image resulting from inverting E G using the map Q^. Although produces a smaller inverse image using the same sized E, the correlation amongst the two Qol composing produces an inverse image that is more difficult to approximate accurately with finite sampling.......................... 109 VII.5The inverse of the inflation effect, for both and Q^b\ as a function of the number of samples available. Although Mq^) < Mq(a), for N < 400, the inflation effect determines that is the OED. This is due to the increased difficulty in approximating the solution to a SIP defined using Q(b\ For N > 400, the choice changes and is the OED.................. 110 VII.6The EIE-1 over the indexing set Q x Q for the design space Q for various numbers of samples N. In the top left, the EIE-1 looks very similar to the ESK-1 shown in Figure V.5. As we increase the number of samples, the location of the OED does not change, however, the general description of Q changes noticably. We begin to see similar features to that of the ESE-1 seen in Figure V.4............................................. 112 VII.7The MC approximation to Error(Q~l(E)) and the outer and inner set approximation using a regular grid of samples for N = 192, 400, 784,1600 (from top to bottom)................................................. 117 VII.8The MC approximation to Error(Q~1(E)) and the outer and inner set approximation using uniform random samples for N = 192, 400, 784,1600 (from top to bottom)................................................ 118 VIII.IThe domain 0 is constructed by welding together nine square plates, each of which has constant, but uncertain, thermal conductivity k........ 121 VIIL2The ESE-1 over the indexing set for the design space Q. Notice the OED is shown in white, near the center of the domain where the source is located. 123 xv VIII.^top) The ESE 1 for Q(2\ Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1. (bottom) The ESE-1 for Q^3\ Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1............................................................ 125 VIII.^top) The ESE-1 for Q(4d Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1. (bottom) The ESE-1 for Q(-5\ Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1............................................................ 126 VIII.Ii(top) The ESE-1 for Q^. Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1. (bottom) The ESE-1 for Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1............................................................ 127 VIII.Q> top) The ESE-1 for Q(8\ Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1. (bottom) The ESE-1 for Notice that the previously placed sensors are shown in black, and the optimal location for the next sensor is shown in white at the maximum of the ESE-1........................................................ 128 A.l The convergence of the expected relative error for each component of Eq. (A.l). Notice each component of V/ converges at approximately the rate O(j^)............................................................ 142 xvi A.2 The convergence of the expected relative error for each component of Eq. (A.2). Notice each converges at approximately the rate 0(A).......... 143 xvii CHAPTER I INTRODUCTION 1.1 Stochastic Inverse Problems Scientists and engineers strive to accurately model physical systems with the intent to predict future and/or unobservable quantities of interest (Qol). For example, a scientist modeling the transport of a contaminant source may be interested in predicting the average concentration of the contaminant in a region of the domain at some future time. That same scientist may also be interested in the flux of the contaminant through some boundary that is physically unobservable, i.e., cannot be determined by direct measurements. Both of these scenarios require the construction of a computational model that simulates the physical process of interest. In many applications, the extraordinary complexity of the model induces a set of model inputs that are uncertain. Typically, model inputs are described as belonging to a general set of physically plausible values using either engineering or domain-specific knowledge. Using the computational model, this set of plausible model inputs can be propagated through the model to make predictions on unobservable Qol. When the Qol are sensitive to perturbations in model inputs, the predictive capabilities of even the most sophisticated computational models may be severely limited by a coarse description of uncertainties in the model inputs. Oftentimes, researchers look to reduce these uncertainties by using data gathered from the actual physical process simulated by the model. The idea being, by fitting the model to observable data, the probability that the model accurately predicts unobservable data is increased. This process is commonly referred to as model calibration. However, fitting or calibrating the model to observed data can be a difficult task. It is common for the observed data to be subject to uncertainty, e.g., measurement instrument error reduces the confidence one has in the exact values provided by the data. Such uncertainties are often described using probability models, 1 and specification of a probability measure on the range of values for a Qol is one approach to represent uncertainties in observable data. Then, rather than calibrating the model to a fixed set of observed data, we formulate and solve a stochastic inverse problem (SIP) in order to determine a probability measure on the space of model inputs informed by the probability measure on the Qol. There are many frameworks for formulating SIPs based on different assumptions and models for the uncertainties in model inputs, observed data, and even the Qol maps that links these. A popular framework is based on the famous Bayes rule [62] where the Qol map is effectively replaced by a statistical likelihood function. More recently, a framework based on measure-theoretic principles has been developed and analyzed for the formulation and solution of a SIP that does not replace the Qol map for the model [16, 10, 12, 13]. The solution of a SIP formulated in the measure-theoretic framework is defined as a probability measure on the space of model inputs that is consistent with the model and data in the sense that the pushfoward of this probability measure through the Qol map exactly matches the probability measure given on the Qol. In this thesis, we adopt this measure-theoretic framework for formulating a SIP under two scenarios and consider two separate methods for the numerical solution of the SIP. 1.2 Optimal Experimental Design The collection of data for an observable Qol can be costly and time consuming. For example, exploratory drilling can reveal valuable information about subsurface hydrocarbon reservoirs, but each well can cost upwards of tens of millions of US dollars. In such situations, we can only afford to gather data for some limited number of observable Qol. However, we may be able to define certain aspects of an experiment that define the types of observable Qol for which we collect data. For example, we may be able to choose the locations of a set of exploratory wells from some predetermined feasible domain. The choice of observable Qol for which we may collect data is 2 defined as an experimental design (ED). Thus, an ED defines a particular Qol map from model inputs to model outputs. Since the Qol map plays a significant role in the solution of a SIP, it is evident that two different EDs may lead to significantly different solutions to a SIP. For example, consider a simple contaminant transport problem in which the diffusivity properties of the domain are uncertain. We expect that some measurements in space-time are more useful than others at reducing uncertainties in the diffusion tensor. To reduce the expense of collecting data on less useful (i.e., less informative) Qol, we consider a simulation-based approach to automatically select the useful Qol from a set of possible EDs. Simulation-based optimal experimental design (SB-OED, or sometimes just OED) looks to determine the optimal ED by way of computational simulations of the physical process. The main idea being to exploit the sensitivity information in a Qol map corresponding to an ED to determine the optimal Qol to gather data in the held. The criteria for optimality of an ED can be defined in various ways depending on the goals of the researchers. Here, we consider batch design over a discrete set of possible EDs. Batch design, also known as open-loop design, involves selecting a set of experiments concurrently such that the outcome of any experiment does not effect the selection of the other experiments. Such an approach is often necessary when one cannot wait for the results of one experiment before starting another, but is limited in terms of the number of observations we can consider. The pursuit of a computationally efficient approach for coupling the SB-OED methods developed within this thesis with continuous optimization techniques is an intriguing topic that we leave for future work. 1.3 Overview The rest of this thesis is outlined as follows. In Chapter II, we provide a review of past and recent work in SB-OED. For completeness, in Chapter III, we review the measure-theoretic framework for formulating a SIP and two methods for numerical 3 solution of the SIP. The two methods are referred to as a consistent Bayesian approach and a measure-theoretic approach. The consistent Bayesian approach adopts the use of a prior probability density in the solution to the SIP whereas the measure-theoretic approach does not require any prior knowledge of the probability on the model inputs to solve the SIP. We provide relevant algorithms and figures depicting the implementation of each of these approaches. We follow a Bayesian methodology for SB-OED to develop a consistent Bayesian approach to SB-OED in Chapter IV. The consistent Bayesian approach to SIPs proves useful in reducing the cost of the iterated integral defining the utility of a given ED. In Chapter V, we develop an entirely unique approach to SB-OED inspired by the set-based approximations of the measure-theoretic approach. This approach deviates from the Bayesian methodology for SB-OED in that it quantifies the utility of a given ED entirely in terms of the local geometric properties of the corresponding Qol map and does not require solution to any SIP. Thus, this approach is independent of both the methodology employed to solve the chosen SIP and the specification of a probability measure on the space of observable Qol. Consequently, under certain assumptions, these two SB-OED approaches produce similar qualitative descriptions of a space of EDs. Thus, in Chapter VI, we provide an in depth comparison between these two methods focused specifically on a handful of challenging SB-OED scenarios. Following Chapter VI, we focus entirely on the measure-theoretic approach to SB-OED and look to resolve and improve upon the current state of the method. In Chapter VII, we focus on three major topics. First, Section VII. 1 proposes a reformulation of the SIP so that, under certain assumptions, we guarantee the true parameter1 is contained within a region of nonzero probability in the solution to the SIP. Although this approach is developed to improve reliability in a solution to a given SIP, it provides useful insight into SB-OED interests. In Section VII.2, we 1By true parameter, we are referring to an application in which the SIP fundamentally is defining a problem of parameter identification under epistemic uncertainties. 4 scalarize the multi-objective optimization problem, defined in Chapter V, implicitly as a function of the sensitivities of the Qol map, the dimension of the model input space, and the number of model solves available to ultimately approximate the solution to a chosen SIP. In Section VII. 3, we provide a bound on the error in the approximation to an inverse image which offers an alternative description of the computational complexity, i.e., condition number, of a given SIP. In Chapter VIII, we illustrate the measure-theoretic approach to SB-OED on a toy problem that yields high-dimensional parameter, data, and design spaces. We employ a greedy algorithm to abate this curse of dimensionality, thus, we approximate the OED from a combinatorial enumeration of a discretization of the design space. In Chapter IX, we provide concluding remarks and discuss future research opportunities and directions. 5 CHAPTER II LITERATURE REVIEW It is worth noting that the term optimal experimental design can mean a great many things depending on the context. The term experimental design (sometimes referred to as design of experiments), was likely first popularized within the statistical community, e.g., see [42, 43, 29, 41, 5] and the references therein for earlier references on the topic. In this context, an experimental design refers to some placement of samples within the predictor space. An optimal experimental design refers to an optimal placement of samples within the predictor space, where the optimality criterion may be application specific. The so-called alphabetic optimality criteria [5, 40] look to optimize properties of parameter estimates defining a model, e.g., D-optimality to maximize the differential Shannon entropy, A-optimality to minimize the average variance of parameter estimates, and G-optimality to minimize the maximum variance of model predictions. These criteria have been developed in both Bayesian and non-Bayesian settings [5, 40, 6, 25, 18, 19]. In summary, this held of experimental design research focuses on optimizing the quality of a statistical model relating the predictors to the responses. II. 1 Bayesian Simulation-Based Optimal Experimental Design Although developing appropriate statistical models is of utmost importance in many predictive modeling applications, we emphasize that this differs substantially from the approaches proposed in this thesis. The approaches proposed below are developed specifically for physics-based computationally expensive and potentially nonlinear models with uncertain model input parameters. Given a parameter (predictor), the computational model provides the value of the Qol (responses). Hence, we need not approximate an optimal model. However, uncertainties in model input parameters lead to uncertainties in model predictions. If uncertainties on the parameters can be quantified and reduced using 6 experimental data, then the uncertainties in model predictions can also be quantified and reduced. We accomplish this with the formulation and solution of a SIP. Given a probability distribution on some collection of observable Qol, the solution of the SIP is a probability distribution on the space of model input parameters. The probability distribution on the observable Qol may be determined by combining available information on the experimental data and observation instrument error. Different collections of Qol may provide information about different directions within the parameter space. Hence, in this context, an optimal experimental design (OED) refers to an optimal collection of Qol, where optimality criterion are often derived from an information theoretic foundation [46, 65, 8]. The directional information contained in a particular collection of Qol can be analyzed by numerical simulations of the model. Thus, we may be able to rank different EDs by some performance metric based entirely on simulations. We follow recent literature [35, 36, 54] and refer to this type of experimental design as simulation-based optimal experimental design (SB-OED) to emphasize the necessity of a computational model relating the uncertain parameters to the Qol. Given a probability distribution on the chosen space of observable Qol, the Bayesian paradigm is often used to formulate and solve a SIP to determine a proba-blity distribution on the space of model input parameters [59, 62], This perspective has lead to a Bayesian approach to SB-OED, where approximate Bayesian solutions to the SIP determine the expected utility of a given ED [20, 8, 63]. The expected utility of an ED is often written as an iterated integral over both the parameter space and the space of Qol defining the ED. Hence, it is computationally expensive to approximate many useful forms of the expected utility, e.g., the commonly used expected information gain (EIG) [8] derived from the Kullback-Leibler (KL) divergence [44, 64], Therefore, much of the literature in this held is directed towards reducing computational complexity in approximating the expected utility of an ED. 7 Locally optimal techniques reduce cost but require a best guess of the value of the unknown parameter [30]. This approach is not suitable when the posterior parameter distribution is broad. In [57], Markov chain Monte Carlo (MCMC) methods and Laplacian approximations are used to compute the EIG. In [9, 21], the authors reduce computational cost by approximating the posteriors with Gaussian distributions. When the posterior differs significantly from a Gaussian profile, such an approach is not applicable. In [48, 49, 50], the authors use Laplace approximations to accelerate the computations of the EIG. In [35], the authors utilize a generalized polynomial chaos surrogate to accelerate computations of the EIG and stochastic optimization techniques to determine an OED in a high-dimensional setting. In [55, 4], the authors use MCMC and simulated annealing methods to accelerate the computation of the expected utility. In [1, 2], the authors develop an OED framework for inverse problems with infinite dimensional parameter spaces resulting from an unknown held in the physics-based model. All of the methods described here have proved valuable in the appropriate context. However, the general formulation of Bayesian SB-OED requires the solutions, or approximate solutions, of SIPs to determine an OED. Thus, these approaches require first formulating a specific type of SIP within a particular framework, knowledge of the method to be used to solve the resulting SIP, and, finally, an estimate of the uncertainties on the observed Qol. II.2 Thesis Objectives The objective of the approaches described above is to approximate the expected utility of an ED based on approximate solutions to the classical Bayesian SIP [59, 62], Although this approach has proved immensely useful in the context of classical Bayesian inference, in this thesis, we focus specifically on developing SB-OED approaches inspired by consistent solutions to the SIP [16, 10, 12, 13]. The consistent Bayesian SB-OED approach, developed in Chapter IV, contributes to the 8 line of research regarding reduction in computational cost by employing the recently developed consistent Bayesian formulation and solution to the SIP [16] to efficiently integrate over the space of observed densities to approximate the EIG. Future work will explore the coupling of this method with continuous optimization techniques to determine OEDs in high-dimensional design spaces. The measure-theoretic approach to SB-OED, developed in Chapter V, derives a computationally tractable formulation for SB-OED in which not a single SIP is solved and, therefore, is independent of both the framework used to formulate the SIP, the method used to solve the SIP, and also the estimated uncertainties on the observed Qol. An additional benefit is that the measure-theoretic method provides a quantification of the computational complexity involved in solving the final SIP using sample-based approaches. In Chapter VI, we present a thorough comparison between these two SB-OED approaches and illustrate the strengths and weaknesses of each. Chapter VII continues to extend the measure-theoretic approach to SB-OED by both scalarizing a previously defined multi-objective optimization problem and contributing further to the quantification of the computational complexity of a SIP. In Chapter VIII, we illustrate a greedy implementation of the measure-theoretic approach to SB-OED in a high-dimensional setting. 9 CHAPTER III A MEASURE-THEORETIC FRAMEWORK FOR FORMULATING STOCHASTIC INVERSE PROBLEMS In this chapter, we review two recently developed methods for solving a SIP formulated within a measure-theoretic framework. In Section III. 1, we review the measure-theoretic formulation of a SIP solved by both methods. The main result is a form of the disintegration theorem that states the conditions under which a probability measure solves the SIP. Conceptually, this involves decomposing the probability measure into a marginal and family of conditional probability measures. The two methods we present for solving the SIP go about defining these conditional probabilities in fundamentally different ways that lead to different approximation procedures for the solution to the SIP. In Section III. 2, we review the consistent Bayesian method which follows the Bayesian philosophy to define a unique solution to the SIP. The solution is defined in terms of a probability density function as is common in Bayesian approaches. In Section III.3, we present a measure-theoretic method which exploits the geometry of the Qol map to define a unique solution to the SIP. There, the solution is defined as the actual probability measure on the prescribed u-algebra. Whereas the Bayesian approach leads to algorithms for which we can easily draw samples from the distribution using simple schemes (e.g. accept/reject), the measure-theoretic approach leads to an algorithm that is based on set (i.e. event) approximation resulting in a direct approximation of probabilities of these sets. In Section III.4, we compare these two methods. In particular, we focus on the family of conditional probabilities each method employs to define a unique solution to the SIP and the subsequent numerical methods utilized to approximate solutions resulting from each method. In Section III.5, we illustrate both the consistency property of the solution to a SIP formulated within a measure-theoretic framework and the dependence of 10 this solution on the choice of observable Qol that compose the Qol map. III. 1 Formulating the Stochastic Inverse Problem Using Measure Theory Here, we summarize the work of [10, 12, 13]. In order to provide a complete description of the measure-theoretic framework to the SIP, we first introduce some necessary notation and then carefully discuss a taxonomy of three forward problems, with increasing levels of uncertainty, along with their direct inverse problems. For a more thorough description and analysis of these problems and solutions, see [10, 13]. Let M(Y, A) denote a deterministic model with solution T(A) that is an implicit function of model parameters Ag Ac R. The set A represents the largest physically meaningful domain of parameter values, and, for simplicity, we assume that A is compact. In practice, modelers are often only concerned with computing a relatively small set of Qol, {Qi(Y)}f=1, where each Qi is a real-valued functional dependent on the model solution Y. Since Y is a function of parameters A, so are the Qol and we write Qi(A) to make this dependence explicit. Given a set of Qol, we define the Qol map Q(A) := (Qi(A),--- ,Q V C W1 where V := Q(A) denotes the range of the Qol map.Level 1: Deterministic Point-Based Analysis The first forward problem is the simplest in predictive science which is to evaluate the map Q for a fixed A G A to determine the corresponding output datum Q(A) = q G V. In other words, once the inputs of a model are specified, the simplest forward problem is to solve the model in order to predict the output datum. There is no uncertainty, save for model evaluation error, in these point values. The corresponding inverse problem is to determine the parameter(s) A G A that produce a particular value of q G V. Oftentimes Q~1(q) defines a set of values in A either due to non-linearities in the map Q and/or the dimension of A being greater than the dimension of T>. Thus, the simplest inverse problem often has uncertainty as to the particular value of A G A that produced a fixed output datum. However, there 11 is no uncertainty about which set-valued inverse produced the fixed output datum. When n = 2 and d = 1, a set-valued inverse, Q~1(q), defines a contour in A that we are familiar with from contour maps, see the left plot in Figure III. 1. In general, we refer to the set-valued inverses of the map Q as generalized contours. Assuming d < n and that the Jacobian of Q has full row rank everywhere in A, the generalized contours exist as piecewise-defined (n d)-dimensional manifolds in A. The assumption that the Jacobian of Q has full row rank is a practically important one since we prefer to use Qol that are sensitive to the parameters since otherwise it is difficult to infer useful information about the uncertain parameters. Moreover, we prefer Qol to exhibit these sensitivities in unique directions in A as this allows different Qol to inform about different uncertain model input parameters. We formally characterize the property that the Jacobian of Q has full row rank in the following definition. Definition III. 1 The components of the d-dimensional vector-valued map Q(A) are geometrically distinct (GD) if the Jacobian of Q has full row rank at every point in A. When the component maps are GD, we say that the map Q is GD. Henceforth, we assume that any map Q is GD. This implies that d < n, i.e., the number of observable Qol is no larger than the number of uncertain model parameters. Level 2: Deterministic Set-Based Analysis The second type of forward problem can often be written mathematically as analyzing Q(B) C V given a set B C A. In other words, there is generally uncertainty as to the precise value A takes in a set B C A and subsequently there is uncertainty as to the particular datum to predict in the set Q(B). This is a commonly studied, and well understood, topic in the mathematical and engineering sciences. For example, it is common in dynamical systems to study how sets of initial conditions evolve in time. Oftentimes we are interested in describing the size of the sets and how that 12 Figure III. 1: (left) The inverse of Q_1(Q(A)) is often set-valued even when A G A specifies a particular Q(A) G V. (middle) The representation of C, as a transverse parameterization, (right) A probability measure described as a density on (V, Bp) maps uniquely to a probability density on (C,Bc). Figures adopted from [13] and [10]- size changes from model inputs to model outputs. Thus, we assume (A,Â£>a,^/a) and (V, Bp, fiv) are measure spaces where Â£>A and Bp are the Borel u-algebras inherited from the metric topologies on E and Ed, respectively, and the measures fj,\ and /ip are volume measures. The volume measures on A and V allow us to quantitatively assess how model inputs change to model outputs in terms of the the measures of A and Q(A), respectively. The corresponding deterministic inverse sensitivity analysis problem is to analyze Q-1(A) for some A G Bp. While Q-1(A) G Â£>A, the practical computation of Q-1(A) is complicated by the fact that Q-1 does not map individual points in V to individual points in A. As described above, assuming d < n and Q is GD, Q-1 maps a point in V to a generalized contour described as an (n d)-dimensional manifold embedded in A. Thus, Q-1(A) defines a generalized contour event, or just contour event, belonging to an induced contour a-algebra CA C Â£>A (where the inclusion is often proper). To observe that the inclusion is often proper, observe that for any B G Â£>A, BCQ-\Q(B)), (III.l) but in many cases B yt Q 1(Q(B)) even when n = cl (see Figure III.2 where n = 2 13 and cl = 1 for an illustration). In other words, solutions to the corresponding inverse problem can be described in the measurable space (A, Ca) and the volume measure can be used to provide quantitative assessments of solutions to this inverse sensitivity analysis problem. Figure III.2: Illustration of the mapping Q : A > V. Note that Q maps B to a particular region Q(B) E V, and while the inverse image of this set, given by Q~1(Q(B)), contains B, other points in A may also map to Q(B). (Figure adopted from [16]). It is useful to consider solutions to this deterministic inverse problem in a space of equivalence classes. Specifically, we may use the generalized contours to define an equivalence class representation of A where two points are considered equivalent if they he on the same generalized contour. We let Â£ denote the space of such equivalence classes and let ttc : A > Â£ denote the projection map where 7qc(A) = Â£ E Â£ defines the equivalence class corresponding to a particular A and ic~^l{Â£) = Ct is the generalized contour in A corresponding to the point Â£ E Â£. It is possible to explicitly represent Â£ in A by choosing a specific representative element from each equivalence class. As described in [10, 13], such a representation of Â£ can be constructed by piecewise d-dimensional manifolds that index the (n cl)-dimensional generalized contours. We refer to any such indexing manifold as a transverse parameterization (see the middle plot of Figure III. 1). Given a particular indexing manifold representing Â£, Q defines a Injection between Â£ and V. The measure space (Â£,Bc,tl>c) can 14 defined as an induced space using the bijection Q and (P, Pp, pp). Then, solutions to the deterministic inverse problem can be described and analyzed as measurable sets of points in Â£ instead of measurable generalized contour events in A. Level 3: Stochastic Analysis We now assume that there is uncertainty, described in terms of probabilities, as to which set A e PA the model parameters A belong, and the goal is to analyze the probabilities of sets in Pp. In the language of probability theory, measurable sets are referred to as events. Thus, we assume that a probability measure PA is given on (A, PA) describing uncertainty in the events for which parameters may belong, and the goal is to determine Pp on (V, Pp). We refer to this as the stochastic forward problem. When PA (resp., Pp) is absolutely continuous with respect to the volume measure La (resp., pp), the corresponding Radon-Nikodym derivative (i.e., the probability density function) pA (resp., pp) is usually given in place of the probability measure. We assume this is the case so that we can refer to probabilities of events in terms of the more common representation using integrals and density functions, e.g., Solution to this stochastic forward problem is given by the induced pushforward probability measure Pp defined for any A e Pp by This is a familiar problem in uncertainty quantification. The approximate solution can be obtained by a classic Monte Carlo method. Assuming that Q is GD, the pushforward volume measure, by the map Q, is absolutely continuous with respect to the usual Lebesgue measure on Rd [13, 16]. Therefore, unless otherwise noted, we assume pA and pp are the usual Lebesgue measures on E and Ed, respectively. (III.2) (III.3) 15 The corresponding SIP assumes that we are given an observed probability measure, Pgbs, on (V,Bt>), which can be described in terms of an observed probability density, p^s, i.e., p^s is the Radon-Nikodym derivative of Pgbs with respect to pp. The SIP is then defined as determining a probability measure, Pa, described as a probability density, pa, such that the pushforward measure of Pa agrees with Pgbs on We use P^(-Pa-) to denote the pushforward of Pa through Q, i.e., />yp*>(.4) = Pa(C-1(.4)). for all A e Bv- Using this notation, a consistent solution to the SIP is defined formally as follows: Definition III.2 (Consistency) Given a probability measure Pfibs described as a density prfis with respect to pp on (V,Bx>), the SIP seeks a probability measure Pa described as a probability density pa with respect to pa on (A, Â£>A) such that the subsequent pushforward measure induced by the map Q satisfies Pa(Q-'(A)) = fg(PAp4) = Pg(.4), (1114) for any A e Â£>p. We refer to any probability measure Pa that satisfies (III.4) as a consistent solution to the SIP. To determine such a solution to the SIP, we first consider the SIP posed on (Â£, Bfi)-Since Q defines a bijection between Â£ and V, Pgbs defines a unique Pc on (Â£, Be) (see the right-hand plot in Figure III. 1 and [13]). We can then use the projection map 7tc to prove the following theorem as was done in [13]. Theorem III.3 The SIP has a unique solution on (A,C\). However, the goal is to define a probability measure Pa on the measurable space 16 (A,Ba) not on a space involving contour events that are complicated to describe. This requires an application of the Disintegration Theorem, which allows for the rigorous description of conditional probability measures defined on sets of zero /iA-nieasure [24, 13, 32], Theorem III.4 (The Disintegration Theorem) Assume Q : A V is BA- rneasurable, Pa is a probability measure on (A,Ba) and Pfibs is the pushforward measure of Pa on (V,Bx>)- There exists a Pfibs-a.e. uniquely defined family of conditional probability measures {Pq}qeT> on (A, Â£>A) such that for any A E Ba, Pq(A) = Pq(AnQ~1(q)), so Pq(A \ Q~1(q)) = 0, and there exists the following disintegration of Pa, P^(A)= f P,(A)dPg(q) = f l f dP,(X))dPg(q), (III.5) Jv Jv yJAnQ-1^) J for A E Ba Equivalently, because there is a bijection between C, and V, we may write this iterated integral in terms of C, Pa(A)= [ ([ dPe(\))dPc(e), (IH.6) Jc KJArmp1^) J for A E Ba where Pc is the unique probability measure induced on C, by Pfibs. For proofs related to the above disintegration theorem see [13, 16]. It follows from the disintegration theorem that if a probability measure Ppbs on (V, Â£>p) is given, then the specification of a family of conditional probability measures {Pq}qeT> (or, equivalently, {Pe}Â£ec) on (A, Ba) can be used to define a specific probabilty measure Pa using Eq. (III.5) (or, equivalently, Eq. (III.6)) that is consistent in the sense of Definition III.2. However, a consistent solution may not be unique, i.e., there may be 17 multiple families of conditional probability measures on (A, Ha) that define multiple consistent probability measures. A unique solution may be obtained by imposing additional constraints or structure on the SIP that leads to a unique specification of the family of conditional probability measures on (A, Ha). The methods explored in Section III.2 and Section III.3 describe how we may use either assumed or existing measure-theoretic information to define the families of conditional probabilities. III.2 A Consistent Bayesian Solution to the SIP Here, we summarize the work of [16]. III.2.1 Mathematical Formulation Following the Bayesian philosophy [62], we introduce a finite prior probability measure PÂ£nor described as a probability density p^lor on (A, Ha) with respect to //a-The prior probability measure encapsulates any existing knowledge about the uncertain parameters and serves to define the necessary family of conditional probability measures on (A, Ha) to obtain a unique solution to the SIP. The prior information can come from various sources. For example, engineering knowledge of the feasible parameter domain may inform bounds on the support of the prior density and past experimental results may inform an appropriate distribution for the prior over this feasible region. In the absence of a physically meaningful prior measure, or any previous results from which to build one, it may be possible to develop a prior using a coarse solution to the SIP using the measure-theoretic method which does not require a prior probability measure, see Section III.3. The quality of a prior probability measure and suitable methods for development of such a prior is a current topic of scientific research and debate, and this topic is beyond the scope of this exposition of the consistent Bayesian solution to the SIP. Assuming that Q is at least measurable, then the prior probability measure on (A, Ha), HÂ£nor, and the map, Q, induce a pushforward measure pP(pnob Gn V, which 18 is defined for all A e Â£>: v, P^(prior) (A) = PfOI(Q-\A)). (III. 7) Recall from classical probability theory that if P is a probability measure, B is an event of interest, and A is an arbitrary event, then P(B) = P{B\A)P{A) + P(B\AC)P(AC). Furthermore, if B C A, then this reduces to P(B) = P(B\A)P(A). (III.8) In this context, the term P(B) represents the consistent probability measure of the event B that we seek and we let A := Q~1(Q(B)), which ensures that B C A. We are motivated to use P(A) = P-^S(Q(A)) in Eq. (III.8) since the definition of the pushforward measure implies that the consistency condition is then satisfied because Q(A) = Q(B). The remaining term in Eq. (III.8) is determined using the prior measure PÂ£nor, the classical Bayes theorem for events, and the fact that B C A, P(B\A) PZnor(A\B)PZnor(B) Pprior(7l) prr(B) Pfor(A)' It follows from the definition of p^(pnod pPnor(^4) = Pp<-prior-)((5(R)). Thus, we utilize the following formal expression for Pa, Pa(B) : pprior /g\ PyS(Q(B)) a J p^prior\Q(B)) if PfOI(B) > 0, 0, otherwise, (III.9) The following proposition and ensuing theorem highlight the major steps needed to 19 prove the existence and uniqueness of a consistent Bayesian solution to the SIP given by a probablity measure on (A, Ba)- Proposition III.5 The probability measure given by P(/.(IIL9) defines a probability measure on (A ,CA). We note that, although Proposition III.5 says that Pa, as defined above, defines a probability measure on the contour Pa(A)= [ ( [ dPfrior(X)\ dPfibs(q), MA e BA, (IH.10) Jv ydanq-pg) J where prior / \ \ dPFnor{A) = JA K }-------diaAqW,Vq e V q p%{pnor)(Q( A)) ,q is a consistent solution to the SIP in the sense of Definition III.2. We note that PÂ£nor is used only to define the family of conditional probability measures {PfmoY}qeV on (A, Ba ), needed to obtain a unique solution to the SIP, in terms of the prior probability density and the map Q. It follows that Pa in Equation (IIP 10) is given by Pa(A) Ppr( a) AnQ~Hq) pv Q(prior) (Q(A)) dpA,qifii) dPfihS (q) (iii.ii) for all A E Ba- Furthermore, since dPffis(q) = p^sdpx>(q), we can substitute this into 20 Eq. (III. 11) to obtain the density MA)=prr(A) which, when integrated in terms of the above iterated integral, induces a probability measure P\ on (A,Ba) that is a consistent solution to the SIP. III.2.2 Numerical Approximation of Solutions Approximating pa using the consistent Bayesian approach requires an approximation of the pushforward of the prior probability measure on the model parameters, which is fundamentally a forward propagation of uncertainty. We consider the most basic of methods, namely Monte Carlo sampling, to sample from the prior. We evaluate the computational model for each of the samples from the prior and use a standard kernel density estimator (KDE) to approximate the pushforward of the prior. Given the approximation of the pushforward of the prior, we can evaluate pa at any point A G A if we have Q(A). Thus, we can construct an approximation of pa- We summarize the steps required to numerically approximate the pushforward of the prior in Algorithm 1, and summarize the steps required to approximate the solution to the SIP in Algorithm 2. We note that, the two prominent costs in producing a numerical approximation to pa are: (1) the evaluation of the model, and (2) the construction of an approximation to p^wlor\ Algorithm 1: Computing the Pushforward Density Induced by the Prior and the Model______________________________________________________________ 1. Given a set of samples from the prior density: A^, i = 1,... N; 2. Evaluate the model at each A^ and compute the Qol: = Q(A^); 3. Use the set of Qol and a standard technique, such as KDE, to estimate pU'W./). III.3 A Measure-Theoretic Solution to the SIP 21 Algorithm 2: Numerical Approximation of the Consistent Bayesian Solution 1. Given a prior density on A, p^nor; 2. Given an observed density on V, p^s; 3. Given the pushforward of the prior density on V, p^(prior) 4. Given a set of samples in A: A^, i = 1,..., N; 5. Evaluate the model at each A^; 6. Compute the value of pa at each A^: PA(A(i>) ftA"or)(<2(A(>)) (III. 13) 7. Use Pa(A<-*-)) to construct an approximation to pa- We follow [13, 32] and adopt what is referred to as the standard Ansatz determined by the disintegration of the volume measure pa to compute probabilities of events inside of a contour event, i.e., to define the family of probability measures on (A, Â£>A) required to obtain a unique solution to the SIP. The standard Ansatz is given by pm (P) HCiiCe) VB G BCe, for all Â£ G C, where pcÂ£ is the disintegrated volume measure on the generalized contour Ce. Note that, this is the same as pa,9 discussed above where Cg = Q~1(q)- Also, Bce denotes the standard Borel u-algebra on the Cg. The standard Ansatz can be used as long as pcÂ£ is finite for a.e. Â£ G C, e.g., as happens when A C Rd is compact, pa is the standard Lebesgue measure, and Q is continuously differentiable. Assuming such conditions hold to apply the standard Ansatz, the approximation method and resulting non-intrusive computational algorithm can be easily modified for almost any other Ansatz. See [13] for more details and theory regarding general choices of the Ansatz. Combining an Ansatz with the 22 disintegration theorem proves the following theorem. Theorem III.7 The probability measure Pa on (A,Â£>a) defined by Pa(A)= [ ( [ dPe(\))dPc(e), (in* 14) Jc yJArm^ie) J where Pt{B) = VB e BCi, WeÂ£ is a consistent solution to the SIP in the sense of Definition III.2. The standard Ansatz results in a probability measure PA that inherits key geometric features from the generalized contour map. Any other choice of Ansatz imposes some other geometric constraints on the solution to the inverse problem that are not coming from the map Q. If we choose not to use any Ansatz, then we can always solve the SIP on (A,Ca), i.e., we can always compute the unique probability measure when restricted to contour events. III.3.1 Numerical Approximation of Solutions Fundamental to approximating solutions PA to the SIP is the approximation of events in the various u-algebras, Â£>p, CA and Â£>A. Since CA C Â£>A, we can simultaneously approximate events in both of these u-algebras using the same set of events partitioning A. Let {denote such a partition of A where Vi G Â£>A for each i. Assume that we are given a collection of sets {Dk}^=1 C partitioning V, see Figure III.3. The basic algorithmic procedure for approximating PA(Vi) for each i is to determine which of the {Vfif=1 approximate Q~1(Dk), and then to apply the Ansatz on this approximation of the contour event which has known probability Pv(Dk), see Figure III.4. Letting pA;i denote the approximation of PA(Vi), we can then define an approximation to PA as N Pa(A) Pa,n(A) = Sa(A n Vi), A G Ba, i= 1 23 We summarize this basic procedure for approximating Pa in Algorithm 3. Algorithm 3: Numerical Approximation of the Measure-Theoretic Solution 1. Let {Vic Â£>a partition A, often defined by a set of samples {A^}^ . 2. Compute Vi = /^(W) for i = 1,N. 3. Determine a nominal value qW for the map Q(A) on Vi for i = 1,N, often q = Q{ A). 4. Choose a partitioning of V, {Dk}^=1 C V. 5. Compute P^s(Dk) for k = 1,M. 6. Let Ck = {i\q^ E Dk} for k = 1,M. 7. Let Oi = {k\q^ E Dk} for i = 1,N. 8. Set Pa(K) = V3)Pvs(D0z) for = 1,N. In Algorithm 3, Ck is used to determine which sets from {Vi}^=l approximate the contour event Q~1(Dk) for each k. Similarly, Oi is used to determine which contour event Q~1(Dk) is associated to Vi for each i. The Ansatz is applied in the final step where the probability of each Vi is determined by the probability P^s(D0i) multiplied by the ratio of the volume of Vi to the volume of the approximate contour event of Q~\D0i). We note that, the prominent costs in producing a numerical approximation to Pa are : (1) the evaluation of the model, and (2) the nearest neighbor searches employed in the various steps of Algorithm 3, sometimes referred to as binning. Depending on the assumption we make, we may reduce this cost in certain ways, i.e., given a set of i.i.d. random samples defining {Vi}f=1, we may make the MC assumption that fj,A(Vi) = //a(A)/A for each i. III.4 Comparison of Methods A brief discussion on the difference in how each of the above methods defines a unique solution to the SIP is warranted. As noted, the definition of a family of 24 A V Figure III.3: We summarize the steps in Algorithm 3. (top row) We depict Steps 1-3. We partition A with {Vi}f=1 that are implicitly defined by the set of samples A}" we make the MC assumption to determine the volume of each Vj. Then we determine the nominal value = Q(A^). Here, N = 2000. (bottom row) We depict Steps 4-5. We partition V with a regular grid and prescribe the observed probablity measure to be constant over the darkened set of W Here M = 100. 25 A V Figure III.4: Summary of Algorithm 3 continued, (top row) We depict Steps 5-6. For each k, we determine which set of A^ map into Dk. In red, we see the union of the Vi corresponding to the Ai that map into D\ seen in red in V. We see the same for D2 in blue, (bottom row) We depict Step 8. We compute the approximate solution to the SIP where we proportion out the probability of each Dk (determined by the observed measure) to its corresponding Vi. Darker colors correspond to higher probability. 26 conditional probability measures on (A, B\) defines a unique solution to the SIP. The consistent Bayesian method utilizes the data space V and prior information on the parameter space to define this family of conditional probability measures which are ultimately described in terms of their corresponding probability densities, see Eq. (III. 11). This method requires the pushforward of the prior density and results in an approximation to the value of Pa (A) at any point A G A that we have Q( A). We may then use the {pa(A(-A)}^1 to construct an approximation to pa- Alternatively, we may generate samples from pA using an accept/reject algorithm without actually constructing pA. The measure-theoretic method utilizes the transverse parameterization C, implicitly, and defines the family of conditional probability measures on (A, B\) based on the underlying volume measure This does not require any prior information on the input parameters. This method requires various nearest neighbor searches and results in an approximation to Pa on a discretized parameter space. We may use this approximation to Pa to construct an approximation to p\ and generate samples from pa using an accept/reject algorithm. However, the samples used to discretize and approximate Pa need not come from an underlying probability measure and can be placed optimally using only information about the map Q and data space discretization, see [15]. III.5 Numerical Example: Consistency Here, we consider an accessible nonlinear system of equations to display the consistency of the solution to the SIP. We note that, given a uniform prior, the consistent Bayesian solution is identical to the measure-theoretic solution under the standard Ansatz [16]. In other words, the two methods produce the same consistent solution to the SIP under assumption of a uniform prior. Therefore, in this example, we utilize the measure-theoretic algorithm (i.e., Algorithm 3), under the standard Ansatz, for approximating solutions to the SIP. The following consistent solutions to the SIP are 27 computed using the Butler, Estep, Tavener (BET) Python package, which is designed to solve measure-theoretic stochastic inverse and forward problems [32], We formulate and solve three SIPs in order to both illustrate the consistency property of the solution to a given SIP and to motivate the remaining chapters. Consider the following 2-component nonlinear system of equations introduced in [10]: Xixf + x\ = 1 x\ A2^2 = 1 The parameter ranges are given by Ai G [0.79,0.99] and A2 G [1 4.5-\/0T, 1 + 4.5-\/0T] which are chosen as in [10] to induce an interesting variation in the solution (aq(A),Â£2(A)). We consider three Qol maps defined by the solution to this system, Q[l) =ari(A), Q{2) = x2(X), Q(3) = (xi(X),x2(X)). The corresponding solutions to the SIPs are denoted, respectively, as Pa\pa^> and (3) Pa- Consider and assume that the observed density on is a truncated normal distribution with mean 1.015 and standard deviation of 0.007, see Figure III.5 (bottom). We generate IE + 5 independent identically distributed (i.i.d.) samples in A, use the BET Python package to construct an approximation to P^\ and a simple accept/reject algorithm to generate a set of samples from the resulting p^. We propagate this set of samples from p^ through the model and approximate the resulting pushforward density using a KDE, see Figure III.5. The bottom image in Figure III.5 displays the consistency property of this solution to the SIP, i.e., the 28 pushforward of p^A matches the observed density. 0.80 0.85 0.90 0.95 Al 5.6 4.8 3.2 CL> 3D (D 1.6 0.8 Samples from A* 0.80 0.85 0.90 0.95 Ai 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 Xl(A) Figure III.5: (top left) Approximation of p^ obtained using p^s on Q^, (top right) a set of samples from pA\ (bottom) and a comparison of the observed density Pps on Qd) with the pushforward of pA\ Consider Qd) and assume that the observed density on Qd) js a truncated normal (2) distribution with mean 0.3 and standard deviation of 0.035. We approximate PA (2) using the same IE + 5 samples, generate a set of samples from the resulting pA and (2) propagate these samples through the model to approximate the pushforward of pA , see Figure III.6. Again, we observe the consistency of pA in that the pushforward of (2) pA matches the observed density. Next, we consider using the vector-valued Qol map Qd). We take the observed (joint) density on Qd) to |3e the product of the (marginal) densities on Qd) and 29 (2) 0.80 0.85 0.90 0.95 Al Samples from p? 0.1 0.2 0.3 0.4 0.5 *2 (A) Figure III.6: (top left) Approximation of p^ obtained using p^s on Q<'2\ (top right) a set of samples from p^\ (bottom) and a comparison of the observed density Pps on Q^ with the pushforward of p^\ 30 Q^ as specified above. Again, we approximate PA^ and generate a set of samples from the resulting pA using the same IE + 5 samples, see Figure III.T. With the information from both ;ri(A) and #2(A), we see a substantial decrease in the support ("S') of pA Intuitively, with the assumptions above, we expect that supp(p^) C supp(p^) Pi supp(p^). (M Figure III.7: (left) The approximation of p^ using the joint density p^s on Qd, (right) a set of samples from pA\ ("3) We propagate the set of samples from pA through the model and calculate the means and standard deviations of the resulting set of samples to construct the push-forward of pA In Figure III.8, we show the pushforward of pA the observed density, and the error between the two scaled by the maximum of the observed density to display a relative error. In the scenario in which we gather data leading to an observed density on both Qd) and compared to the scenarios where we only gather data on Qd) or Q(2) but not both, there is a clear reduction of uncertainties as is evidenced by the reduced support in pA and its overall structure. However, suppose we could only gather data on one of these Qol in an experimental/held setting. Choosing either Qd) or q(2) 31 Pushforward of p^3) Observed 0.5 0.4 -c 0.3 - 0.2 0.1 - 607.5 540.0 472.5 ^ =5 405.0 (U 337.5 -g _l 270 0 T, 202.5 _H* Q. 135.0 67.5 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 x\ (A) 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 Xl (A) Error xi(A) Ho.0300 Bo.0225 Bo.0150 - 0.0075 0.0000 -0.0075 B 0.0150 B 0.0225 B -0.0300 B 0.0375 594 528 462 396 330 264 198 132 66 Figure III.8: (top left) The pushforward of p^\ (top right) the exact observed density, (bottom) the difference between the pushforwad of p and the observed density scaled by the maximum of the observed density. 32 Pxux, (Lebesgue) defines a separate ED. It is not immediately clear from the figures which ED is more useful. In the remaining chapters, we seek to quantify the utility of an ED under consideration in order to determine the optimal Qol map to be used in formulating and solving a SIP. In Chapter IV, we quantify the expected utility of a given ED using the KL divergence. In Chapter V, we quantify the expected utility of an ED in terms of local geometric properties of the corresponding Qol map that describe the expected precision and accuracy of the resulting solution to a SIP. In Chapter VI, we provide an in depth comparison between these two SB-OED methods focused specifically on a handful of challenging SB-OED scenarios. In Chapter VII, we continue to extend the measure-theoretic approach to SB-OED by contributing further to the quantification of the computational complexity of a SIP. In Chapter VIII, we illustrate a greedy implementation of the measure-theoretic approach to SB-OED in a high-dimensional setting. 33 CHAPTER IV CONSISTENT BAYESIAN OPTIMAL EXPERIMENTAL DESIGN In this chapter, we develop a SB-OED approach based on the consistent Bayesian method for solving SIPs. We follow the Bayesian methodology for SB-OED discussed in Chapter II and [35, 36, 9]. This methodology quantifies the utility of a given ED defined by a Qol map Q based on a set of densities corresponding to solutions of SIPs for Q which, in turn, are based on the types of data that we expect to observe in the held for the given Q. Since the consistent Bayesian method for solving SIPs has different assumptions from the classical Bayesian method, we modify the classical Bayesian SB-OED methodology as needed. In Section IV. 1, we consider the information content of a given ED and define the OED problem. In Section IV.2, we address the issue of infeasible data and discuss computational considerations. We provide numerical examples in Section IV.3 to display the efficacy of this method on two toy problems for which we have significant intuition regarding the location of the OED. IV. 1 The Information Content of an Experimental Design Conceptually, an ED is considered informative if the resulting solution to the SIP is significantly different from the prior probability measure. To quantify the information gain of an ED we use the KL divergence [64], which is a measure of the difference between two probability densities. While the KL divergence is by no means the only way to compare two probability densities, it does provide a reasonable measure of the information gained in the sense of Shannon information [22] and is commonly used in Bayesian OED [35, 9]. In this section, we discuss computation of the KL divergence and define an OED formulation based upon the expected information gain over a specified space of probable observed densities. IV. 1.1 Information Gain: Kullback-Leibler Divergence 34 Suppose we are given a description of the uncertainty on the observed data for a Qol map Q in terms of a probability measure Pgbs that can be represented as a density p^s. In other words, Pgbs is absolutely continuous with respect to pp> and p^s is the Radon-Nikodym derivative of Ppbs. As presented in Chapter III, given prior information about the parameters described as a probability measure PÂ£nor, the SIP has a unique solution given by a probability measure Pa that is absolutely continuous with respect to pa, and we let pa denote the probability density. The KL divergence, i.e., information gain, of pa from p^nor, denoted KLq to emphasize the role Q plays in determining pA, is given by KLq{pT Pa) := PA log (IV. 1) Since p^rior is fixed for a given ED, KLq is simply a function of p\. However, from Eq. (III. 12), we see that p\ is simply a function of pps. Therefore, to make this dependence explicit, we write KLq as a function of the observed density, KLq(j%*) := KLQ(pf^ : pA). (IV.2) The observation that KLq is a function of only p^s is critical to defining the expected information gain on a well-defined space of observed densities, as done in Section IV. 1.2. Example IV. 1 (Information Gain) Consider the follovjing 2-component nonlinear system of equations with two parameters discussed in Section III. 5, A\Xi T X2 = 1, x\ A2^2 = 1. 35 Recall that the parameter ranges are given by Ai G [0.79,0.99] and A2 G [1 4.5-\/oT, 1 + 4.5v/0^I] As before, we consider three Qol maps defined by the solution to this system, Q{1) =ari(A), Q{2) = x2(X), Q(3) = (xi(X),x2(X)). The corresponding solutions to the SIPs are denoted, respectively, as pA >Pa >Pa Consider QAl and assume the observed density cm QAl is a truncated normal distribution with mean 0.3 and standard deviation of 0.01, see Figure IV. 1. We generate 40,000 samples from the uniform prior and construct an approximation to the pushforward of the prior with Algorithm 1, see Figure IV. 1 (right). Then, we use Algorithm 2 to construct an approximation to p^A using the same 40,000 samples, see Figure IV.1 (left). Notice the support of p^A lies in a relatively large region of the parameter space. The information gain from p^A approximated using Equation (IV. 1), is KLq(i)(p))s) 0.466. Next, we consider p^A associated with solving the SIP using QA\ We assume the observed density on QAl is a truncated normal distribution with mean 1.015 and standard deviation of 0.01. We approximate the pushforward of the prior and pA with Algorithm 1 and Algorithm 2, using the same 40,000 samples, see Figure IV.2. Although the observed densities cm QAl and QAl have the same standard deviation, clearly p^A and p^A are different. Visually, p^A has a much larger region of support within the parameter space compared to pA This is quantified with the information gain from p^A KLq(2)(p))s) ~ 2.015. Given these two maps, QA) and QA), and the specified observed densities on each of these data spaces, the experiment leading to QAl and the corresponding pfi)s is more informative than the experiment 36 Observed / pushforward of prior Figure IV. 1: (left) Approximation of p^ obtained using p^s on which gives ALq(i)(Pps) 0 .466. (right) p^s on Qd) anc[ the pushforward of the prior. Observed / pushforward of prior Figure IV.2: (left) Approximation of p^ obtained using Qd) anc[ pg>s on Q(2\ which gives K Lq(2)(p^s) ~ 2.015. (right) p^s on Qd) anc[ the pushforward of the prior. 37 leading to Q^ and the corresponding pfifs. Next, we consider using the vector-valued Qol map We take the observed (joint) density on Q^ to be the product of the (marginal) densities on and Q^ as specified above. Again, we approximate the pushforward of the prior and pA using the same 40,000 samples, see Figure IV.3. With the joint information, we see a substantial decrease in the support of pA This is quantified in the information gain of this KLQ(3)(p$s) 2 .98. Thus, we see the largest information gain by using Q^ and the joint density observed on Q^. However, in practice, we may be limited to specify a single Qol for which we can collect data to construct an observed density. Based on the computed information gains, we may conclude that Q^ is more informative of the parameters than However, due to nonlinearities of the maps, it is not necessarily true that Q^ is more informative than if different observed densities are specified. Since we do not know the observed densities prior to collecting actual data, we are left to determine which of these Qol we expect to produce the most informative p\. IV. 1.2 Expected Information Gain OED must select an ED before experimental data becomes available. In the absence of data, we use the computational model to quantify the expected information gain (EIG) of a given ED defined by a Qol map Q. Given Q and V = Q(A), let Ov denote the space of all probability density functions over V. Defining the EIG over Ox> is computationally impossible since Ox> contains uncountably infinitely many densities. Moreover, Ox> includes densities that are unlikely to be observed in practice for a given Q. Therefore, we restrict Ox> to be a space more representative of densities that may be observed in practice for a particular ED defined by Q. With no experimental data available to specify an observed density on V for a given Qol map Q, we assume the observed density is a Gaussian with mean belonging to V and a diagonal m x m covariance matrix, i.e., we assume the uncertainties in 38 Pushforward of prior Observed Figure IV.3: (top left) The approximation of the pushforward of the prior, (top right) The exact p^s on Q{~i\ (bottom) The approximation of p^ using p^s on Q which gives KLQ(3){p^s) 2.98. 39 each Qol are uncorrelated. For a given Qol map Q, we let Oq denote the space of all densities of this profile with mean belonging to V = Q(A), Oq = jpp G Ot> pv ~ N(p, E), p G V. (IV.3) Thus, px> is assumed to be a Gaussian density function with mean p G V and diagonal covariance matrix E. We may restrict Oq in other ways as well. For example, we may expect the uncertainty in each Qol to be described by a uniform density of a given width, in which case we would define the restriction on Oq accordingly. This choice of profile for the observed densities within Oq is largely dependent on the application and specification of Q. The only limitation on the observed density space comes from the pushforward measure on the data space, p^pnof>) ag described below. The restriction of possible p^s to this specific space of densities allows us to index each density by a unique q G V. Based on our prior knowledge of the parameters and the sensitivities of the map Q, the model informs us that some data are more likely to be observed than other data, this is seen in the plot of p^(prior) in Figure IV.3. Therefore, we average over V with respect to the pushforward of the prior. This utilizes both the prior knowledge of the parameters and the sensitivity information provided by the model. We define the EIG, denoted E(KLq), as E(.KLq) := [ KLQ^r^'iqWv- (IV.4) Jv From Eq. (IV. 1), KLq itself is defined in terms of an integral. The expanded form for E(KLq) is then an iterated integral, E(A'io) = jf MA;9)log (^^WdfÂ£
40 |

Full Text |

PAGE 1 SIMULATION-BASEDOPTIMALEXPERIMENTALDESIGN:THEORIES, ALGORITHMS,ANDPRACTICALCONSIDERATIONS by SCOTTN.WALSH B.S.,Mathematics,ClevelandStateUniversity,2009 B.A.,Physics,ClevelandStateUniversity,2009 M.S.,AppliedMathematics,UniversityofColoradoDenver,2016 Athesissubmittedtothe FacultyoftheGraduateSchoolofthe UniversityofColoradoinpartialfulllment oftherequirementsforthedegreeof DoctorofPhilosophy AppliedMathematics 2017 PAGE 2 ThisthesisfortheDoctorofPhilosophydegreeby ScottN.Walsh hasbeenapprovedforthe AppliedMathematicsProgram by StephenBillups,Chair TroyButler,Advisor JulienLangou VarisCarey YailJimmyKim July29,2017 ii PAGE 3 Walsh,ScottN.Ph.D.,AppliedMathematics Simulation-BasedOptimalExperimentalDesign:Theories,Algorithms,and PracticalConsiderations ThesisdirectedbyAssistantProfessorTroyD.Butler ABSTRACT Severalnewapproachesforsimulation-basedoptimalexperimentaldesignare developedbasedonameasure-theoreticformulationforastochasticinverseproblem SIP.Here,anexperimentaldesignEDreferstoanexperimentdeningaparticular mapfromthespaceofmodelinputstothespaceofobservablemodeloutputs.The termoptimalexperimentaldesignOEDthenreferstothechoiceofamapfrom modelinputstoobservablemodeloutputswithoptimalproperties.Wedevelopa consistent BayesianapproachtoOED,whichfollowstheclassicalBayesianapproach toOED,anddeterminestheOEDbasedonpropertiesofapproximatesolutionstoa setofrepresentativeSIPs. Subsequently,wedevelopanOEDapproachbasedentirelyonquantiablegeometricpropertiesofthemapsdeningtheEDs.Weproveecientcomputableapproximationstothesequantitiesbasedonsingularvaluedecompositionsofsampled Jacobianmatricesoftheproposedmaps.Weexaminethesimilaritiesanddierences amongthesetwoapproachesonasetofchallengingOEDscenarios.Anewdescription ofthecomputationalcomplexityisintroducedand,undercertainassumptions,shown toprovideaboundonthesetapproximationerrorofaninverseimage.Furthermore, agreedyalgorithmisdescribedforimplementationoftheseOEDapproachesincomputationallyintensivemodelingscenarios.Severalnumericalexamplesillustratethe variousconceptsthroughoutthisthesis. Theformandcontentofthisabstractareapproved.Irecommenditspublication. Approved:TroyD.Butler iii PAGE 4 ACKNOWLEDGMENTS Firstandforemost,IwouldliketothankmyadvisorTroyButlerforhisdedication andsupportthroughoutmygraduatecareer.Iamincrediblyfortunatetohavecrossed pathswithTroyduringhisrstyearsasafacultymemberatCUDenver.Duringa timewhenthefutureofmygraduatecareerwasuncertain,Troyintroducedmetoa fascinatingeldofresearchwhichencouragedmetocontinuetopursuethisdegree. Heprovidedmewithopportunityafteropportunitytocontributetoafascinatingeld ofresearchwhilealsobuildingcollaborationswithresearchersatvariousinstitutions andlaboratories.Hiswillingnesstomentormeandprovidemewiththeskillsand knowledgerequiredtocontributetothiseldofresearchisthereasonthisthesisis completetoday.Iwouldalsoliketothankmycommitteemembers:StephenBillups, JulienLangou,VarisCarey,andYailJimmyKim.Ihavebenetedgreatlyfromtheir supportandadviceinthemakingofthisthesis. IwouldliketothankLindleyGraham,SteveMattisandClintDawsonfortheir guidanceandsupportduringmysummerspentattheInstituteforComputational andEngineeringSciencesatUTAustin.Thediscussionswehad,andperspectives thattheyprovided,wereinvaluabletomysuccess.IwouldliketothankTimWildey andJohnJakemanfortheirthoughtfuldiscussionsduringmysummerspentatSandiaNationalLaboratories.Theiropenmindsandwillingnesstoentertainmyideas enabledmetogrowasaresearcher,bothduringthatsummerandthroughoutmy graduatecareer. Therearemanyfriendsandfellowgraduatestudentswhohelpedmethrough graduateschoolinavarietyways.IwouldliketothankAbulitibuTugulukefor spendinghisSaturdaymorningsstudyingfortheanalysisprelimwithme.Iwould liketothankDevonSiglerforhisinsightfulquestionsanddiscussionsonthepatioof TheMarket.Withoutadoubt,theseconversationscontributedgreatlytothequality iv PAGE 5 andclarityofpartsofthisthesis.Icannotlistallofmyfriendsandfellowgraduate studentswhohavehelpedmethroughthisprocess,butthankyouEmileighWillems, MeganSorenson,MichaelPilosov,JordanHall,LucasOrtiz,TakaoMiller,andTrever Hallock. Last,butnotleast,Iwouldliketothankmyparents.MyMom,forherencouragementandsupportduringthediculttimesofgraduateschool.MyDad,forhis timespentbothhelpingmetoovercomethechallengesoftheprelimsanddiscussing myobscureanalysisquestions.Hisabilitytoexplainchallengingconceptsinvarious waysprovidedmewiththedeepunderstandingrequiredtocompletetheprelimsand ultimatelycontributetotheeldofappliedmathematics. v PAGE 6 TABLEOFCONTENTS CHAPTER I.INTRODUCTION..............................1 I.1StochasticInverseProblems....................1 I.2OptimalExperimentalDesign...................2 I.3Overview...............................3 II.LITERATUREREVIEW..........................6 II.1BayesianSimulation-BasedOptimalExperimentalDesign....6 II.2ThesisObjectives..........................8 III.AMEASURE-THEORETICFRAMEWORKFORFORMULATINGSTOCHASTICINVERSEPROBLEMS........................10 III.1FormulatingtheStochasticInverseProblemUsingMeasureTheory11 III.2AConsistentBayesianSolutiontotheSIP............18 III.3AMeasure-TheoreticSolutiontotheSIP.............22 III.4ComparisonofMethods......................24 III.5NumericalExample:Consistency.................27 IV.CONSISTENTBAYESIANOPTIMALEXPERIMENTALDESIGN..34 IV.1TheInformationContentofanExperimentalDesign.......34 IV.2InfeasibleData...........................42 IV.3NumericalExamples........................47 V.MEASURE-THEORETICOPTIMALEXPERIMENTALDESIGN...53 V.1Notation...............................53 V.2 d -dimensionalParallelepipeds...................54 V.3SkewnessandAccuracy.......................57 V.4ScalingandPrecision........................61 V.5AMulti-ObjectiveOptimizationProblem.............63 V.6NumericalExamples........................65 vi PAGE 7 V.7Summary..............................70 VI.EXPECTEDSCALINGEFFECTVSEXPECTEDINFORMATIONGAIN71 VI.1TechnicalLemmas..........................72 VI.2ComputationalConsiderations...................74 VI.3HighlySensitiveQoI........................76 VI.4UnknownProleoftheObservedDensity.............79 VI.5RedundantData..........................95 VI.6Summary..............................97 VII.THECOMPUTATIONALCOMPLEXITYOFTHESIP.........99 VII.1AReformulationoftheSIPforEpistemicUncertainty......100 VII.2Single-ObjectiveOptimizationProblem..............105 VII.3ABoundontheErrorofSetApproximations..........111 VII.4Summary..............................115 VIII.AGREEDYIMPLEMENTATIONFORSB-OED.............119 VIII.1AGreedyAlgorithm........................119 VIII.2NumericalExample.........................120 IX.CONCLUSIONS...............................129 IX.1Summary..............................129 IX.2FutureWorkandOpenProblems.................130 REFERENCES...................................133 APPENDIX A.RADIALBASISFUNCTIONSFORDERIVIATIVEAPPROXIMATIONS139 A.1UsingRBFstoInterpolateUnstructuredData..........139 A.2ConvergenceoftheExpectedRelativeError...........141 vii PAGE 8 FIGURES III.1leftTheinverseof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 Q isoftenset-valuedevenwhen 2 speciesaparticular Q 2D .middleTherepresentationof L asa transverseparameterization.rightAprobabilitymeasuredescribedas adensityon D ; B D mapsuniquelytoaprobabilitydensityon L ; B L Figuresadoptedfrom[13]and[10]......................13 III.2Illustrationofthemapping Q : !D .Notethat Q maps B toa particularregion Q B 2D ,andwhiletheinverseimageofthisset,given by Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 Q B ,contains B ,otherpointsin mayalsomapto Q B Figureadoptedfrom[16]..........................14 III.3WesummarizethestepsinAlgorithm3.toprowWedepictSteps13.Wepartition with fV i g N i =1 thatareimplicitlydenedbythesetof samples i N i =1 ,wemaketheMCassumptiontodeterminethevolume ofeach V i .Thenwedeterminethenominalvalue q i = Q i .Here, N =2000.bottomrowWedepictSteps4-5.Wepartition D witha regulargridandprescribetheobservedprobablitymeasuretobeconstant overthedarkenedsetof V i .Here M =100.................25 III.4SummaryofAlgorithm3continued.toprowWedepictSteps5-6.For each k ,wedeterminewhichsetof i mapinto D k .Inred,weseethe unionofthe V i correspondingtothe i thatmapinto D 1 seeninredin D .Weseethesamefor D 2 inblue.bottomrowWedepictStep8.We computetheapproximatesolutiontotheSIPwhereweproportionout theprobabilityofeach D k determinedbytheobservedmeasuretoits corresponding V i .Darkercolorscorrespondtohigherprobability.....26 III.5topleftApproximationof obtainedusing obs D on Q ,topright asetofsamplesfrom ,bottomandacomparisonoftheobserved density obs D on Q withthepushforwardof ..............29 viii PAGE 9 III.6topleftApproximationof obtainedusing obs D on Q ,topright asetofsamplesfrom ,bottomandacomparisonoftheobserved density obs D on Q withthepushforwardof ..............30 III.7leftTheapproximationof usingthejointdensity obs D on Q ,right asetofsamplesfrom ...........................31 III.8topleftThepushforwardof ,toprighttheexactobserveddensity, bottomthedierencebetweenthepushforwadof andtheobserved densityscaledbythemaximumoftheobserveddensity..........32 IV.1leftApproximationof obtainedusing obs D on Q ,whichgives KL Q obs D 0 : 466.right obs D on Q andthepushforwardofthe prior......................................37 IV.2leftApproximationof obtainedusing Q and obs D on Q ,which gives KL Q obs D 2 : 015.right obs D on Q andthepushforwardof theprior....................................37 IV.3topleftTheapproximationofthepushforwardoftheprior.topright Theexact obs D on Q .bottomTheapproximationof using obs D on Q ,whichgives KL Q obs D 2 : 98....................39 IV.4leftThepushforwardofthepriorandobserveddensitieson Q .right Thepushforwardofthepriorandobserveddensitieson .Noticethe supportofeachobserveddensityiscontainedwithintherangeofthe model,i.e.,theobserveddensitiesareabsolutelycontinuouswithrespect totheircorrespondingpushforwarddensities................44 IV.5topleftThepushforwardofthepriorforthemap Q .toprightThe observeddensityusingthejointinformationwhichnowextendsbeyond therangeofthemap.bottomThenormalizedobserveddensitythat doesnotextendbeyondtherangeofthemap................46 ix PAGE 10 IV.6SmoothedplotoftheEIGovertheindexingsetforthedesignspace Q Noticethehighervaluesinthecenterofthedomainandtowardsthetop rightinthedirectionoftheconvectionvectorfromthelocationofthe source,thisisconsistentwithourintuition.................49 IV.7leftSmoothedplotoftheEIGovertheindexingset forthedesign space Q .Noticethehighvaluesinthebottomrightcornerof .This locationcorrespondstoplacingthermometersnearseparateendsofthe rod.However,weobservemultiplelocalmaximawithinthedesignspace, denotedR1,R2,R3andR4.rightTheEIGforthelocalOEDineach ofthefourregions...............................52 V.1TheESEdescribeshowuncertaintiesofaxedsizeonthedataspaceare reducedundertheinverseoftheQoImap..................63 V.2TheESEdescribeshowmuchuncertaintyonthedataspaceisreducedto thesizeoftheunitcube............................64 V.3SmoothedplotoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingsetforthedesignspace Q .Noticethehighervaluesinthecenterofthedomainandtowardsthe toprightinthedirectionoftheconvectionvectorfromthelocationof thesource,thisisconsistentwithbothintuitionandtheresultsfrom ExampleIV.3.................................67 V.4leftSmoothedplotoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 overtheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof .Thislocationcorrespondstoplacingthermometersneareachend oftherod.However,weobservefourprominantlocalmaxima,denoted R1,R2,R3andR4.rightTheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 forthelocalOEDineachofthe fourregions...................................68 x PAGE 11 V.5leftSmoothedplotoftheESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 overtheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof Thislocationcorrespondstoplacingthermometersneareachendofthe rod.However,weobservefourprominantlocalmaxima,denotedR1,R2, R3andR4.rightTheESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthelocalOEDineachofthefourregions.69 V.6leftSmoothedplotof d Y p;y )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,with =0 : 5,overtheindexingset forthedesignspace Q .Noticethehighvaluesinthebottomright cornerof .Thislocationcorrespondstoplacingthermometersnear eachendoftherod.Here,weobserveonlyoneprominantlocalmaxima. right d Y p;y )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,with =0 : 5,forthelocalOEDinthesingleregion ofinterest....................................69 VI.1ThetwogroupsofEDsshownintheindexingset .Thegroupin thebottomrightoveraredregionistheoptimalgroupofEDsandthe groupneartheupperrightoverablueregionisthesub-optimalgroup. Forreference,weshowthesetwoclustersplottedontopoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 computedusingall5,000availablesamplesin asdescribedabove...77 VI.2leftTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthe20EDsunderconsideration.Noticetheclear dierenceinESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 valuesbetweentheoptimalgroupredandthesuboptimalgroupblue.Theoptimalgrouplookstobejustasingleline, however,thereareactually10linesplottedhereandtheysimplylieso closetogetherthattheyareindistinguishable.rightTheEIGforthe20 EDsunderconsideration.Inthisimage,wedonotseeacleardistinction betweenthetwogroupsofEDs.Again,thereare10redlinesand10 blueslineshere,theyarejustnotdistinguishable.SeeTableVI.1fora quantitativedescriptionoftheseEIGvalues.................79 xi PAGE 12 VI.3ThetwogroupsofEDsshownintheindexingset .Thegroupin thebottomrightoveraredregionistheoptimalgroupofEDsandthe othergroupoverablueregionisthesub-optimalgroup.Forreference, weshowthesetwoclustersplottedontopoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 computedusing N =100availablesamplesin asdescribedabove............82 VI.4leftWeshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 asafunctionof toemphsizethattheESEis denedindependentoftheexpectedproleoftheobserveddensityoneach QoI.rightTheEIGforthe100EDsunderconsiderationasafunction of .Theoptimalgroupisshowninredandthesub-optimalgroupis showninblue.Notethat,as increase,theEIGfortheoptimalcluster becomessmallerthantheEIGforthesub-optimalgroup..........84 VI.5TheEIGovertheindexingset andthetop100OEDsforincreasing .Forverysmall =1 E )]TJ/F15 11.9552 Tf 12.028 0 Td [(05,weseethatthetop100OEDsappearto berandompointsin .For0 : 1 1 : 0,thetop100OEDsareall clusteredinthebottomrightof .Then,assigmaincreasebeyond 1 : 0,thetop100OEDsmoveawayfromthebottomrightof and settleintworegionsnearthecenterofthespace..............85 VI.6topleftApproximationof using obs D on Q with =1E-07,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E-07,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............87 VI.7topleftApproximationof using obs D on Q with =1E-03,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E-03,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............88 xii PAGE 13 VI.8topleftApproximationof using obs D on Q with =1E-02,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E-02,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............89 VI.9topleftApproximationof using obs D on Q with =2E-02,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =2E-02,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............90 VI.10topleftApproximationof using obs D on Q with =3E-02,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =3E-02,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............91 VI.11topleftApproximationof using obs D on Q with =5E-02,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =5E-02,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............92 VI.12topleftApproximationof using obs D on Q with =1E-01,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E-01,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ..............93 xiii PAGE 14 VI.13topleftApproximationof using obs D on Q with =1E+00,top rightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q .topleftApproximationof using obs D on Q with =1E+00,toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardoftheprioron Q ...........94 VI.14SmoothedplotsoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andtheEIGovertheindexingset forthedesignspace Q .............................95 VI.15SmoothedplotsoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 andtheEIGovertheindexingsetforthe designspace Q .................................96 VII.1leftIngreen,weshowtheapproximateinverseimagewiththegiven Voronoitesselationof .Theexactinverseimageisshowninblue.Notice thetrueparameter true inrediscontainedwithintheexactinverseimage, however,itisnotcontainedwithintheapproximation.rightInblue,we showtheevent E 2B D thatdenes Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,thetruedatuminred,and theobserveddatuminblackthatisresponsibleforthedenitionof E 2B D .101 VII.2Inpurpleweshow Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,inyellowweshowtheinatedinverseimage resultingfromextendingtheboundariesoutward,andingreenweshow theapproximationtotheinatedinverseimagethatcovers Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Note that,thisinationisdeterminedbyextendingtheboundariesoutward eachdirectionbyadistanceof R N ,themaximumradiusoftheVoronoi cellsinthetesselation.............................102 VII.3topleftTheinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E andtheapproximation.topright The E 2B D deningtheuncertainitesontheobserveddata.bottom rightTheinated ~ E 2B D determinedbytheJacobianof Q and R N bottomleftTheexactinverseimage Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,theinatedinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E ,andtheresultingapproximationto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E thatcovers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E .104 xiv PAGE 15 VII.4leftArepresentativeinverseimageresultingfrominverting E 2B D usingthemap Q a .rightArepresentativeinverseimageresultingfrom inverting E 2B D usingthemap Q b .Although Q b producesasmaller inverseimageusingthesamesized E ,thecorrelationamongstthetwo QoIcomposing Q b producesaninverseimagethatismoredicultto approximateaccuratelywithnitesampling.................109 VII.5Theinverseoftheinationeect,forboth Q a and Q b ,asafunctionof thenumberofsamplesavailable.Although M Q b PAGE 16 VIII.3topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensorsare showninblack,andtheoptimallocationforthenextsensorisshown inwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q Noticethatthepreviouslyplacedsensorsareshowninblack,andthe optimallocationforthenextsensorisshowninwhiteatthemaximumof theESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ...................................125 VIII.4topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensorsare showninblack,andtheoptimallocationforthenextsensorisshown inwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q Noticethatthepreviouslyplacedsensorsareshowninblack,andthe optimallocationforthenextsensorisshowninwhiteatthemaximumof theESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ...................................126 VIII.5topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensorsare showninblack,andtheoptimallocationforthenextsensorisshown inwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q Noticethatthepreviouslyplacedsensorsareshowninblack,andthe optimallocationforthenextsensorisshowninwhiteatthemaximumof theESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ...................................127 VIII.6topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensorsare showninblack,andtheoptimallocationforthenextsensorisshown inwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q Noticethatthepreviouslyplacedsensorsareshowninblack,andthe optimallocationforthenextsensorisshowninwhiteatthemaximumof theESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ...................................128 A.1Theconvergenceoftheexpectedrelativeerrorforeachcomponentof Eq.A.1.Noticeeachcomponentof r f convergesatapproximatelythe rate O 1 N ...................................142 xvi PAGE 17 A.2Theconvergenceoftheexpectedrelativeerrorforeachcomponentof Eq.A.2.Noticeeachconvergesatapproximatelytherate O 1 N .....143 xvii PAGE 18 CHAPTERI INTRODUCTION I.1StochasticInverseProblems Scientistsandengineersstrivetoaccuratelymodelphysicalsystemswiththe intenttopredictfutureand/orunobservablequantitiesofinterestQoI.Forexample, ascientistmodelingthetransportofacontaminantsourcemaybeinterestedin predictingtheaverageconcentrationofthecontaminantinaregionofthedomain atsomefuturetime.Thatsamescientistmayalsobeinterestedintheuxofthe contaminantthroughsomeboundarythatisphysicallyunobservable,i.e.,cannotbe determinedbydirectmeasurements.Bothofthesescenariosrequiretheconstruction ofacomputationalmodelthatsimulatesthephysicalprocessofinterest.Inmany applications,theextraordinarycomplexityofthemodelinducesasetofmodelinputs thatareuncertain.Typically,modelinputsaredescribedasbelongingtoageneralset ofphysicallyplausiblevaluesusingeitherengineeringordomain-specicknowledge. Usingthecomputationalmodel,thissetofplausiblemodelinputscanbepropagated throughthemodeltomakepredictionsonunobservableQoI.WhentheQoIare sensitivetoperturbationsinmodelinputs,thepredictivecapabilitiesofeventhemost sophisticatedcomputationalmodelsmaybeseverelylimitedbyacoarsedescription ofuncertaintiesinthemodelinputs. Oftentimes,researcherslooktoreducetheseuncertaintiesbyusingdatagatheredfromtheactualphysicalprocesssimulatedbythemodel.Theideabeing,by ttingthemodeltoobservabledata,theprobabilitythatthemodelaccuratelypredictsunobservabledataisincreased.Thisprocessiscommonlyreferredtoasmodel calibration.However,tting"orcalibratingthemodeltoobserveddatacanbea diculttask.Itiscommonfortheobserveddatatobesubjecttouncertainty,e.g., measurementinstrumenterrorreducesthecondenceonehasintheexactvaluesprovidedbythedata.Suchuncertaintiesareoftendescribedusingprobabilitymodels, 1 PAGE 19 andspecicationofaprobabilitymeasureontherangeofvaluesforaQoIisone approachtorepresentuncertaintiesinobservabledata.Then,ratherthancalibrating themodeltoaxedsetofobserveddata,weformulateandsolveastochasticinverse problemSIPinordertodetermineaprobabilitymeasureonthespaceofmodel inputsinformed"bytheprobabilitymeasureontheQoI. TherearemanyframeworksforformulatingSIPsbasedondierentassumptions andmodelsfortheuncertaintiesinmodelinputs,observeddata,andeventheQoI mapsthatlinksthese.ApopularframeworkisbasedonthefamousBayes'rule [62]wheretheQoImapiseectivelyreplacedbyastatisticallikelihoodfunction. Morerecently,aframeworkbasedonmeasure-theoreticprincipleshasbeendeveloped andanalyzedfortheformulationandsolutionofaSIPthatdoesnotreplacethe QoImapforthemodel[16,10,12,13].ThesolutionofaSIPformulatedinthe measure-theoreticframeworkisdenedasaprobabilitymeasureonthespaceofmodel inputsthatis consistent withthemodelanddatainthesensethatthepushfoward ofthisprobabilitymeasurethroughtheQoImapexactlymatchestheprobability measuregivenontheQoI.Inthisthesis,weadoptthismeasure-theoreticframework forformulatingaSIPundertwoscenariosandconsidertwoseparatemethodsforthe numericalsolutionoftheSIP. I.2OptimalExperimentalDesign ThecollectionofdataforanobservableQoIcanbecostlyandtimeconsuming. Forexample,exploratorydrillingcanrevealvaluableinformationaboutsubsurface hydrocarbonreservoirs,buteachwellcancostupwardsoftensofmillionsofUS dollars.Insuchsituations,wecanonlyaordtogatherdataforsomelimitednumber ofobservableQoI.However,wemaybeabletodenecertainaspectsofanexperiment thatdenethetypesofobservableQoIforwhichwecollectdata.Forexample,wemay beabletochoosethelocationsofasetofexploratorywellsfromsomepredetermined feasibledomain.ThechoiceofobservableQoIforwhichwemaycollectdatais 2 PAGE 20 denedasanexperimentaldesignED.Thus,anEDdenesaparticularQoImap frommodelinputstomodeloutputs.SincetheQoImapplaysasignicantroleinthe solutionofaSIP,itisevidentthattwodierentEDsmayleadtosignicantlydierent solutionstoaSIP.Forexample,considerasimplecontaminanttransportproblemin whichthediusivitypropertiesofthedomainareuncertain.Weexpectthatsome measurementsinspace-timearemoreusefulthanothersatreducinguncertaintiesin thediusiontensor. Toreducetheexpenseofcollectingdataonlessusefuli.e.,lessinformativeQoI, weconsiderasimulation-basedapproachtoautomaticallyselecttheusefulQoIfrom asetofpossibleEDs.Simulation-basedoptimalexperimentaldesignSB-OED,or sometimesjustOEDlookstodeterminetheoptimalEDbywayofcomputational simulationsofthephysicalprocess.Themainideabeingtoexploitthesensitivity informationinaQoImapcorrespondingtoanEDtodeterminetheoptimalQoIto gatherdataintheeld.ThecriteriaforoptimalityofanEDcanbedenedinvarious waysdependingonthegoalsoftheresearchers.Here,weconsiderbatchdesignover adiscretesetofpossibleEDs.Batchdesign,alsoknownasopen-loopdesign,involves selectingasetofexperimentsconcurrentlysuchthattheoutcomeofanyexperiment doesnoteecttheselectionoftheotherexperiments.Suchanapproachisoften necessarywhenonecannotwaitfortheresultsofoneexperimentbeforestarting another,butislimitedintermsofthenumberofobservationswecanconsider.The pursuitofacomputationallyecientapproachforcouplingtheSB-OEDmethods developedwithinthisthesiswithcontinuousoptimizationtechniquesisanintriguing topicthatweleaveforfuturework. I.3Overview Therestofthisthesisisoutlinedasfollows.InChapterII,weprovideareview ofpastandrecentworkinSB-OED.Forcompleteness,inChapterIII,wereviewthe measure-theoreticframeworkforformulatingaSIPandtwomethodsfornumerical 3 PAGE 21 solutionoftheSIP.ThetwomethodsarereferredtoasaconsistentBayesianapproach andameasure-theoreticapproach.TheconsistentBayesianapproachadoptsthe useofapriorprobabilitydensityinthesolutiontotheSIPwhereasthemeasuretheoreticapproachdoesnotrequireanypriorknowledgeoftheprobabilityonthe modelinputstosolvetheSIP.Weproviderelevantalgorithmsandguresdepicting theimplementationofeachoftheseapproaches. WefollowaBayesianmethodologyforSB-OEDtodevelopaconsistentBayesian approachtoSB-OEDinChapterIV.TheconsistentBayesianapproachtoSIPsproves usefulinreducingthecostoftheiteratedintegraldeningtheutilityofagivenED. InChapterV,wedevelopanentirelyuniqueapproachtoSB-OEDinspiredbythe set-basedapproximationsofthemeasure-theoreticapproach.Thisapproachdeviates fromtheBayesianmethodologyforSB-OEDinthatitquantiestheutilityofa givenEDentirelyintermsofthelocalgeometricpropertiesofthecorrespondingQoI mapanddoesnotrequiresolutiontoanySIP.Thus,thisapproachisindependent ofboththemethodologyemployedtosolvethechosenSIPandthespecicationof aprobabilitymeasureonthespaceofobservableQoI.Consequently,undercertain assumptions,thesetwoSB-OEDapproachesproducesimilarqualitativedescriptions ofaspaceofEDs.Thus,inChapterVI,weprovideanindepthcomparisonbetween thesetwomethodsfocusedspecicallyonahandfulofchallengingSB-OEDscenarios. FollowingChapterVI,wefocusentirelyonthemeasure-theoreticapproachto SB-OEDandlooktoresolveandimproveuponthecurrentstateofthemethod.In ChapterVII,wefocusonthreemajortopics.First,SectionVII.1proposesareformulationoftheSIPsothat,undercertainassumptions,weguaranteethetrue parameter 1 iscontainedwithinaregionofnonzeroprobabilityinthesolutiontothe SIP.Althoughthisapproachisdevelopedtoimprovereliabilityinasolutiontoa givenSIP,itprovidesusefulinsightintoSB-OEDinterests.InSectionVII.2,we 1 Bytrueparameter,wearereferringtoanapplicationinwhichtheSIPfundamentallyisdening aproblemofparameteridenticationunderepistemicuncertainties. 4 PAGE 22 scalarizethemulti-objectiveoptimizationproblem,denedinChapterV,implicitly asafunctionofthesensitivitiesoftheQoImap,thedimensionofthemodelinput space,andthenumberofmodelsolvesavailabletoultimatelyapproximatethesolutiontoachosenSIP.InSectionVII.3,weprovideaboundontheerrorinthe approximationtoaninverseimagewhichoersanalternativedescriptionofthecomputationalcomplexity,i.e.,conditionnumber,ofagivenSIP.InChapterVIII,we illustratethemeasure-theoreticapproachtoSB-OEDonatoyproblemthatyields high-dimensionalparameter,data,anddesignspaces.Weemployagreedyalgorithm toabatethiscurseofdimensionality,thus,weapproximatetheOEDfromacombinatorialenumerationofadiscretizationofthedesignspace.InChapterIX,weprovide concludingremarksanddiscussfutureresearchopportunitiesanddirections. 5 PAGE 23 CHAPTERII LITERATUREREVIEW Itisworthnotingthattheterm optimalexperimentaldesign canmeanagreat manythingsdependingonthecontext.Thetermexperimentaldesignsometimes referredtoasdesignofexperiments,waslikelyrstpopularizedwithinthestatistical community,e.g.,see[42,43,29,41,5]andthereferencesthereinforearlierreferences onthetopic.Inthiscontext,anexperimentaldesignreferstosomeplacementofsampleswithinthepredictorspace.Anoptimalexperimentaldesignreferstoanoptimal placementofsampleswithinthepredictorspace,wheretheoptimalitycriterionmay beapplicationspecic.Theso-calledalphabeticoptimality"criteria[5,40]look tooptimizepropertiesofparameterestimatesdeningamodel,e.g., D -optimality tomaximizethedierentialShannonentropy, A -optimalitytominimizetheaverage varianceofparameterestimates,and G -optimalitytominimizethemaximumvarianceofmodelpredictions.ThesecriteriahavebeendevelopedinbothBayesianand non-Bayesiansettings[5,40,6,25,18,19].Insummary,thiseldofexperimental designresearchfocusesonoptimizingthequalityofa statistical modelrelatingthe predictorstotheresponses. II.1BayesianSimulation-BasedOptimalExperimentalDesign Althoughdevelopingappropriatestatisticalmodelsisofutmostimportancein manypredictivemodelingapplications,weemphasizethatthisdierssubstantially fromtheapproachesproposedinthisthesis.Theapproachesproposedbeloware developedspecicallyforphysics-basedcomputationallyexpensiveandpotentially nonlinearmodelswithuncertainmodelinputparameters.Givenaparameterpredictor,thecomputationalmodelprovidesthevalueoftheQoIresponses.Hence, weneednotapproximateanoptimalmodel. However,uncertaintiesinmodelinputparametersleadtouncertaintiesinmodel predictions.Ifuncertaintiesontheparameterscanbequantiedandreducedusing 6 PAGE 24 experimentaldata,thentheuncertaintiesinmodelpredictionscanalsobequantied andreduced.WeaccomplishthiswiththeformulationandsolutionofaSIP.Given aprobabilitydistributiononsomecollectionofobservableQoI,thesolutionofthe SIPisaprobabilitydistributiononthespaceofmodelinputparameters.TheprobabilitydistributionontheobservableQoImaybedeterminedbycombiningavailable informationontheexperimentaldataandobservationinstrumenterror. DierentcollectionsofQoImayprovideinformationaboutdierentdirections withintheparameterspace.Hence,inthiscontext,anoptimalexperimentaldesign OEDreferstoanoptimalcollectionofQoI,whereoptimalitycriterionareoften derivedfromaninformationtheoreticfoundation[46,65,8].ThedirectionalinformationcontainedinaparticularcollectionofQoIcanbeanalyzedbynumerical simulationsofthemodel.Thus,wemaybeabletorankdierentEDsbysomeperformancemetricbasedentirelyonsimulations.Wefollowrecentliterature[35,36,54] andrefertothistypeofexperimentaldesignas simulation-basedoptimalexperimental design SB-OEDtoemphasizethenecessityofacomputationalmodelrelatingthe uncertainparameterstotheQoI. GivenaprobabilitydistributiononthechosenspaceofobservableQoI,the BayesianparadigmisoftenusedtoformulateandsolveaSIPtodetermineaprobablitydistributiononthespaceofmodelinputparameters[59,62].Thisperspective hasleadtoaBayesianapproachtoSB-OED,whereapproximateBayesiansolutions totheSIPdeterminetheexpectedutilityofagivenED[20,8,63].Theexpected utilityofanEDisoftenwrittenasaniteratedintegraloverboththeparameterspace andthespaceofQoIdeningtheED.Hence,itiscomputationallyexpensiveto approximatemanyusefulformsoftheexpectedutility,e.g.,thecommonlyused expectedinformationgain EIG[8]derivedfromtheKullback-LeiblerKLdivergence [44,64].Therefore,muchoftheliteratureinthiseldisdirectedtowardsreducing computationalcomplexityinapproximatingtheexpectedutilityofanED. 7 PAGE 25 Locallyoptimaltechniquesreducecostbutrequireabestguess"ofthevalueof theunknownparameter[30].Thisapproachisnotsuitablewhentheposteriorparameterdistributionisbroad.In[57],MarkovchainMonteCarloMCMCmethods andLaplacianapproximationsareusedtocomputetheEIG.In[9,21],theauthorsreducecomputationalcostbyapproximatingtheposteriorswithGaussiandistributions. WhentheposteriordierssignicantlyfromaGaussianprole,suchanapproachis notapplicable.In[48,49,50],theauthorsuseLaplaceapproximationstoaccelerate thecomputationsoftheEIG.In[35],theauthorsutilizeageneralizedpolynomial chaossurrogatetoacceleratecomputationsoftheEIGandstochasticoptimization techniquestodetermineanOEDinahigh-dimensionalsetting.In[55,4],theauthors useMCMCandsimulatedannealingmethodstoacceleratethecomputationofthe expectedutility.In[1,2],theauthorsdevelopanOEDframeworkforinverseproblemswithinnitedimensionalparameterspacesresultingfromanunknowneldin thephysics-basedmodel. Allofthemethodsdescribedherehaveprovedvaluableintheappropriatecontext. However,thegeneralformulationofBayesianSB-OEDrequiresthesolutions,or approximatesolutions,ofSIPstodetermineanOED.Thus,theseapproachesrequire rstformulatingaspecictypeofSIPwithinaparticularframework,knowledgeof themethodtobeusedtosolvetheresultingSIP,and,nally,anestimateofthe uncertaintiesontheobservedQoI. II.2ThesisObjectives Theobjectiveoftheapproachesdescribedaboveistoapproximatetheexpected utilityofanEDbasedonapproximatesolutionstotheclassicalBayesianSIP[59, 62].Althoughthisapproachhasprovedimmenselyusefulinthecontextofclassical Bayesianinference,inthisthesis, wefocusspecicallyondevelopingSB-OED approachesinspiredby consistentsolutions totheSIP [16,10,12,13].The consistentBayesianSB-OEDapproach,developedinChapterIV,contributestothe 8 PAGE 26 lineofresearchregardingreductionincomputationalcostbyemployingtherecently developedconsistentBayesianformulationandsolutiontotheSIP[16]toeciently integrateoverthespaceofobserveddensitiestoapproximatetheEIG.Futurework willexplorethecouplingofthismethodwithcontinuousoptimizationtechniquesto determineOEDsinhigh-dimensionaldesignspaces. Themeasure-theoreticapproachtoSB-OED,developedinChapterV,derives acomputationallytractableformulationforSB-OEDinwhich notasingleSIPis solved and,therefore,is independent ofboththeframeworkusedtoformulatethe SIP,themethodusedtosolvetheSIP,andalsotheestimateduncertaintiesonthe observedQoI.Anadditionalbenetisthatthemeasure-theoreticmethodprovidesa quanticationofthecomputationalcomplexityinvolvedinsolvingthenalSIPusing sample-basedapproaches. InChapterVI,wepresentathoroughcomparisonbetweenthesetwoSB-OED approachesandillustratethestrengthsandweaknessesofeach.ChapterVIIcontinuestoextendthemeasure-theoreticapproachtoSB-OEDbybothscalarizinga previouslydenedmulti-objectiveoptimizationproblemandcontributingfurtherto thequanticationofthecomputationalcomplexityofaSIP.InChapterVIII,we illustrateagreedyimplementationofthemeasure-theoreticapproachtoSB-OEDin ahigh-dimensionalsetting. 9 PAGE 27 CHAPTERIII AMEASURE-THEORETICFRAMEWORKFORFORMULATING STOCHASTICINVERSEPROBLEMS Inthischapter,wereviewtworecentlydevelopedmethodsforsolvingaSIP formulatedwithinameasure-theoreticframework.InSectionIII.1,wereviewthe measure-theoreticformulationofaSIPsolvedbybothmethods.Themainresultisa formofthedisintegrationtheoremthatstatestheconditionsunderwhichaprobabilitymeasuresolvestheSIP.Conceptually,thisinvolvesdecomposingtheprobability measureintoamarginalandfamilyofconditionalprobabilitymeasures.Thetwo methodswepresentforsolvingtheSIPgoaboutdeningtheseconditionalprobabilitiesinfundamentallydierentwaysthatleadtodierentapproximationprocedures forthesolutiontotheSIP. InSectionIII.2,wereviewtheconsistentBayesianmethodwhichfollowsthe BayesianphilosophytodeneauniquesolutiontotheSIP.Thesolutionisdened intermsofaprobabilitydensityfunctionasiscommoninBayesianapproaches.In SectionIII.3,wepresentameasure-theoreticmethodwhichexploitsthegeometryof theQoImaptodeneauniquesolutiontotheSIP.There,thesolutionisdenedas theactualprobabilitymeasureontheprescribed -algebra. WhereastheBayesianapproachleadstoalgorithmsforwhichwecaneasilydraw samplesfromthedistributionusingsimpleschemese.g.accept/reject,themeasuretheoreticapproachleadstoanalgorithmthatisbasedonseti.e.eventapproximationresultinginadirectapproximationofprobabilitiesofthesesets.InSectionIII.4, wecomparethesetwomethods.Inparticular,wefocusonthefamilyofconditional probabilitieseachmethodemploystodeneauniquesolutiontotheSIPandthe subsequentnumericalmethodsutilizedtoapproximatesolutionsresultingfromeach method.InSectionIII.5,weillustrateboththe consistency propertyofthesolution toaSIPformulatedwithinameasure-theoreticframeworkandthedependenceof 10 PAGE 28 thissolutiononthechoiceofobservableQoIthatcomposetheQoImap. III.1FormulatingtheStochasticInverseProblemUsingMeasureTheory Here,wesummarizetheworkof[10,12,13].Inordertoprovideacomplete descriptionofthemeasure-theoreticframeworktotheSIP,werstintroducesome necessarynotationandthencarefullydiscussataxonomyofthreeforwardproblems, withincreasinglevelsofuncertainty,alongwiththeirdirectinverseproblems.Fora morethoroughdescriptionandanalysisoftheseproblemsandsolutions,see[10,13]. Let M Y; denoteadeterministicmodelwithsolution Y thatisanimplicit functionofmodelparameters 2 R n .Theset representsthelargestphysically meaningfuldomainofparametervalues,and,forsimplicity,weassumethat is compact.Inpractice,modelersareoftenonlyconcernedwithcomputingarelatively smallsetofQoI, f Q i Y g d i =1 ,whereeach Q i isareal-valuedfunctionaldependenton themodelsolution Y .Since Y isafunctionofparameters ,soaretheQoIandwe write Q i tomakethisdependenceexplicit.GivenasetofQoI,wedenetheQoI map Q := Q 1 ; ;Q d > : !D R d where D := Q denotesthe rangeoftheQoImap. Level1:DeterministicPoint-BasedAnalysis Therstforwardproblemisthesimplestinpredictivesciencewhichistoevaluate themap Q foraxed 2 todeterminethecorrespondingoutputdatum Q = q 2D .Inotherwords,oncetheinputsofamodelarespecied,thesimplestforward problemistosolvethemodelinordertopredicttheoutputdatum.Thereisno uncertainty,saveformodelevaluationerror,inthesepointvalues. Thecorrespondinginverseproblemistodeterminetheparameters 2 that produceaparticularvalueof q 2D .Oftentimes Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 q denesasetofvaluesin eitherduetonon-linearitiesinthemap Q and/orthedimensionof beinggreater thanthedimensionof D .Thus,thesimplestinverseproblemoftenhasuncertaintyas totheparticularvalueof 2 thatproducedaxedoutputdatum.However,there 11 PAGE 29 isnouncertaintyaboutwhichset-valuedinverseproducedthexedoutputdatum. When n =2and d =1,aset-valuedinverse, Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 q ,denesacontourin thatwe arefamiliarwithfromcontourmaps,seetheleftplotinFigureIII.1.Ingeneral,we refertotheset-valuedinversesofthemap Q as generalizedcontours Assuming d n andthattheJacobianof Q hasfullrowrankeverywherein ,thegeneralizedcontoursexistaspiecewise-dened n )]TJ/F20 11.9552 Tf 11.961 0 Td [(d -dimensionalmanifolds in .TheassumptionthattheJacobianof Q hasfullrowrankisapractically importantonesinceweprefertouseQoIthataresensitivetotheparameterssince otherwiseitisdiculttoinferusefulinformationabouttheuncertainparameters. Moreover,wepreferQoItoexhibitthesesensitivitiesinuniquedirectionsin asthis allowsdierentQoItoinformaboutdierentuncertainmodelinputparameters.We formallycharacterizethepropertythattheJacobianof Q hasfullrowrankinthe followingdenition. DenitionIII.1 Thecomponentsofthe d -dimensionalvector-valuedmap Q are geometricallydistinctGD iftheJacobianof Q hasfullrowrankateverypoint in .WhenthecomponentmapsareGD,wesaythatthemap Q isGD. Henceforth,weassumethatanymap Q isGD.Thisimpliesthat d n ,i.e.,the numberofobservableQoIisnolargerthanthenumberofuncertainmodelparameters. Level2:DeterministicSet-BasedAnalysis Thesecondtypeofforwardproblemcanoftenbewrittenmathematicallyas analyzing Q B D givenaset B .Inotherwords,thereisgenerallyuncertainty astotheprecisevalue takesinaset B andsubsequentlythereisuncertaintyas totheparticulardatumtopredictintheset Q B .Thisisacommonlystudied,and wellunderstood,topicinthemathematicalandengineeringsciences.Forexample, itiscommonindynamicalsystemstostudyhowsetsofinitialconditionsevolvein time.Oftentimesweareinterestedindescribingthesizeofthesetsandhowthat 12 PAGE 30 FigureIII.1: leftTheinverseof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q isoftenset-valuedevenwhen 2 speciesaparticular Q 2D .middleTherepresentationof L asatransverse parameterization.rightAprobabilitymeasuredescribedasadensityon D ; B D mapsuniquelytoaprobabilitydensityon L ; B L .Figuresadoptedfrom[13]and [10]. sizechangesfrommodelinputstomodeloutputs.Thus,weassume ; B ; and D ; B D ; D aremeasurespaceswhere B and B D aretheBorel -algebrasinherited fromthemetrictopologieson R n and R d ,respectively,andthemeasures and D arevolumemeasures.Thevolumemeasureson and D allowustoquantitatively assesshowmodelinputschangetomodeloutputsintermsofthethemeasuresof A and Q A ,respectively. Thecorresponding deterministic inversesensitivityanalysisproblemistoanalyze Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A forsome A 2B D .While Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A 2B ,thepracticalcomputationof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A iscomplicatedbythefactthat Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 doesnotmapindividualpointsin D toindividual pointsin .Asdescribedabove,assuming d n and Q isGD, Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 mapsapointin D toageneralizedcontourdescribedasan n )]TJ/F20 11.9552 Tf 10.654 0 Td [(d -dimensionalmanifoldembeddedin .Thus, Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A denesa generalizedcontourevent ,orjustcontourevent,belonging toan inducedcontour -algebra C B wheretheinclusionisoftenproper.To observethattheinclusionisoftenproper,observethatforany B 2B B Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q B ; III.1 butinmanycases B 6 = Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q B evenwhen n = d seeFigureIII.2where n =2 13 PAGE 31 and d =1foranillustration.Inotherwords,solutionstothecorrespondinginverse problemcanbedescribedinthemeasurablespace ; C andthevolumemeasure canbeusedtoprovidequantitativeassessmentsofsolutionstothisinversesensitivity analysisproblem. FigureIII.2: Illustrationofthemapping Q : !D .Notethat Q maps B to aparticularregion Q B 2D ,andwhiletheinverseimageofthisset,givenby Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q B ,contains B ,otherpointsin mayalsomapto Q B .Figureadopted from[16]. Itisusefultoconsidersolutionstothisdeterministicinverseprobleminaspace ofequivalenceclasses.Specically,wemayusethegeneralizedcontourstodene anequivalenceclassrepresentationof wheretwopointsareconsideredequivalent iftheylieonthesamegeneralizedcontour.Welet L denotethespaceofsuch equivalenceclassesandlet L : !L denotetheprojectionmapwhere L = ` 2 L denestheequivalenceclasscorrespondingtoaparticular and )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 L ` = C ` isthe generalizedcontourin correspondingtothepoint ` 2L .Itispossibletoexplicitly represent L in bychoosingaspecicrepresentativeelementfromeachequivalence class.Asdescribedin[10,13],sucharepresentationof L canbeconstructedby piecewise d -dimensionalmanifoldsthat index the n )]TJ/F20 11.9552 Tf 13.231 0 Td [(d -dimensionalgeneralized contours.Werefertoanysuchindexingmanifoldasa transverseparameterization see themiddleplotofFigureIII.1.Givenaparticularindexingmanifoldrepresenting L Q denesabijectionbetween L and D .Themeasurespace L ; B L ; L canbe 14 PAGE 32 denedasaninducedspaceusingthebijection Q and D ; B D ; D .Then,solutions tothedeterministicinverseproblemcanbedescribedandanalyzedasmeasurable setsofpointsin L insteadofmeasurablegeneralizedcontoureventsin Level3:StochasticAnalysis Wenowassumethatthereisuncertainty,describedintermsofprobabilities,as towhichset A 2B themodelparameters belong,andthegoalistoanalyzethe probabilitiesofsetsin B D .Inthelanguageofprobabilitytheory,measurablesetsare referredtoasevents.Thus,weassumethataprobabilitymeasure P isgivenon ; B describinguncertaintyintheeventsforwhichparametersmaybelong,andthe goalistodetermine P D on D ; B D .Werefertothisasthe stochastic forwardproblem. When P resp., P D isabsolutelycontinuouswithrespecttothevolumemeasure resp., D ,thecorrespondingRadon-Nikodymderivativei.e.,theprobability densityfunction resp., D isusuallygiveninplaceoftheprobabilitymeasure. Weassumethisisthecasesothatwecanrefertoprobabilitiesofeventsintermsof themorecommonrepresentationusingintegralsanddensityfunctions,e.g., P A = Z A d ;A 2B : III.2 Solutiontothisstochasticforwardproblemisgivenbytheinducedpushforward probabilitymeasure P D denedforany A 2B D by P D A = Z A D d D = Z Q )]TJ/F19 5.9776 Tf 5.757 0 Td [(1 A d = P Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A : III.3 Thisisafamiliarprobleminuncertaintyquantication.Theapproximatesolution canbeobtainedbyaclassicMonteCarlomethod.Assumingthat Q isGD,the pushforwardvolumemeasure,bythemap Q ,isabsolutelycontinuouswithrespect totheusualLebesguemeasureon R d [13,16].Therefore,unlessotherwisenoted,we assume and D aretheusualLebesguemeasureson R n and R d ,respectively. 15 PAGE 33 ThecorrespondingSIPassumesthatwearegivenanobservedprobabilitymeasure, P obs D ,on D ; B D ,whichcanbedescribedintermsofanobservedprobability density, obs D ,i.e., obs D istheRadon-Nikodymderivativeof P obs D withrespectto D TheSIPisthendenedasdeterminingaprobabilitymeasure, P ,describedasa probabilitydensity, ,suchthatthepushforwardmeasureof P agreeswith P obs D on D ; B D .Weuse P Q P D todenotethepushforwardof P through Q ,i.e., P Q P D A = P Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A ; forall A 2B D .Usingthisnotation,aconsistentsolutiontotheSIPisdened formallyasfollows: DenitionIII.2Consistency Givenaprobabilitymeasure P obs D describedasa density obs D withrespectto D on D ; B D ,theSIPseeksaprobabilitymeasure P describedasaprobabilitydensity withrespectto on ; B suchthatthe subsequentpushforwardmeasureinducedbythemap Q satises P Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 A = P Q P D A = P obs D A ; III.4 forany A 2B D .Werefertoanyprobabilitymeasure P thatsatises III.4 asa consistent solutiontotheSIP. TodeterminesuchasolutiontotheSIP,werstconsidertheSIPposedon L ; B L Since Q denesabijectionbetween L and D P obs D denesaunique P L on L ; B L seetheright-handplotinFigureIII.1and[13].Wecanthenusetheprojection map L toprovethefollowingtheoremaswasdonein[13]. TheoremIII.3 TheSIPhasauniquesolutionon ; C However,thegoalistodeneaprobabilitymeasure P onthemeasurablespace 16 PAGE 34 ; B notonaspaceinvolvingcontoureventsthatarecomplicatedtodescribe.This requiresanapplicationoftheDisintegrationTheorem,whichallowsfortherigorous descriptionofconditionalprobabilitymeasuresdenedonsetsofzero -measure [24,13,32]. TheoremIII.4TheDisintegrationTheorem Assume Q : !D is B measurable, P isaprobabilitymeasureon ; B and P obs D isthepushforwardmeasureof P on D ; B D .Thereexistsa P obs D -a.e.uniquelydenedfamilyofconditional probabilitymeasures f P q g q 2D on ; B suchthatforany A 2B P q A = P q )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(A Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 q ; so P q n Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 q =0 ,andthereexiststhefollowingdisintegrationof P P A = Z D P q A dP obs D q = Z D Z A Q )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 q dP q dP obs D q ; III.5 for A 2B .Equivalently,becausethereisabijectionbetween L and D ,wemaywrite thisiteratedintegralintermsof L P A = Z L Z A )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 L ` dP ` dP L ` ; III.6 for A 2B where P L istheuniqueprobabilitymeasureinducedon L by P obs D Forproofsrelatedtotheabovedisintegrationtheoremsee[13,16].Itfollowsfrom thedisintegrationtheoremthatifaprobabilitymeasure P obs D on D ; B D isgiven, thenthespecicationofafamilyofconditionalprobabilitymeasures f P q g q 2D or, equivalently, f P ` g ` 2L on ; B canbeusedtodeneaspecicprobabiltymeasure P usingEq.III.5or,equivalently,Eq.III.6thatisconsistentinthesenseof DenitionIII.2.However,aconsistentsolutionmaynotbeunique,i.e.,theremaybe 17 PAGE 35 multiplefamiliesofconditionalprobabilitymeasureson ; B thatdenemultiple consistentprobabilitymeasures.Auniquesolutionmaybeobtainedbyimposing additionalconstraintsorstructureontheSIPthatleadstoauniquespecicationof thefamilyofconditionalprobabilitymeasureson ; B .Themethodsexploredin SectionIII.2andSectionIII.3describehowwemayuseeitherassumedorexisting measure-theoreticinformationtodenethefamiliesofconditionalprobabilities. III.2AConsistentBayesianSolutiontotheSIP Here,wesummarizetheworkof[16]. III.2.1MathematicalFormulation FollowingtheBayesianphilosophy[62],weintroduceanite prior probability measure P prior describedasaprobabilitydensity prior on ; B withrespectto Thepriorprobabilitymeasureencapsulatesanyexistingknowledgeabouttheuncertainparametersandservestodenethenecessaryfamilyofconditionalprobability measureson ; B toobtainauniquesolutiontotheSIP. Thepriorinformationcancomefromvarioussources.Forexample,engineering knowledgeofthefeasibleparameterdomainmayinformboundsonthesupportofthe priordensityandpastexperimentalresultsmayinformanappropriatedistribution fortheprioroverthisfeasibleregion.Intheabsenceofaphysicallymeaningful priormeasure,oranypreviousresultsfromwhichtobuildone,itmaybepossibleto developapriorusingacoarsesolutiontotheSIPusingthemeasure-theoreticmethod whichdoes not requireapriorprobabilitymeasure,seeSectionIII.3.The quality of apriorprobabilitymeasureandsuitablemethodsfordevelopmentofsuchaprioris acurrenttopicofscienticresearchanddebate,andthistopicisbeyondthescope ofthisexpositionoftheconsistentBayesiansolutiontotheSIP. Assumingthat Q isatleastmeasurable,thenthepriorprobabilitymeasureon ; B P prior ,andthemap, Q ,induceapushforwardmeasure P Q prior D on D ,which 18 PAGE 36 isdenedforall A 2B D P Q prior D A = P prior Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 A : III.7 Recallfromclassicalprobabilitytheorythatif P isaprobabilitymeasure, B isan eventofinterest,and A isanarbitraryevent,then P B = P B j A P A + P B j A c P A c : Furthermore,if B A ,thenthisreducesto P B = P B j A P A : III.8 Inthiscontext,theterm P B representstheconsistentprobabilitymeasureofthe event B thatweseekandwelet A := Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q B ,whichensuresthat B A .We aremotivatedtouse P A = P obs D Q A inEq.III.8sincethedenitionofthe pushforwardmeasureimpliesthattheconsistencyconditionisthensatisedbecause Q A = Q B .TheremainingterminEq.III.8isdeterminedusingtheprior measure P prior ,theclassicalBayes'theoremforevents,andthefactthat B A P B j A = P prior A j B P prior B P prior A = P prior B P prior A : Itfollowsfromthedenitionof P Q prior D that P prior A = P Q prior D Q B .Thus,we utilizethefollowingformalexpressionfor P P B := 8 > > < > > : P prior B P obs D Q B P Q prior D Q B ; if P prior B > 0 ; 0 ; otherwise ; III.9 Thefollowingpropositionandensuingtheoremhighlightthemajorstepsneededto 19 PAGE 37 provetheexistenceanduniquenessofaconsistentBayesiansolutiontotheSIPgiven byaprobablitymeasureon ; B PropositionIII.5 TheprobabilitymeasuregivenbyEq. III.9 denesaprobability measureon ; C Wenotethat,althoughPropositionIII.5saysthat P ,asdenedabove,denesa probabilitymeasureonthecontour -algebraon ; C ,itisnotnecessarilytrue thatthis P abovedenesaprobabilitymeasureon ; B ,see[16]foradiscussion onthistopic.Weuse P prior todenetheconditionalprobabilitydensitiesasRadonNikodymderivativeswithrespecttothedisintegrationofthevolumemeasure ;q TheoremIII.6utilizesthedisintegrationtheoremtoconstructaconsistentprobability measureon ; B TheoremIII.6 Theprobabilitymeasure P on ; B denedby P A = Z D Z A Q )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 q dP prior q dP obs D q ; 8 A 2B ; III.10 where dP prior q = prior Q prior D Q d ;q ; 8 q 2D isaconsistentsolutiontotheSIPinthesenseofDenitionIII.2. Wenotethat P prior isusedonlytodenethefamilyofconditionalprobabilitymeasures P prior q q 2D on ; B ,neededtoobtainauniquesolutiontotheSIP,interms ofthepriorprobabilitydensityandthemap Q .Itfollowsthat P inEquationIII.10 isgivenby P A = Z D Z A Q )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 q prior Q prior D Q d ;q dP obs D q III.11 forall A 2B .Furthermore,since dP obs D q = obs D d D q ,wecansubstitutethisinto 20 PAGE 38 Eq.III.11toobtainthedensity = prior obs D Q Q prior D Q ; 2 ; III.12 which,whenintegratedintermsoftheaboveiteratedintegral,inducesaprobability measure P on ; B thatisaconsistentsolutiontotheSIP. III.2.2NumericalApproximationofSolutions Approximating usingtheconsistentBayesianapproachrequiresanapproximationofthepushforwardofthepriorprobabilitymeasureonthemodelparameters, whichisfundamentallyaforwardpropagationofuncertainty.Weconsiderthemost basicofmethods,namelyMonteCarlosampling,tosamplefromtheprior.Weevaluatethecomputationalmodelforeachofthesamplesfromtheprioranduseastandard kerneldensityestimatorKDEtoapproximatethepushforwardoftheprior.Given theapproximationofthepushforwardoftheprior,wecanevaluate atanypoint 2 ifwehave Q .Thus,wecanconstructanapproximationof Wesummarizethestepsrequiredtonumericallyapproximatethepushforwardof thepriorinAlgorithm1,andsummarizethestepsrequiredtoapproximatethesolutiontotheSIPinAlgorithm2.Wenotethat,thetwoprominentcostsinproducing anumericalapproximationto are:theevaluationofthemodel,andthe constructionofanapproximationto Q prior D Algorithm1: ComputingthePushforwardDensityInducedbythePriorand theModel 1.Givenasetofsamplesfromthepriordensity: i i =1 ;:::;N ; 2.Evaluatethemodelateach i andcomputetheQoI: q i = Q i ; 3.UsethesetofQoIandastandardtechnique,suchasKDE,toestimate Q prior D q III.3AMeasure-TheoreticSolutiontotheSIP 21 PAGE 39 Algorithm2: NumericalApproximationoftheConsistentBayesianSolution 1.Givenapriordensityon prior ; 2.Givenanobserveddensityon D obs D ; 3.Giventhepushforwardofthepriordensityon D Q prior D ; 4.Givenasetofsamplesin : i i =1 ;:::;N ; 5.Evaluatethemodelateach i ; 6.Computethevalueof ateach i : i = prior i obs D Q i Q prior D Q i : III.13 7.Use i toconstructanapproximationto Wefollow[13,32]andadoptwhatisreferredtoasthe standardAnsatz determined bythedisintegrationofthevolumemeasure tocomputeprobabilitiesofevents insideofacontourevent,i.e.,todenethefamilyofprobabilitymeasureson ; B requiredtoobtainauniquesolutiontotheSIP.ThestandardAnsatzisgivenby P ` B = C ` B C ` C ` ; 8 B 2B C ` ; forall ` 2L where C ` isthedisintegratedvolumemeasureonthegeneralizedcontour C ` .Notethat,thisisthesameas ;q discussedabovewhere C ` = Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 q .Also, B C ` denotesthestandardBorel -algebraonthe C ` ThestandardAnsatzcanbeusedaslongas C ` isnitefora.e. ` 2L ,e.g., ashappenswhen R d iscompact, isthestandardLebesguemeasure,and Q iscontinuouslydierentiable.Assumingsuchconditionsholdtoapplythestandard Ansatz,theapproximationmethodandresultingnon-intrusivecomputationalalgorithmcanbeeasilymodiedforalmostanyotherAnsatz.See[13]formoredetails andtheoryregardinggeneralchoicesoftheAnsatz.CombininganAnsatzwiththe 22 PAGE 40 disintegrationtheoremprovesthefollowingtheorem. TheoremIII.7 Theprobabilitymeasure P on ; B denedby P A = Z L Z A )]TJ/F19 5.9776 Tf 5.756 0 Td [(1 L ` dP ` dP L ` ; III.14 where P ` B = C ` B C ` C ` ; 8 B 2B C ` ; 8 ` 2L isaconsistentsolutiontotheSIPinthesenseofDenitionIII.2. ThestandardAnsatzresultsinaprobabilitymeasure P thatinheritskeygeometricfeaturesfromthegeneralizedcontourmap.AnyotherchoiceofAnsatzimposes someothergeometricconstraintsonthesolutiontotheinverseproblemthatarenot comingfromthemap Q .IfwechoosenottouseanyAnsatz,thenwecanalways solvetheSIPon ; C ,i.e.,wecanalwayscomputetheuniqueprobabilitymeasure whenrestrictedtocontourevents. III.3.1NumericalApproximationofSolutions Fundamentaltoapproximatingsolutions P totheSIPistheapproximationof eventsinthevarious -algebras, B D C and B .Since C B ,wecansimultaneouslyapproximateeventsinbothofthese -algebrasusingthesamesetofevents partitioning .Let fV i g N i =1 denotesuchapartitionof where V i 2B foreach i Assumethatwearegivenacollectionofsets f D k g M k =1 B D partitioning D ,seeFigureIII.3.Thebasicalgorithmicprocedureforapproximating P V i foreach i isto determinewhichofthe fV i g N i =1 approximate Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 D k ,andthentoapplytheAnsatz onthisapproximationofthecontoureventwhichhasknownprobability P D D k ,see FigureIII.4.Letting p ;i denotetheapproximationof P V i ,wecanthendenean approximationto P as P A P ;N A = N X i =1 p ;i A V i ;A 2B ; 23 PAGE 41 Wesummarizethisbasicprocedureforapproximating P inAlgorithm3. Algorithm3: NumericalApproximationoftheMeasure-TheoreticSolution 1.Let fV i g N i =1 B partition ,oftendenedbyasetofsamples i N i =1 2.Compute V i = V i for i =1 ;::;N 3.Determineanominalvalue q i forthemap Q on V i for i =1 ;::;N ,often q i = Q i 4.Chooseapartitioningof D f D k g M k =1 D 5.Compute P obs D D k for k =1 ;:::;M 6.Let C k = f i j q i 2 D k g for k =1 ;:::;M 7.Let O i = f k j q i 2 D k g for i =1 ;::;N 8.Set P V i = V i = P j 2C O i V j P obs D D O i for i =1 ;::;N InAlgorithm3, C k isusedtodeterminewhichsetsfrom fV i g N i =1 approximatethe contourevent Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 D k foreach k .Similarly, O i isusedtodeterminewhichcontour event Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 D k isassociatedto V i foreach i .TheAnsatzisappliedinthenalstep wheretheprobabilityofeach V i isdeterminedbytheprobability P obs D D O i multiplied bytheratioofthevolumeof V i tothevolumeoftheapproximatecontoureventof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 D O i Wenotethat,theprominentcostsinproducinganumericalapproximationto P are:theevaluationofthemodel,andthenearestneighborsearchesemployed inthevariousstepsofAlgorithm3,sometimesreferredtoasbinning.Dependingon theassumptionwemake,wemayreducethiscostincertainways,i.e.,givenaset ofi.i.d.randomsamplesdening fV i g N i =1 ,wemaymaketheMCassumptionthat V i = =N foreach i III.4ComparisonofMethods Abriefdiscussiononthedierenceinhoweachoftheabovemethodsdenes auniquesolutiontotheSIPiswarranted.Asnoted,thedenitionofafamilyof 24 PAGE 42 D FigureIII.3: WesummarizethestepsinAlgorithm3.toprowWedepictSteps 1-3.Wepartition with fV i g N i =1 thatareimplicitlydenedbythesetofsamples i N i =1 ,wemaketheMCassumptiontodeterminethevolumeofeach V i .Thenwe determinethenominalvalue q i = Q i .Here, N =2000.bottomrowWedepict Steps4-5.Wepartition D witharegulargridandprescribetheobservedprobablity measuretobeconstantoverthedarkenedsetof V i .Here M =100. 25 PAGE 43 D FigureIII.4: SummaryofAlgorithm3continued.toprowWedepictSteps5-6. Foreach k ,wedeterminewhichsetof i mapinto D k .Inred,weseetheunion ofthe V i correspondingtothe i thatmapinto D 1 seeninredin D .Weseethe samefor D 2 inblue.bottomrowWedepictStep8.Wecomputetheapproximate solutiontotheSIPwhereweproportionouttheprobabilityofeach D k determined bytheobservedmeasuretoitscorresponding V i .Darkercolorscorrespondtohigher probability. 26 PAGE 44 conditionalprobabilitymeasureson ; B denesauniquesolutiontotheSIP. TheconsistentBayesianmethodutilizesthedataspace D andpriorinformationon theparameterspacetodenethisfamilyofconditionalprobabilitymeasureswhich areultimatelydescribedintermsoftheircorrespondingprobabilitydensities,see Eq.III.11.Thismethodrequiresthepushforwardofthepriordensityandresults inanapproximationtothevalueof atanypoint 2 thatwehave Q .We maythenusethe i N i =1 toconstructanapproximationto .Alternatively, wemaygeneratesamplesfrom usinganaccept/rejectalgorithmwithoutactually constructing Themeasure-theoreticmethodutilizesthetransverseparameterization L ,implicitly,anddenesthefamilyofconditionalprobabilitymeasureson ; B basedon theunderlyingvolumemeasure .Thisdoes not requireanypriorinformationon theinputparameters.Thismethodrequiresvariousnearestneighborsearchesand resultsinanapproximationto P onadiscretizedparameterspace.Wemayuse thisapproximationto P toconstructanapproximationto andgeneratesamples from usinganaccept/rejectalgorithm.However,thesamplesusedtodiscretize andapproximate P neednotcomefromanunderlyingprobabilitymeasureand canbeplacedoptimallyusingonlyinformationaboutthemap Q anddataspace discretization,see[15]. III.5NumericalExample:Consistency Here,weconsideranaccessiblenonlinearsystemofequationstodisplaythe consistency ofthesolutiontotheSIP.Wenotethat,givenauniformprior,theconsistent Bayesiansolutionisidenticaltothemeasure-theoreticsolutionunderthestandard Ansatz[16].Inotherwords,thetwomethodsproducethesameconsistentsolutionto theSIPunderassumptionofauniformprior.Therefore,inthisexample,weutilize themeasure-theoreticalgorithmi.e.,Algorithm3,underthestandardAnsatz,for approximatingsolutionstotheSIP.ThefollowingconsistentsolutionstotheSIPare 27 PAGE 45 computedusingtheButler,Estep,TavenerBETPythonpackage,whichisdesigned tosolvemeasure-theoreticstochasticinverseandforwardproblems[32]. WeformulateandsolvethreeSIPsinordertobothillustratetheconsistency propertyofthesolutiontoagivenSIPandtomotivatetheremainingchapters. Considerthefollowing2-componentnonlinearsystemofequationsintroducedin[10]: 1 x 2 1 + x 2 2 =1 x 2 1 )]TJ/F20 11.9552 Tf 11.955 0 Td [( 2 x 2 2 =1 Theparameterrangesaregivenby 1 2 [0 : 79 ; 0 : 99]and 2 2 [1 )]TJ/F15 11.9552 Tf 13.182 0 Td [(4 : 5 p 0 : 1 ; 1+ 4 : 5 p 0 : 1]whicharechosenasin[10]toinduceaninterestingvariationinthesolution )]TJ/F20 11.9552 Tf 5.479 -9.683 Td [(x 1 ;x 2 .WeconsiderthreeQoImapsdenedbythesolutiontothissystem, Q = x 1 ; Q = x 2 ; Q = )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(x 1 ;x 2 : ThecorrespondingsolutionstotheSIPsaredenoted,respectively,as ; ; and Consider Q andassumethattheobserveddensityon Q isatruncatednormaldistributionwithmean1.015andstandarddeviationof0.007,seeFigureIII.5 bottom.Wegenerate1 E +5independentidenticallydistributedi.i.d.samples in ,usetheBETPythonpackagetoconstructanapproximationto P ,anda simpleaccept/rejectalgorithmtogenerateasetofsamplesfromtheresulting Wepropagatethissetofsamplesfrom throughthemodelandapproximatethe resultingpushforwarddensityusingaKDE,seeFigureIII.5.Thebottomimagein FigureIII.5displaysthe consistency propertyofthissolutiontotheSIP,i.e.,the 28 PAGE 46 pushforwardof matchestheobserveddensity. FigureIII.5: topleftApproximationof obtainedusing obs D on Q ,top rightasetofsamplesfrom ,bottomandacomparisonoftheobserveddensity obs D on Q withthepushforwardof Consider Q andassumethattheobserveddensityon Q isatruncatednormal distributionwithmean0.3andstandarddeviationof0.035.Weapproximate P usingthesame1 E +5samples,generateasetofsamplesfromtheresulting and propagatethesesamplesthroughthemodeltoapproximatethepushforwardof seeFigureIII.6.Again,weobservetheconsistencyof inthatthepushforwardof matchestheobserveddensity. Next,weconsiderusingthevector-valuedQoImap Q .Wetaketheobserved jointdensityon Q tobetheproductofthemarginaldensitieson Q and 29 PAGE 47 FigureIII.6: topleftApproximationof obtainedusing obs D on Q ,top rightasetofsamplesfrom ,bottomandacomparisonoftheobserveddensity obs D on Q withthepushforwardof 30 PAGE 48 Q asspeciedabove.Again,weapproximate P andgenerateasetofsamples fromtheresulting usingthesame1 E +5samples,seeFigureIII.7.Withthe informationfromboth x 1 and x 2 ,weseeasubstantialdecreaseinthesupport of .Intuitively,withtheassumptionsabove,weexpectthat supp supp supp : FigureIII.7: leftTheapproximationof usingthejointdensity obs D on Q rightasetofsamplesfrom Wepropagatethesetofsamplesfrom throughthemodelandcalculatethe meansandstandarddeviationsoftheresultingsetofsamplestoconstructthepushforwardof .InFigureIII.8,weshowthepushforwardof ,theobserveddensity, andtheerrorbetweenthetwoscaledbythemaximumoftheobserveddensitytodisplayarelativeerror. Inthescenarioinwhichwegatherdataleadingtoanobserveddensityonboth Q and Q ,comparedtothescenarioswhereweonlygatherdataon Q or Q butnotboth,thereisaclearreductionofuncertaintiesasisevidencedbythereduced supportin anditsoverallstructure.However,supposewecouldonlygatherdata ononeoftheseQoIinanexperimental/eldsetting.Choosingeither Q or Q 31 PAGE 49 FigureIII.8: topleftThepushforwardof ,toprighttheexactobserved density,bottomthedierencebetweenthepushforwadof andtheobserved densityscaledbythemaximumoftheobserveddensity. 32 PAGE 50 denesaseparateED.ItisnotimmediatelyclearfromthegureswhichEDismore useful. Intheremainingchapters,weseektoquantifythe utility ofanEDunderconsiderationinordertodeterminetheoptimalQoImaptobeusedinformulatingand solvingaSIP.InChapterIV,wequantifytheexpectedutilityofagivenEDusingthe KLdivergence.InChapterV,wequantifytheexpectedutilityofanEDintermsof localgeometricpropertiesofthecorrespondingQoImapthatdescribetheexpected precisionandaccuracyoftheresultingsolutiontoaSIP.InChapterVI,weprovide anindepthcomparisonbetweenthesetwoSB-OEDmethodsfocusedspecicallyona handfulofchallengingSB-OEDscenarios.InChapterVII,wecontinuetoextendthe measure-theoreticapproachtoSB-OEDbycontributingfurthertothequantication ofthecomputationalcomplexityofaSIP.InChapterVIII,weillustrateagreedy implementationofthemeasure-theoreticapproachtoSB-OEDinahigh-dimensional setting. 33 PAGE 51 CHAPTERIV CONSISTENTBAYESIANOPTIMALEXPERIMENTALDESIGN Inthischapter,wedevelopaSB-OEDapproachbasedontheconsistentBayesian methodforsolvingSIPs.WefollowtheBayesianmethodologyforSB-OEDdiscussed inChapterIIand[35,36,9].ThismethodologyquantiestheutilityofagivenED denedbyaQoImap Q basedonasetofdensitiescorrespondingtosolutionsof SIPsfor Q which,inturn,arebasedonthetypesofdatathatweexpecttoobserve intheeldforthegiven Q .SincetheconsistentBayesianmethodforsolvingSIPs hasdierentassumptionsfromtheclassicalBayesianmethod,wemodifytheclassical BayesianSB-OEDmethodologyasneeded. InSectionIV.1,weconsidertheinformationcontentofagivenEDanddenethe OEDproblem.InSectionIV.2,weaddresstheissueofinfeasibledataanddiscuss computationalconsiderations.WeprovidenumericalexamplesinSectionIV.3to displaytheecacyofthismethodontwotoyproblemsforwhichwehavesignicant intuitionregardingthelocationoftheOED. IV.1TheInformationContentofanExperimentalDesign Conceptually,anEDisconsideredinformativeiftheresultingsolutiontotheSIP issignicantlydierentfromthepriorprobabilitymeasure.Toquantifythe informationgain ofanEDweusetheKLdivergence[64],whichisameasureofthedierence betweentwoprobabilitydensities.WhiletheKLdivergenceisbynomeanstheonly waytocomparetwoprobabilitydensities,itdoesprovideareasonablemeasureofthe informationgainedinthesenseofShannoninformation[22]andiscommonlyusedin BayesianOED[35,9].Inthissection,wediscusscomputationoftheKLdivergence anddeneanOEDformulationbasedupontheexpectedinformationgainovera speciedspaceofprobableobserveddensities. IV.1.1InformationGain:Kullback-LeiblerDivergence 34 PAGE 52 Supposewearegivenadescriptionoftheuncertaintyontheobserveddatafor aQoImap Q intermsofaprobabilitymeasure P obs D thatcanberepresentedasa density obs D .Inotherwords, P obs D isabsolutelycontinuouswithrespectto D and obs D istheRadon-Nikodymderivativeof P obs D .AspresentedinChapterIII,givenprior informationabouttheparametersdescribedasaprobabilitymeasure P prior ,theSIP hasauniquesolutiongivenbyaprobabilitymeasure P thatisabsolutelycontinuous withrespectto ,andwelet denotetheprobabilitydensity.TheKLdivergence, i.e.,informationgain,of from prior ,denoted KL Q toemphasizetherole Q plays indetermining ,isgivenby KL Q prior : := Z log prior d : IV.1 Since prior isxedforagivenED, KL Q issimplyafunctionof .However,from Eq.III.12,weseethat issimplyafunctionof obs D .Therefore,tomakethis dependenceexplicit,wewrite KL Q asafunctionoftheobserveddensity, KL Q obs D := KL Q prior : : IV.2 Theobservationthat KL Q isafunctionofonly obs D iscriticaltodeningtheexpectedinformationgainonawell-denedspaceofobserveddensities,asdonein SectionIV.1.2. ExampleIV.1InformationGain Considerthefollowing2-componentnonlinearsystemofequationswithtwoparametersdiscussedinSectionIII.5, 1 x 2 1 + x 2 2 =1 ; x 2 1 )]TJ/F20 11.9552 Tf 11.956 0 Td [( 2 x 2 2 =1 : 35 PAGE 53 Recallthattheparameterrangesaregivenby 1 2 [0 : 79 ; 0 : 99] and 2 2 [1 )]TJ/F15 11.9552 Tf -422.702 -23.98 Td [(4 : 5 p 0 : 1 ; 1+4 : 5 p 0 : 1] .Asbefore,weconsiderthreeQoImapsdenedbythesolutiontothissystem, Q = x 1 ; Q = x 2 ; Q = )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(x 1 ;x 2 : ThecorrespondingsolutionstotheSIPsaredenoted,respectively,as ; ; Consider Q andassumetheobserveddensityon Q isatruncatednormal distributionwithmean0.3andstandarddeviationof0.01,seeFigureIV.1.Wegenerate40,000samplesfromtheuniformpriorandconstructanapproximationtothe pushforwardofthepriorwithAlgorithm1,seeFigureIV.1right.Then,weuse Algorithm2toconstructanapproximationto usingthesame40,000samples,see FigureIV.1left.Noticethesupportof liesinarelativelylargeregionoftheparameterspace.Theinformationgainfrom ,approximatedusingEquation IV.1 is KL Q obs D 0 : 466 Next,weconsider associatedwithsolvingtheSIPusing Q .Weassume theobserveddensityon Q isatruncatednormaldistributionwithmean1.015and standarddeviationof0.01.Weapproximatethepushforwardofthepriorand withAlgorithm1andAlgorithm2,usingthesame40,000samples,seeFigureIV.2. Althoughtheobserveddensitieson Q and Q havethesamestandarddeviation,clearly and aredierent.Visually, hasamuchlargerregionof supportwithintheparameterspacecomparedto .Thisisquantiedwiththeinformationgainfrom KL Q obs D 2 : 015 .Giventhesetwomaps, Q and Q ,andthespeciedobserveddensitiesoneachofthesedataspaces,theexperiment leadingto Q andthecorresponding obs D ismoreinformativethantheexperiment 36 PAGE 54 FigureIV.1: leftApproximationof obtainedusing obs D on Q ,whichgives KL Q obs D 0 : 466.right obs D on Q andthepushforwardoftheprior. FigureIV.2: leftApproximationof obtainedusing Q and obs D on Q whichgives KL Q obs D 2 : 015.right obs D on Q andthepushforwardofthe prior. 37 PAGE 55 leadingto Q andthecorresponding obs D Next,weconsiderusingthevector-valuedQoImap Q .Wetaketheobserved jointdensityon Q tobetheproductofthemarginaldensitieson Q and Q asspeciedabove.Again,weapproximatethepushforwardofthepriorand using thesame40,000samples,seeFigureIV.3.Withthejointinformation,weseea substantialdecreaseinthesupportof .Thisisquantiedintheinformationgain ofthis KL Q obs D 2 : 98 .Thus,weseethelargestinformationgainbyusing Q andthejointdensityobservedon Q .However,inpractice,wemaybelimited tospecifyasingleQoIforwhichwecancollectdatatoconstructanobserveddensity. Basedonthecomputedinformationgains,wemayconcludethat Q ismore informativeoftheparametersthan Q .However,duetononlinearitiesofthemaps, itisnotnecessarilytruethat Q ismoreinformativethan Q ifdierentobserved densitiesarespecied.Sincewedonotknowtheobserveddensitiespriortocollecting actualdata,wearelefttodeterminewhichoftheseQoIwe expect toproducethe mostinformative IV.1.2ExpectedInformationGain OEDmustselectanED before experimentaldatabecomesavailable.Inthe absenceofdata,weusethecomputationalmodeltoquantifythe expected information gainEIGofagivenEDdenedbyaQoImap Q .Given Q and D = Q ,let O D denotethespaceofallprobabilitydensityfunctionsover D .DeningtheEIG over O D iscomputationallyimpossiblesince O D containsuncountablyinnitelymany densities.Moreover, O D includesdensitiesthatareunlikelytobeobservedinpractice foragiven Q .Therefore,werestrict O D tobeaspacemorerepresentativeofdensities thatmaybeobservedinpracticeforaparticularEDdenedby Q Withnoexperimentaldataavailabletospecifyanobserveddensityon D fora givenQoImap Q ,weassumetheobserveddensityisaGaussianwithmeanbelonging to D andadiagonal m m covariancematrix,i.e.,weassumetheuncertaintiesin 38 PAGE 56 FigureIV.3: topleftTheapproximationofthepushforwardoftheprior.top rightTheexact obs D on Q .bottomTheapproximationof using obs D on Q whichgives KL Q obs D 2 : 98. 39 PAGE 57 eachQoIareuncorrelated.ForagivenQoImap Q ,welet O Q denotethespaceofall densitiesofthisprolewithmeanbelongingto D = Q O Q = D 2O D : D N ; ; 2D : IV.3 Thus, D isassumedtobeaGaussiandensityfunctionwithmean 2D anddiagonal covariancematrix Wemayrestrict O Q inotherwaysaswell.Forexample,wemayexpectthe uncertaintyineachQoItobedescribedbyauniformdensityofagivenwidth,in whichcasewewoulddenetherestrictionon O Q accordingly.Thischoiceofprole fortheobserveddensitieswithin O Q islargelydependentontheapplicationand specicationof Q .Theonlylimitationontheobserveddensityspacecomesfromthe pushforwardmeasureonthedataspace, P Q prior D ,asdescribedbelow. Therestrictionofpossible obs D tothisspecicspaceofdensitiesallowsustoindex eachdensitybyaunique q 2D .Basedonourpriorknowledgeoftheparametersand thesensitivitiesofthemap Q ,themodelinformsusthatsomedataaremorelikely tobeobservedthanotherdata,thisisseenintheplotof Q prior D inFigureIV.3. Therefore,weaverageover D withrespecttothepushforwardoftheprior.This utilizesboththepriorknowledgeoftheparametersandthesensitivityinformation providedbythemodel.WedenetheEIG,denoted E KL Q ,as E KL Q := Z D KL Q q Q prior D q d D : IV.4 FromEq.IV.1, KL Q itselfisdenedintermsofanintegral.Theexpandedform for E KL Q isthenaniteratedintegral, E KL Q = Z D Z ; q log ; q prior d dP Q prior D ; IV.5 40 PAGE 58 wherewemakeexplicitthat isafunctionoftheobserveddensity,and,bythe restrictionofthespaceofobserveddensitiesinEq.IV.3,isthereforeafunctionof q 2D .WeutilizeMonteCarlosamplingtoapproximatetheintegralinEq.IV.4as describedinAlgorithm4. Algorithm4: ApproximatingtheEIGofanED 1.Givenasetofsamplesfrom : i i =1 ;:::;N ; 2.Givenasetofsamplesfromthepushforwardoftheprior: q j j =1 ;:::;M ; 3.Giventhevaluesof ; q j at: i i =1 ;:::;N ; 4.For j =1 ;:::;M approximate KL Q q j : KL Q q j N N X i =1 i ; q j log i ; q j prior i 5.Approximate E KL Q : E KL Q 1 M M X j =1 KL Q q j RemarkIV.1 InStep1ofAlgorithm4,weassumewearegivenasetofsamples fromthevolumemeasure ontheparameterspace.Assumingthisvolumemeasure istheLebesguemesure,thengivenauniformprior,thissetofsamplesisequivalentto asetofsamplesfromthisprior.Inthisthesis,wefocusprimarilyonapplicationsin whichwehaveauniformpriorand,hence,usethesamplesfromthepriorinStep1of Algorithm4.Givenanapplicationwithanonuniformprior,wehavetworemedies: considerasetofsamplesfromtheposteriorinStep1andmodifytheintegralin Eq. IV.1 tobewithrespectto P ,orapproximateEq. IV.1 intermsofthevalues oftheobserveddensityandpushforwardofthepriordensitywhichallowsintegration withrespecttothepriormeasureontheparameterspace.Thistopicisbeyondthe focusofthisthesis;theinterestedreaderisreferredto[66]foracompletediscussion 41 PAGE 59 onthistopic. Algorithm4appearstobeacomputationallyexpensiveproceduresinceitrequiressolving M separateSIPsand,asnotedin[16],approximating Q prior D canbe expensive.However,weonlyneedtocomputethisapproximationonce,aseach KL Q inStep4ofAlgorithm4iscomputedusingthesamepriorandmap Q and,therefore, thesame Q prior D .Inotherwords,thefactthattheconsistentBayes'methodonly requiresapproximatingthepushforwardofthepriorimpliesthatthisinformationcan beusedtoapproximate 'sfordierentobserveddensitieswithoutrequiringadditionalmodelevaluations.Thissignicantlyimprovesthecomputationaleciencyof theconsistentBayesianapproachinthecontextofSB-OED. IV.1.3DeningtheOED Givenadeterministicmodel,priorinformationonthemodelparameters,aspace ofpotentialEDs,andagenericdescriptionoftheuncertaintiesforeachQoI,wedene theOEDasfollows. DenitionIV.2OED Let Q representthedesignspace,i.e.,thespaceofallpossibleEDs,and Q 2Q beaspecicED.Then,theOEDisthe Q 2Q thatmaximizes theEIG, Q opt :=argmax Q 2Q E KL Q : IV.6 Ingeneral,thedesignspace Q ,caneitherbeacontinuousordiscretespace.Inthis chapter,ourfocusisontheutilizationoftheconsistentBayesianmethodologywithin theOEDframework,hence,wesimplyndtheOEDoveradiscretesetofcandidate EDsthatmaycomefromasamplingofacontinuousdesignspace. IV.2InfeasibleData TheOEDprocedureproposedaboveisbaseduponconsistentBayesianinference whichrequiresthatthepushforwardmeasure,inducedbythepriorandthemodel, 42 PAGE 60 dominates theobservedmeasure,thatis,anyeventthatweobservewithnon-zero probabilitywillbepredictedusingthemodelandpriorwithnon-zeroprobability, seeSectionIII.2.Duringtheprocessofcomputing E KL Q ,itispossiblethatwe violatethisassumption.Specically,dependingonthemeanandvarianceofthe observationaldensity,wemayencounter D 2O Q suchthat R D D d D < 1,i.e.,the supportof D extendsbeyondtherangeofthemap Q ,seeFigureIV.5upperright. Werefertoobserveddatathatliesoutsidethedataspace D as infeasibledata .Here, wediscusspotentialcausesofinfeasibledataandoptionsfordealingwithinfeasible datawhenestimatinganOED. IV.2.1InfeasibleDataandConsistentBayesianInference WheninferringmodelparametersusingconsistentBayesianinference,acommon causeforinfeasibledataisthatthemodelbeingusedtoestimatetheOEDisinadequate.Thatis,thedeviationbetweenthecomputationalmodelandrealityislarge enoughtoprohibitthemodelfrompredictingalloftheobservationaldata.Thedeviationbetweenthemodelpredictionandtheobservationaldataisoftenreferredtoas modelstructureerrorandcanoftenbeamajorsourceofuncertainty.Thisisanissue formost,ifnotall,inverseparameterestimationproblems[38].Recentlytherehas beenanumberofattemptstoquantifythiserror[58].Suchapproachesarebeyond thescopeofthisthesis.Inthefollowing,weassumethatthemodelstructureerror doesnotpreventthemodelfrompredictingalltheobservationaldata. IV.2.2InfeasibleDataandOED When D isapropersubsetof R m ,thedensitiesin O Q givenbyEq.IV.3may produceinfeasibledata.Theeectofthisviolationincreasesas approachesthe boundaryof D .Toremedythisviolation,wemustmodify O Q .Wechoosetotruncate andnormalize D 2O Q sothatsupp^ D D ,where^ D isthetruncateddensity. 43 PAGE 61 Wedenethisobserveddensityspace ^ O Q accordingly, ^ O Q = ^ D = D C : D 2O Q ;C = Z D D D : IV.7 ExampleIV.2InfeasibleData Inthisexample,weusethenonlinearmodel fromSectionIII.5todemonstratethatinfeasibledatacanarisefromrelativelybenign assumptions.Supposetheobserveddensityon Q isatruncatednormaldistribution withmean0.982andstandarddeviationof0.01.Inthisone-dimensionaldataspace, thisobserveddensityisabsolutelycontinuouswithrespecttothepushforwardofthe prioron Q ,seeFigureIV.4left.Next,supposetheobserveddensityon Q isa truncatednormaldistributionwithmean0.3andstandarddeviationof0.04.Again, inthisone-dimensionaldataspace,thisobserveddensityisabsolutelycontinuouswith respecttothepushforwardoftheprioron Q ,seeFigureIV.4right.Bothofthese observedensitiesaredominatedbytheircorrespondingpushforwarddensities,i.e.,the modelcanreachalloftheobserveddata ineachcase FigureIV.4: leftThepushforwardofthepriorandobserveddensitieson Q rightThepushforwardofthepriorandobserveddensitieson .Noticethesupport ofeachobserveddensityiscontainedwithintherangeofthemodel,i.e.,theobserved densitiesareabsolutelycontinuouswithrespecttotheircorrespondingpushforward densities. 44 PAGE 62 However,considerthedataspacedenedby Q = )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(x 1 ;x 2 andthecorrespondingpushforwardandobserveddensitiesonthisspace,seeFigureIV.5.The non-rectangularshapeofthecombineddataspaceisinducedbythenonlinearityinthe modelandthecorrelationsbetween x 1 and x 2 .AsweseeinFigureIV.5,the observeddensityusingtheproductofthe1-dimensionalGaussiandensitiesisnotabsolutelycontinuouswithrespecttothepushforwarddensityon Q ,i.e.,thesupportof obs D extendsbeyondthesupportof Q prior D .ReferringtoEq. IV.7 ,wenormalizethis observeddensityover D ,seeFigureIV.5bottom.Nowthatthenewobserveddensity obeystheassumptionsneeded,wecansolvetheSIPasdescribedinSectionIII.2. IV.2.3ComputationalConsiderations Next,weaddressthecomputationalissueofnormalizing obs D over D .Fromthe plotof Q prior D inFigureIV.5topleft,itisclearthedataspacemayhaveacomplex shapeandbenon-convex.Hence,normalizing obs D ,asinFigureIV.5bottom,over D maybecomputationallyexpensive.Fortunately,theconsistentBayesianapproach providesameanstoavoidthisexpense.NotethatfromEq.III.9wehave, P = P prior P obs D Q P Q prior D Q ; IV.8 where P prior = P Q prior D Q =1whichimplies, P = P obs D Q : IV.9 Therefore,normalizing obs D over D isequivalenttosolvingtheSIPandthennormalizing~ wherethetildeindicatesthisfunctionmaynotintegrateto1over D over .Although maynotalwaysbeageneralizedrectangle,ourpreviousassumption thatwehaveadescriptionofthepriorknowledgeoftheparametersimplieswe haveacleardenitionof ,andwecanthereforeecientlyintegrate~ over and 45 PAGE 63 FigureIV.5: topleftThepushforwardofthepriorforthemap Q .topright Theobserveddensityusingthejointinformationwhichnowextendsbeyondtherange ofthemap.bottomThenormalizedobserveddensitythatdoesnotextendbeyond therangeofthemap. 46 PAGE 64 normalize~ by = ~ R ~ d : IV.10 IV.3NumericalExamples Inthissection,weconsidertwomodelsofphysicalsystems.First,weconsider astationaryconvection-diusionmodelwithasingleuncertainparameterdening themagnitudeofthesourceterm.Then,weconsideratime-dependentdiusion modelwithtwouncertainparametersdeningthethermalconductivitiesofametal rod.Ineachexample,wehaveaparameterspace ,asetofpossibleQoI,anda speciednumberofQoIforwhichwecangatherdatainanexperiment.Thisinturn denesadesignspace Q ,andwelet Q 2Q representasingleEDand D = Q the correspondingdataspace.ForeachED,welet O Q denotetheassociatedobserved densityspace. Thefollowingtwoexampleshavecontinuousdesignspaces.WechoosetoapproximatetheOEDbyselectingitfromalargesetofcandidatedesigns.Thisapproach waschosenbecauseitismuchmoreecienttoperformtheforwardpropagationof uncertaintyusingrandomsamplingonlyonceandtocomputeallofthecandidate measurementsforeachoftheserandomsamples.Alternatively,onecouldpursuea continuousoptimizationformulationwhichwouldrequireafullforwardpropagation ofuncertaintyforeachnewdesign. ExampleIV.3StationaryConvection-Diusion:UncertainSourceAmplitude Here,weconsiderastationaryconvection-diusionproblemwithasingleuncertain parametercontrollingthemagnitudeofasourceterm.ThisexampleservestodemonstratethattheOEDformulationcangiveintuitiveresultsforsimpleproblems. Problemsetup 47 PAGE 65 Considerastationaryconvection-diusionmodelonasquaredomain: 8 > > > > > > < > > > > > > : )]TJ/F20 11.9552 Tf 9.299 0 Td [(D r 2 u + r vu = S;x 2 ; r u n =0 ;x 2 )]TJ/F21 7.9701 Tf 7.314 -1.793 Td [(N @ ; u =0 ;x 2 )]TJ/F21 7.9701 Tf 7.314 -1.793 Td [(D @ ; IV.11 with S x = A exp )]TJ 13.15 8.088 Td [(jj x src )]TJ/F20 11.9552 Tf 11.956 0 Td [(x jj 2 2 h 2 ; where =[0 ; 1] 2 u istheconcentrationeld,thediusioncoecient D =0 : 01 ,the convectionvector v =[1 ; 1] ,and S isaGaussiansourcewiththefollowingparameters: x src isthelocation, A istheamplitude,and h isthewidth.Weimposehomogeneous Neumannboundaryconditionson )]TJ/F21 7.9701 Tf 7.315 -1.793 Td [(N rightandtopboundariesandhomogeneous Dirichletconditionson )]TJ/F21 7.9701 Tf 7.314 -1.794 Td [(D leftandbottomboundaries.Forthisproblem,wechose x src =[0 : 5 ; 0 : 5] ,and h =0 : 05 .Welet A beuncertainwithin [50 ; 150] .Thus,the parameterspaceforthisproblemis =[50 ; 150] Givenasourceamplitude A ,weapproximatesolutionstothePDEinEq. IV.11 usinganiteelementdiscretizationwithcontinuouspiecewisebilinearbasisfunctions denedonauniform 25 25 spatialgrid. Results Assumethatwehaveonlyonesensortoplaceinthespatialdomaintogathera singleconcentrationmeasurement.Inotherwords,thereisabijectionbetween and Q sopoints x 0 ;x 1 2 canbeusedtoindextheEDs.Thegoalistoplacethissingle sensorin tomaximizetheEIGabouttheamplitudeofthesource.Wediscretizethe continuousdesignspace Q using2,000uniformrandompointsin f x k 0 ;x k 1 g 2000 k =1 whichcorrespondsto2,000EDsdenotedby f Q k g 2000 k =1 where Q k = u x k 0 ;x k 1 .We assumetheuncertaintyinanymeasurementforeachpossibleQoIisdescribedbya truncatedGaussianprolewithaxedstandarddeviationof0.1.Thisproduces 2 ; 000 48 PAGE 66 observeddensityspaces, fO Q k g 2000 k =1 ,asdescribedinEq. IV.7 Wegenerate5,000samples, f i g 5000 i =1 ,fromtheuniformprioron andcompute f Q k i g 5000 i =1 2000 k =1 .Wecalculate f E KL Q k g 2000 k =1 usingAlgorithm4andplot E KL Q asafunctionoftheindexingsetdenedby inFigureIV.6.Noticethe EIGisgreatestwhentheEDisdenedbymeasurementstakennearthecenterofthe domainnearthelocationofthesourceandinthedirectionoftheconvectionvector awayfromthesource.Thisresultmatchesintuition,asweexpectdatagatheredin regionsofthedomainthatexhibitsensitivitytotheparameterstoproducehighEIG values. FigureIV.6: SmoothedplotoftheEIGovertheindexingsetforthedesign space Q .Noticethehighervaluesinthecenterofthedomainandtowardsthetop rightinthedirectionoftheconvectionvectorfromthelocationofthesource,this isconsistentwithourintuition. ExampleIV.4Time-DependentDiusion:UncertainDiusionCoecients Here,weconsideratime-dependentdiusionproblemwithtwouncertainparameters deningthethermalconductivitiesofametalrod. Problemsetup 49 PAGE 67 Considerthetime-dependentdiusionequation: 8 > > > > > > < > > > > > > : c @u @t = r r u + S;x 2 ;t 2 t 0 ;t f ; @u @n =0 ;x 2 @ ; u x ;0=0 ; IV.12 with S x = A exp )]TJ 13.15 8.088 Td [(jj x src )]TJ/F20 11.9552 Tf 11.955 0 Td [(x jj 2 w ; where =[0 ; 1] representsanalloyrod, =1 : 5 isthedensityoftherod, c =1 : 5 is theheatcapacity, isthethermalconductivitywhichisuncertaininthisproblem, and S isaGaussiansourcewiththefollowingsourceparameters: x src =[0 : 5] isthe location, A =50 istheamplitude,and w =0 : 05 isthewidth.Welet t 0 =0 t f =1 andimposehomogeneousNeumannboundaryconditionsoneachendoftherod. Supposetherodismanufacturedbyweldingtogethertworodsofequallengthand ofsimilaralloytype.However,duetothemanufacturingprocess,theactualalloy compositionsmaydierslightlyleadingtouncertainthermalconductivityproperties ontheleft-halfandright-halfoftherod,denotedby 1 and 2 ,respectively.Specifically,welet =[0 : 01 ; 0 : 2] 2 Giventhethermalconductivityvaluesforeachsideoftherod,weapproximate solutionstothePDEinEq. IV.12 usingthestate-of-the-artopensourceniteelementsoftwareFEniCS[3,47]witha 40 40 triangularmesh,piecewiselinearnite elements,andaCrank-Nicolsontimesteppingscheme. Results Supposetheexperimentsinvolveplacingtwocontactthermometersalongthefull rodthatcanrecordtwoseparatetemperaturemeasurementsat t f .Thus,thereisa bijectionbetween and Q sopoints x 0 ;x 1 2 canbeusedtoindextheEDs. Thegoalistoplacethetwosensorsin tomaximizetheEIGabouttheuncertain 50 PAGE 68 thermalconductivities.Wediscretizethecontinuousdesignspace Q using20,000 uniformrandompointsin f x k 0 ;x k 1 g 20000 k =1 ,whichcorrespondsto20,000EDs denotedby f Q k g 20000 k =1 ,where Q k = )]TJ/F20 11.9552 Tf 5.479 -9.683 Td [(u x k 0 ;t f ;u x k 1 ;t f foreach k .Weassume theuncertaintyinanymeasurementforeachpossibleQoIisdescribedbyatruncated Gaussianprolewithaxedstandarddeviationof0.1.Thisproducesobserveddensity spaces, fO Q k g 20000 k =1 ,asdescribedinEq. IV.7 Wegenerate5,000samples, f i g 5000 i =1 ,fromtheuniformprioron andcompute f Q k i g 5000 i =1 20000 k =1 .Wecalculate f E KL Q k g 20000 k =1 usingAlgorithm4andplot E KL Q asafunctionoftheindexingset for Q inFigureIV.7.Wenotethat, duetothesymmetrywithin alongtheline x 0 = x 1 ,weonlyshowtheEIG overhalfoftheindexingset becausetheotherhalfcorrespondstoarenaming ofcontactthermometeroneascontactthermometertwoandviceversa.Moreover, E KL Q isclearlynonconvexandsubstantiallymorecomplicatedthantheresults observedinExampleIV.3.WelabelthefournotablelocalmaximaoftheEIGas R1,R2,R3,andR4.Placingthecontactthermometerssymmetricallyawayfromthe midpoint{neartheendsoftherodR1{canleadtooptimaldesigns,whichmatches physicalintuitionseethelowerrightcornerofFigureIV.7.Inthetableintheright FigureIV.7,weshowthelocalOEDlocationsandEIGvaluesforeachofthefour regionslabeledinFigureIV.7.TheglobalOEDislocatedinR1. Inthefollowingchapter,weproposeanOEDformulationbasedsolelyonlocallinearapproximationsofQoImaps.Inthisway,weprovideadescriptionof theexpectedinformationgain"ofanEDindependentoftheuncertaintiesonthe observeddata.Moreover,themethoddescribedinthefollowingchapterallowsfor theconsiderationofoptimizationobjectivesotherthanexpectedinformationgain". Inparticular,weareinterestedintheaccuracyofthesolutiontoagivenSIP.We consideramulti-objectiveoptimizationprobleminwhichweoptimizeboththeexpectedinformationgain"andtheexpectedaccuracyoveradesignspace.Weprovide 51 PAGE 69 RegionOptimalEIG R14.909 R24.769 R34.729 R44.269 FigureIV.7: leftSmoothedplotoftheEIGovertheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof .This locationcorrespondstoplacingthermometersnearseparateendsoftherod.However, weobservemultiplelocalmaximawithinthedesignspace,denotedR1,R2,R3and R4.rightTheEIGforthelocalOEDineachofthefourregions. athoroughcomparisonofthetwoSB-OEDmethodsinChapterVI. 52 PAGE 70 CHAPTERV MEASURE-THEORETICOPTIMALEXPERIMENTALDESIGN Inthischapter,wefocusontwomajorgoals:aquanticationofhowamap Q impactstheaccuracyofthesolutiontoagivenSIP[14],andadenitionofOEDindependentoftheuncertaintiesonthedataspace.Therefore,wedonotsolveanySIPs duringthesectionsleadingtothedenitionoftheOED,andwedonotspecifyuncertaintiesonthedataspace.Weconsiderlocallinearapproximationsofagivenmap Q deninganED,andquantifytwodrivinggeometricpropertiesofinverseimages:the expected skewness andexpected scalingeect [17].Theexpectedskewnessdescribes theexpectedaccuracyofthesolutiontoagivenSIP,andtheexpectedscalingeect describestheexpectedprecisionofthesolutiontoagivenSIP.Thisexpectedscaling eectcanbethoughtofasatypeofquanticationoftheexpectedinformationgained bysolutionofagivenSIP.Furthercomparisonbetweentheexpectedscalingeect andtheEIGcanbefoundinChapterVI. WeintroducenotationinSectionV.1.InSectionV.2,weintroducelinearalgebra resultsneededtoecientlyapproximatethelocalskewnessandlocalscalingeect ofagivenQoImap.InSectionsV.3andV.4,wedeneandprovideameansfor approximatingtheexpectedskewnessandtheexpectedscalingeect,respectively.We denetheOEDinSectionV.5andprovideseveralnumericalexamplesinSectionV.6. RemarkV.1 In[14],thecomputationalcomplexityofagivenSIPwascharacterized intermsoflocallinearpropertiesoftheQoImapunderconsideration.However, thisworkdidnotprovideacomputationallyecientmethodforapproximatingthe expectedcomputationalcomplexitynorprovideanoptimalityobjectivefordescribing theexpectedprecisionofthesolutiontoaSIP. V.1Notation Welet Q denotethespaceofallpiecewisedierentiableQoImaps,i.e.,thespace ofEDs,underconsiderationbythemodelerforcollectingdatatosolveastochastic 53 PAGE 71 inverseproblem.Forexample,supposethemodelerisconsideringtwodierentEDs thatleadtoQoImaps, Q a : !D a ; or Q b : !D b : Then, Q = Q a ;Q b : Foranymap Q 2Q ,weareinterestedinhowthismapaectsthelocalgeometric propertiesrelatedtotheshapespecicallytheskewnessandscalingofthevolume measureoftheinverseofanoutputevent E 2B D .Conceptually,theskewness of Q describesageneralincreaseingeometriccomplexityof E under Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,which quantitativelyisrelatedtothenumberofsamplesin requiredtoaccuratelyestimate Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Thelocalscalingeectof E bythemap Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 describestheprecision ofusingthemap Q toidentifyparametersthatmapto E .InSectionsV.3andV.4, weshowthatboththelocalskewnessof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andthelocalscalingof E bythe map Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 canbedescribedintermsofsingularvaluesofaJacobianof Q .Welet J ;Q 2 R d n denotetheJacobianof Q atapoint 2 ,andwhentheQoImapbeing consideredisclearfromcontext,wesimplyuse J Forsimplicityindescribingsets,weignoreanyboundariesof or D .However,in practice,weassumethat iscompact,which,bytheassumedsmoothnessof Q ,gives that D isalsocompact.Inotherwords,weassumethat and D D arenite, whichisoftenthecaseinpracticebytheintroductionofknownorassumedbounds ofparameteranddatavalues.Second,sincetheskewnessandscalingpropertiesof Q mayvarysignicantlythroughout when Q isnonlinear,wemustaccountforthis variabilityindeterminingoptimalQoI. V.2 d -dimensionalParallelepipeds Determiningthesizeof d -dimensionalparallelepipedsembeddedin n -dimensional 54 PAGE 72 spaceisfundamentaltoecientlyapproximatinglocalskewnessandscalingeectsof agivenmap Q .Weareinterestedintwocases: d -dimensionalparallelepipedsdened bytherowsofagivenmatrix J ,and d -dimensionalparallelepipedsdeterminedby thecrosssectionsof n -dimensionalcylindersthataredenedbythepre-imageofa d dimensionalunitcubeunder J .Thefollowingtechnicallemmaandensuingcorollary describeamethodfordeterminingthesizesoftheseobjectsbasedonthesingular valuesofJacobiansof Q LemmaV.2 Let J beafullrank d n matrixwith d n ,and Pa J denotethe d -dimensionalparallelepipeddenedbythe d rowsof J .TheLebesguemeasure d in R d of Pa J isgivenbytheproductofthe d singularvalues f k g d k =1 of J ,i.e., d Pa J = d Y k =1 k : V.1 Proof: Thesingularvaluesof J areequaltothesingularvaluesof J > .Consider thereducedQRfactorizationof J > J > = ~ QR; V.2 where ~ Q is n d and R is d d .BythepropertiesoftheQRfactorization,weknow thesingularvaluesof R arethesameasthesingularvaluesof J > .Let x 2 R d ,then jj ~ Q x jj 2 = ~ Q x > ~ Q x = x > ~ Q > ~ Q x = x > x = jj x jj 2 ; V.3 so ~ Q isanisometry.ThisimpliestheLebesguemeasureoftheparallelepipeddened bytherowsof R isequaltotheLebesguemeasureoftheparallelepipeddenedby thecolumnsof J > ,ortherowsof J d Pa J = d Pa R = d Y k =1 k = d Y k =1 k ; V.4 55 PAGE 73 where f k g 1 k d arethesingularvaluesof R and f k g 1 k d arethesingularvalues of J Wenowturnourattentiontothesecondcaseofdescribingthesizeofa d dimensionalparallelepipeddeterminedbythecrosssectionofan n -dimensionalcylinderdenedbythepre-imageofa d -dimensionalunitcubeunder J .Inthiscase,we considerthepseudo-inverseof J J + = J > JJ > )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .Asisevidentfromtheformula ofthepseudo-inverse,therangeof J + isequaltotherowspaceof J .Thisimplies thatthe d -measureofthecross-sectionofthepre-imageofaunitcubeunder J is equaltothethe d -measureoftheparallelepipeddenedbythecolumnsof J + CorollaryV.3 Let J beafullrank d n matrixwith d n .Then Pa J + > isa d dimensionalparallelepipeddeningacross-sectionofthepre-imageofa d -dimensional unitcubeunder J anditsLebesguemeasure d isgivenbytheinverseoftheproduct ofthe d singularvalues f k g d k =1 of J ,i.e., d Pa J + > = d Y k =1 k )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 : V.5 Proof: Considerthepseudo-inverseof J J + = J > JJ > )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 : Fromthisequation,itisclearthatthecolumnspaceof J + isequaltotherowspace of J .Therowspaceof J denesasubspaceorthogonaltothe n -dimensionalcylinder thatisthepre-imageofaunitcubeunder J .Therefore,thecolumnspaceof J + is orthogonaltothepre-imagecylinderand Pa J + > isa d -dimensionalparallelepiped deninganorthogonalcross-sectionofthiscylinder. Frombasicresultsinlinearalgebra,thesingularvaluesof J + > areequalto thoseof J + .Then,frompropertiesofthepseudo-inverse,thesingularvaluesof J + 56 PAGE 74 aretheinverseofthesingularvaluesof J .Finally,fromLemmaV.2,itfollowsthat d Pa J + > = d Y k =1 k )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ; V.6 where f k g d k =1 arethesingularvaluesof J V.3SkewnessandAccuracy In[14],itwasshownthatthenumberofsamplesdeningregulargridsandthus alsouniformi.i.d.setsofsamplesin requiredtoobtainaccurateapproximationsin -measureof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isproportionaltothemaximum localskewness ofthemap Q Below,wedenethelocalskewnessofaQoImap,andprovideameansforcomputing theexpectedskewness. First,assumethat Q islinearand E 2B D ageneralizedrectangle.When d = n Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isa d -dimensionalparallelepipedin ignoringanyaectsfrompossible intersectionswiththeboundaryof thatisin1-to-1correspondencewith E .If d PAGE 75 J 1 denotesthesubmatrixof J withtherstrowremoved, Pa J 1 isthe d )]TJ/F15 11.9552 Tf 10.994 0 Td [(1 -dimensionalparallelepipeddenedbythe d )]TJ/F15 11.9552 Tf 10.994 0 Td [(1 rowvectors of J 1 ,and d and d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 representthe d -and d )]TJ/F15 11.9552 Tf 12.549 0 Td [(1 -dimensionalLebesguemeasures,respectively. Wenowrelatethisfundamentaldecompositionresulttoinversesets,where,for axed Q 2Q ,welet J k; denotethesubmatrixof J withthe k throwremoved.For simplicity,weassumethat Q islinearand1 PAGE 76 Then,wedenethe localskewness ofthemap Q 2Q atapoint as S Q =max 1 k d S Q J ;Q ; j k : V.8 Conceptually, S Q J ; j k describestheamountof redundant informationpresent inthe k thcomponentoftheQoImapcomparedtowhatispresentintheother d )]TJ/F15 11.9552 Tf 11.346 0 Td [(1 componentswheninvertingnearthepoint 2 .Thesmallestvalue S Q canbe isone.Thereisnolargestvaluesincethereexistsmaps Q thathaveGDcomponent maps,buttheconditionoftheJacobianmaybearbitrarilylarge.IftheJacobianwere toeverfailtobefullrank,then S Q wouldbeinnite.However,theassumptionof GDQoIpreventsthisfromoccurring. ThefundamentaldecompositionofTheoremV.4alongwithLemmaV.2provides aconvenientmethodfordeterminingtheskewnessintermsofthe d -dimensional parallelepipedsdescribedby Q CorollaryV.6 Forany Q 2Q S Q canbecompletelydeterminedbythethe normsof n -dimensionalvectorsandproductsofsingularvaluesoftheJacobianof QoImapsofdimensions d )]TJ/F15 11.9552 Tf 11.955 0 Td [(1 and d S Q =max 1 k d j j k j d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 Pa J k; d Pa J : V.9 Proof: S Q =max 1 k d S Q J ; j k =max 1 k d j j k j j j ? k j =max 1 k d j j k j d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Pa J k; d Pa J ; V.10 thenapplyingLemmaV.2wehave max 1 k d j j k j d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Pa J k; d Pa J =max 1 k d j j k j Q d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 r =1 k;r Q d r =1 r : V.11 59 PAGE 77 where f r g 1 r d arethesingularvaluesof J and f k;r g 1 r d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 arethesingularvalues of J k; CorollaryV.6impliesthatwecanexploitecientsingularvaluedecompositions toalgorithmicallyapproximate S Q atanypoint 2 .Since S Q mayvary substantiallyover ,wemustquantifythisvariabilityinordertooptimallychoose Q 2Q .Althoughwehaveassumedthat Q 2Q isGD,werealizethat,inpractice, thismaynotbetrue.Inordertoaccommodatethisscenario,andtoreducethe impactofoutliers,wedenetheexpectedvalueof S Q inspiredbytheharmonicmean ofanitesetofpositivenumbers,i.e.,if isarandomvariableand f > 0,then theharmonicmeanof f isgivenby H f = 1 Z 1 f d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 : Thismotivatesthefollowing DenitionV.7 Forany Q 2Q ,wedenethe expectedskewnessESK as H S Q = 1 Z 1 S Q d )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ; V.12 Generally,weapproximate H S Q usingMonteCarlotechniquestogeneratea setofindependentidenticallydistributedi.i.d.samples i N i =1 andcompute H S Q H N S Q := 1 N N X i =1 1 S Q i )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 : V.13 RemarkV.8 Noticethedescriptionsofskewnessareindependentofthegeneralized rectangle E 2B D .Inotherwords,skewnessisapropertyinherenttothemap Q itself anddescribesthewayinwhichthegeometryof E 2B D ischangedbyapplying Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 V.4ScalingandPrecision 60 PAGE 78 Tomotivatewhatfollows,considerthesimpleproblemwherewemustchoose betweentwodierentEDsleadingtotwodistinctQoImaps Q a and Q b sothat Q = Q a ;Q b .Let E a and E b representthesetofallprobableobservations fromusingeither Q a or Q b ,respectively.Then,dependingonwhichexperiment weobserve,wewouldconcludethateithertheparametersbelongto )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(Q a )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E a or )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(Q b )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E b almostsurely.Suppose )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(Q a )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E a )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(Q b )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E b ; thenwegenerallyexpectthatensemblesofparametersamplesgeneratedfromresults basedontheexperimentleadingto Q a willhavesmallervariancewhichisadescriptionofprecisioninstatisticaltermsthanthosebasedontheexperimentleading to Q b .Thismotivatesageneralmeasure-theoreticgoalfordesigningexperiments whereeventsofhighprobabilityonadataspacearemadesmallinvolume"onthe parameterspacebyinvertingtheQoImap. Webeginwithasimplifyingassumptionthat Q 2Q islinearwithGDcomponent maps.Then,thereexistsa d n matrix J ,suchthat Q = J .If d = n and= R n itiseasilyshownfromstandardresultsinmeasuretheoryandlinearalgebrathat Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E = D E det J )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 = D E d Y k =1 k )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ; V.14 where f k g 1 k d arethesingularvaluesof J .Notethatif R n isproper,then theaboveequationisnotnecessarilytrueas Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E mayintersecttheboundaryof .Weneglectsuchboundaryeectsinthecomputations,andsimplynotethatin certaincasestheymayplayanimportantrolealthoughthisisnotthetypicalcase inourexperience.If d PAGE 79 skewnessof Q ,inordertoreducetheimpactofoutliers,wedenetheexpectedvalue of M Q bytheharmonicmeanofarandomvariable.Thismotivatesthefollowing DenitionV.9 Forany Q 2Q and 2 ,wedenethe local -measure scalingeect of Q as M Q = d Y k =1 k )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ; V.15 where f k g 1 k d arethesingularvaluesoftheJacobian J ;Q .The expected measurescalingeectESE isgivenby H M Q = 1 Z 1 M Q d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ; V.16 Aswiththeexpectedskewnessof Q ,wegenerallyapproximate H M Q usingasetof i.i.d.randomsamples i N i =1 andcomputing H M Q H N M Q := 1 N N X i =1 1 M Q i )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 : V.17 Wesummarizetheaboveresultsintothefollowing CorollaryV.10 Forany Q 2Q ,thelocalskewness,localscalingeect,average skewness,andaveragescalingeectof Q canbecomputedusingnormsofrow-vectors andsingularvaluesoftheJacobian J ;Q RemarkV.11 Analternativeto H M Q thataccountsforpossiblydierent D E istousesimplemultiplicationof D E assuggestedbyEq. V.14 ThegeometricinterpretationoftheESEoeredabovedescribeshowthesizeof setsarescaledastheyareinvertedwiththemap Q .Thatis,givenaQoImapwith anESEof10,weexpectthecrosssectionoftheinverseimageofanarbitraryset E 2B D to increase insizebyafactorof10.Likewise,givenaQoImapwithan ESEvalueof0.1,weexpectthecrosssectionoftheinverseimageofanarbitrayset 62 PAGE 80 E 2B D to decrease insizebyafactorof10.AnESEvalueof1suggeststhesizeof thecrosssectionoftheinverseimageof E 2B D willremainunchangedfromitssize in D .ThisisillustratedintheschematicinFigureV.1. WeoeranalternativegeometricinterpretationoftheESEthatshowsclearly howtheESEisrelatedtothe reduction ofuncertainty,i.e.,theprecisionofsolving aSIPwithaparticularQoImap.ConsidertwoQoImaps Q a and Q b suchthat theESEof Q a is0.1andtheESEof Q b is0.01.Weexpect Q a )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 maps E 2B D with D E =10toasetthesizeofaunitcubein ,andweexpect Q b )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 maps E 2B D with D E =100toasetthesizeofaunitcubein .Thatis,weexpect themap Q b )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 tomaplargersetsin D intoaregionthesizeoftheunitcubein ThisisillustratedintheschematicinFigureV.2. FigureV.1: TheESEdescribeshowuncertaintiesofaxedsizeonthedataspace arereducedundertheinverseoftheQoImap. V.5AMulti-ObjectiveOptimizationProblem Welet S R denotethesetofallpossiblevaluesof H S Q and M R denote thesetofallpossiblevaluesof H M Q forall Q belongingtoaspecied Q .Clearly, S isboundedbelowby1,whichrepresentsthecaseofoptimalglobalskewness"in aQoImap Q .Similarly, M isboundedbelowby0,whichrepresentsthecasewhere 63 PAGE 81 FigureV.2: TheESEdescribeshowmuchuncertaintyonthedataspaceisreduced tothesizeoftheunitcube. asetofoutputdataexactlyidentiesaparticulargeneralizedcontourofparameters responsibleforthedata. For Q ,wecanclearlyorderall Q 2Q accordingtoeitherthevaluesof H S Q or H M Q .Inotherwords,thevaluesof H S Q and H M Q separatelydescribean orderingindexonthespaceofallpossibleQoIgivenby Q .However,themapping Q 7! H S Q ;H M Q describesadoubleindexingwithnonaturalordering.We denemetricsinordertoquantifythedistancetotheoptimalpointof ; 0inthe Cartesianproductspacedescribedbythepairs H S Q ;H M Q .Whilethereare manyoptionsfordeningmetrics,athoroughinvestigationontheeectofdierent metricsisbeyondthescopeofthisthesiswhichisfocusedontheexpositionofthis generalapproach.Below,wechooseaparticularformforthemetricson S and M thatminimizestheeectofthepossiblydisparaterangesofvalueswemayobserve in S and M onthesolutiontothemulti-objectiveoptimizationproblem. Wedene S ;d S and M ;d M usingthemetrics d S x;y = j x )]TJ/F20 11.9552 Tf 11.955 0 Td [(y j 1+ j x )]TJ/F20 11.9552 Tf 11.955 0 Td [(y j forall x;y 2S ; V.18 64 PAGE 82 and d M x;y = j x )]TJ/F20 11.9552 Tf 11.955 0 Td [(y j 1+ j x )]TJ/F20 11.9552 Tf 11.956 0 Td [(y j forall x;y 2M : V.19 Choose 2 ; 1andlet Y denotetheCartesianproductspace SM ,withmetric denedby d Y x;y = !d S x 1 ;y 1 + )]TJ/F20 11.9552 Tf 11.956 0 Td [(! d M x 2 ;y 2 forall x;y 2 Y : V.20 Notethat determinestherelativeimportanceweplaceoneitherprecisionoraccuracy.Choosing =0implieswedisregardskewnessintheobjectivewhereaschoosing =1implieswedisregardthescalingeect. DenitionV.12 Given Q and 2 ; 1 ,the OEDproblem isdenedbythe multi-objectiveoptimizationproblem min Q 2Q d Y p;y ; V.21 where p = ; 0 isthe idealpoint and y = )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(H S Q ;H M Q for Q 2Q V.6NumericalExamples Inthissection,weagainconsidertwomodelsofphysicalsystemsoriginallyshown inChapterIV.First,weconsiderastationaryconvection-diusionmodelwithasingle uncertainparameterdeningthemagnitudeofthesourceterm.Inthisexample,we considerchoosingasingleQoItodeneanOED.Hence,weoptimizeonlyforthe ESE,asskewnessisnotdenedforone-dimensionaldataspaces.Next,weconsidera time-dependentdiusionmodelwithtwouncertainparametersdeningthethermal conductivitiesofametalrod. InDenitionV.12,wedenetheOEDintermsofa minimization problem.In thissection,weconsider maximizingtheinverse oftheoptimizationobjectivefortwo reasons:tocompareresultstothoseinChapterIVinamoreobviousway,and, 65 PAGE 83 byconsideringtheinverseof d Y p;y andtheESEandESKweaccentuatethe smallervaluesoftheoptimizationobjectivesinsuchawaythatmakestheOEDmore obviousinplotsoverindexingspaces. ExampleV.1StationaryConvection-Diusion:UncertainSourceAmplitude Here,weconsiderastationaryconvection-diusionproblemwithasingleuncertain parameterdeningthemagnitudeofasourceterm.ThisexampleservestodemonstratethattheOEDformulationgivesintuitiveresultsforsimpleproblems.SeeExampleIV.3foradescriptionofthemodelandproblemsetup. Results Wediscretizethecontinuousdesignspace Q fromExampleIV.3using2,000uniformrandompointsin f x k 0 ;x k 1 g 2000 k =1 ,whichcorrespondsto2,000EDsdenoted by f Q k g 2000 k =1 Wegenerate5,000uniformsamplesin andcompute f Q k i g 5000 i =1 2000 k =1 .We approximate f J i ;Q k g 5000 i =1 2000 k =1 usingaradialbasisfunctionRBFinterpolation method.Essentially,thismethodconstructsalocalsurrogateof Q k about i and dierentiatesthesurrogatetoobtainanapproximationto J i ;Q k ,seeAppendixA fordetails.Wecalculate f H N M Q k g 5000 k =1 usingEq. V.17 andplottheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 as afunctionoftheindexingsetdenedby inFigureV.3.Weseeresultssimilarto thoseinExampleIV.3,i.e.,weobservehighervaluesnearthecenterofthedomain nearthelocationofthesourceandinthedirectionoftheconvectionvectoraway fromthesource. ExampleV.2TimeDependentDiusion:UncertainDiusionCoecients Here,weconsideratime-dependentdiusionproblemwithtwouncertainparameters deningthethermalconductivitiesofametalrod.Wechoosethedataspacesunderconsiderationtobetwo-dimensionsaswell.WeconsideroptimizingtheESE, ESK,andsolvingamulti-objectiveoptimizationproblem.SeeExampleIV.4fora descriptionofthemodelandproblemsetup. 66 PAGE 84 FigureV.3: SmoothedplotoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingsetforthedesign space Q .Noticethehighervaluesinthecenterofthedomainandtowardsthetop rightinthedirectionoftheconvectionvectorfromthelocationofthesource,this isconsistentwithbothintuitionandtheresultsfromExampleIV.3. Results Wediscretizethecontinuousdesignspace Q fromExampleIV.3using20,000 uniformrandompointsin f x k 0 ;x k 1 g 20000 k =1 ,whichcorrespondsto20,000EDs denotedby f Q k g 20000 k =1 ,where Q k = )]TJ/F20 11.9552 Tf 5.48 -9.683 Td [(u x k 0 ;t f ;u x k 1 ;t f foreach k .Wegenerate5,000uniformsamples, f i g 5000 i =1 ,in andcompute f Q k i g 5000 i =1 20000 k =1 Weapproximate f J i ;Q k g 5000 i =1 20000 k =1 usinganRBFinterpolationmethod,seeAppendixAfordetails.Wecalculate f H N M Q k g 20000 k =1 and f H N S Q k g 20000 k =1 using Equations V.17 and V.13 ,respectively. InFiguresV.4-V.6,weshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ,ESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,and )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(d Y p;y )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 with =0 : 5 overtheindexingset forthedesignspace Q .Wenotethat,duetothesymmetry within alongtheline x 0 = x 1 ,weonlyshoweachoptimizationobjectiveoverhalf oftheindexingset becausetheotherhalfcorrespondstoarenamingofcontact thermometeroneascontactthermometertwoandviceversa.Eachoftheoptimization objectivesproducenonconvexfunctionsovertheindexingsetandaresubstantially morecomplicatedthantheresultsobservedinExampleV.1.Foreachobjective,we 67 PAGE 85 labelthenotablelocalmaximawithin .FortheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andESK )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ,weobserve fournotablelocalmaxima,andfor d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Y weobserveonenotablelocalmaxima.However, ineachcase,placingthecontactthermometerssymmetricallyawayfromthemidpoint i.e.,neartheendsoftherodR1canleadtooptimaldesigns,whichmatches physicalintuitionseethelowerrightcorneroftheimagesinFiguresV.4-V.6.The tablesintherightofFiguresV.4{V.6showthelocalOEDoptimizationobjective valuesforeachoftheregionslabeledintheircorrespondingplotsovertheindexingset InFigureV.4,theESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthelocalOEDsinR2andR3arerelativelysimilar. Thisisexpected,asthesymmetryofthephysicaldomainandthelocationofthesource suggestthatthesetwoEDlocationswouldyieldasimilarreductioninuncertainties ofthemodelinputparameters.TheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthesetwolocalOEDarenotexactly thesameduetovarioussourcesofapproximationerrors:thediscretizationofthe domain,thesamplingofthe ,andthesamplingof RegionOptimalESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 R16 : 54 E +02 R27 : 00 E +02 R36 : 45 E +02 R46 : 06 E +02 FigureV.4: leftSmoothedplotoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 overtheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof .This locationcorrespondstoplacingthermometersneareachendoftherod.However, weobservefourprominantlocalmaxima,denotedR1,R2,R3andR4.rightThe ESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthelocalOEDineachofthefourregions. 68 PAGE 86 RegionOptimalESK )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 R19 : 95 E )]TJ/F15 11.9552 Tf 11.955 0 Td [(01 R29 : 42 E )]TJ/F15 11.9552 Tf 11.955 0 Td [(01 R39 : 61 E )]TJ/F15 11.9552 Tf 11.955 0 Td [(01 R46 : 13 E )]TJ/F15 11.9552 Tf 11.955 0 Td [(01 FigureV.5: leftSmoothedplotoftheESK )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingset forthe designspace Q .Noticethehighvaluesinthebottomrightcornerof .This locationcorrespondstoplacingthermometersneareachendoftherod.However, weobservefourprominantlocalmaxima,denotedR1,R2,R3andR4.rightThe ESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthelocalOEDineachofthefourregions. RegionOptimal d Y p;y )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 R12 : 17 E +02 FigureV.6: leftSmoothedplotof d Y p;y )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,with =0 : 5,overtheindexing set forthedesignspace Q .Noticethehighvaluesinthebottomrightcorner of .Thislocationcorrespondstoplacingthermometersneareachendofthe rod.Here,weobserveonlyoneprominantlocalmaxima.right d Y p;y )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,with =0 : 5,forthelocalOEDinthesingleregionofinterest. 69 PAGE 87 V.7Summary Inthischapter,wedevelopedaSB-OEDapproachbasedentirelyonlocallinear approximationsofQoImaps.Thisapproachoptimizestwogeometricpropertiesof inverseimageswhicharerelatedtotheaccuracyandprecisionofthesolutiontoa givenSIP.Thenoveltyofthisapproachlieswithintwokeyfeatures:theapproach denesanOEDindependentoftheexpectedandoftenunknownuncertaintieson theobserveddata,and,providesaquanticationoftheexpectedcomputational limitationsinaccuratelyapproximatingsolutionstoagivenSIP.Inthefollowingchapter,wemakedirectcomparisonsbetweenthisapproachandtheconsistentBayesian approachtoSB-OEDdevelopedinChapterIV. Themulti-objectiveoptimizationproblemdenedinEq.V.21issomewhatunsatisfying.Themetricsdenedon S and M arechosentoreducethechancesthat oneoptimizationobjectivedominatesthedecisionbecausethevaluesofthatobjective arelarger.Then ischosentodeterminea weighting ofthepriorityofoptimizing eachobjective.Determininganoptimal tosatisfytheneedsofaspecicproblem isanopenquestion.InChapterVII,weaddressthisquestion,inasomewhatunconventionalway,todenea singleobjective OEDproblemthatisimplicitlyafunction oftheESE,thenumberofmodelsolvesavailabletoapproximatethesolutiontoa givenSIP,andthedimensionoftheparameterspace. 70 PAGE 88 CHAPTERVI EXPECTEDSCALINGEFFECTVSEXPECTEDINFORMATION GAIN Inthischapter,weprovideanin-depthexplorationandcomparisonofthetwo previouslydevelopedSB-OEDapproaches:theconsistentBayesianapproach,describedinChapterIV,andthemeasure-theoreticapproach,describedinChapterV. TheconsistentBayesianapproachisbasedontheclassicalBayesianmethodologyfor SB-OED.ThisapproachquantiestheutilityofagivenEDusingtheEIG,whichis denedintermsofsolutionstoasetofprobableSIPsofinterest.Thus,thisapproach requiresknowledgeoftheproleoftheobserveddensityoneachQoIinordertosolve theSIPandthenapproximatetheOED. Themeasure-theoreticapproachquantiestheutilityofagivenEDintermsof theESE 1 ,whichdescribeshowmeasurablesetsinthedataspaceare scaled asthey areinvertedintotheparameterspacebyagivenQoImap.Thisapproachisbased entirelyonlocallinearapproximationsoftheQoImapunderconsideration.Thus, computationoftheESErequiresgradientapproximations.However,thisapproach does not requirepriorknowledgeoftheproleoftheobserveddensityoneachQoI. Thegeneralityofthisapproach,basedonpropertiesofagivenQoImap not on propertiesofsolutionstoasetofSIPs,provesvaluableinrobustlyapproximating OEDswithlimitednumbersofmodelsolvesavailableandunknownprolesofthe observeddensityoneachQoI. InSectionVI.1,weintroducetwotechnicallemmasusedintheanalysisofthe resultspresentedinthischapter.InSectionVI.2,wediscusscomputationalconsiderationsforeachapproach.Wefocusonthecomputationalcostasafunctionof thenumberofmodelsolves.InSectionVI.3,weprovideanexampletoillustrate theabilityofeachmethodtodistinguishbetweenhighlysensitiveQoIgivenalim1 Themeasure-theoreticapproachalsoconsiderstheESK,however,forsimplicity,wefocusonly ontheESEinthischapter. 71 PAGE 89 itednumberofmodelsolves.InthecontextofOED,weoftenmustprescribean expectedutilitysystematicallytoabroadsetofEDs.ThissetofEDsmayinclude EDswheretheproleoftheobserveddensityislargerelativetotherangeofthedata space.Moreover,itmayincludeEDscomposedofQoIprovidingredundantdata, i.e.,thecomponentsmayfailtobelinearlyindependentinregionsof,orthroughout,theparameterspace.InSectionVI.4,weconsiderthestabilityofeachOED methodwithrespecttovaryingprolesoftheobserveddensity,withaspecicfocus onobserveddensitieswhosesupportextendswellbeyondtherangeofthedataspace. InSectionVI.5,weexploretheabilityofeachOEDmethodtodistinguishoptimal propertiesamongstEDsdenedbyQoImapswithJacobiansthatarenotlocally,or globally,fullrankin .InSectionVI.6,weprovidesummarizingremarksforthe chapter. VI.1TechnicalLemmas Inthissection,weintroducetwotechnicallemmasusedlaterinthischapter.The rstlemmastatesthat,underacertainassumption,theMCapproximationtothe KL-divergenceisboundedfromabovebylog N ,where N isthenumberofsamples usedintheMCapproximation.ThesecondlemmastatesthattheKL-divergenceof from prior isequaltotheKL-divergenceof obs D from Q prior D LemmaVI.1 Suppose Q : !D is B -measurable, prior isuniformover ,and i N i =1 isthesetofi.i.d.sampleswithrespectto prior in usedtoapproximate both and KL Q prior : .ThentheMCapproximationto KL Q prior : denoted KL Q prior : V ,isboundedby log N KL Q prior : V log N : Proof: Theuniformpriorisgivenby prior = 1 8 2 : VI.1 72 PAGE 90 Weobservethat KL Q prior : V ismaximumwhenthereisonlyone i 2 supp Inthiscase,usingthestandardMCapproximationthat V i =N implies thatthecomputed isgivenby = 8 > > < > > : N ; 2V i ; 0 ;= 2V i ; VI.2 where V i istheVoronoicellassociatedwith i 2 supp .Then, KL Q prior : V = N N X i =1 i log i prior i VI.3 N N log N= 1 = VI.4 =log N whereEq.VI.3comesfromtheMCestimateoftheintegralandEq.VI.4results fromthesubstitutionof prior and aboveintoEq.VI.3. LemmaVI.2 Assume Q : !D is B -measurable.Then,theKL-divergenceof from prior isequaltotheKL-divergenceof obs D from Q prior D KL Q prior : = KL Q Q prior D : obs D : 73 PAGE 91 Proof: UsingthedenitionoftheKL-divergencegivenbyEq.IV.1,wehave KL Q prior : = Z log prior d = Z prior obs D Q Q prior D Q log obs D Q Q prior D Q d = Z obs D Q Q prior D Q log obs D Q Q prior D Q dP prior = Z D obs D q Q prior D q log obs D q Q prior D q dP Q prior D q = Z D Q prior D q obs D q Q prior D q log obs D q Q prior D q d D q = Z D obs D q log obs D q Q prior D q d D q = KL Q Q prior D : obs D ; whereweusethedenitionof giveninEq.III.12andthefactthat Z f Q dP prior = Z D f q dP Q prior D q : VI.2ComputationalConsiderations Inthissection,wediscussthecomputationalrequirementofeachoftheOED approachesinthecontextofseveralrelevantcomputationalmodelingscenarios.The variousscenariosdonotlargelyimpactourabilitytocomputeapproximationsto OEDsusingtheconsistentBayesianapproach.TheconsistentBayesianapproach requiresapproximatesolutionstoSIPs,andthissimplynecessitatesaforwardprogagationofuncertaintythroughthecomputationalmodelinordertocomputethe pushforwardofthepriordensity.Asdiscussedpreviously,weemploystandardKDE techniquestoconstructanon-parametricapproximationofthepushforwardofthe 74 PAGE 92 priordensity 2 .Thisrequiresasetofsamplesdrawnfromthepriorandthesolution ofthemodelforeachsample.Thus,forimplementationoftheconsistentBayesian approachtoOED,wesimplyrequireasetofsamplesfromthepriorandcorrespondingdatafromthemodelsolutions.However,accuracyoftheapproximationstothe solutionsofeachSIP,andthereforetheaccuracyoftheOED,maysuergivenasmall numberofsamplesfromthepriorand/orhigh-dimensionalspaces. Themeasure-theoreticapproachtoOEDrequiresgradientinformationatasetof samplesin foreachEDunderconsideration.Thereareseveralcommonmethods forobtaininggradientapproximations.First,itisoftenthecasethattheQoIdening theEDaredenedbylinearfunctionalsofasolutiontoacomputationallyexpensive model.Therefore,solvingthecorrespondingadjointproblemsforeachcomponenetof theQoImapisanaordablemethodforapproximatingJacobians[33,56,60,69,53]. ThesolutionoftheadjointproblemsproducesacomputableJacobianapproximation atacostofapproximatelyoneextramodelsolvepercomponentoftheQoImap.That is,foreachsamplein andeachcomponentoftheQoImap,werequiresolution totheforward"problem and theadjointproblemwhichhassimilarcomputational costtoapproximatethegradientofeachcomponentoftheQoImapatapointin However,manycomputationalmodelsmaynothaveadjointsavailable[51].In thisscenario,wearelefttoapproximategradientsusingothertechniques,e.g.,standardnite-dierencetechniques.Thismaybecomputationallyintractablegivena high-dimensionalparameterspaceasthesemethodsrequireatleastdim +1model solvestoobtainagradientapproximationatasinglepointin .Moreover,we maynotbegiventhefreedomtoplacesamplesoptimallytoemploystandardnitedierencetechniques.Forexample,wemaybegivenasetof N samplesin ,e.g., comingfromsamplingaprior,andmustapproximategradientinformationinorder 2 Wenotethat,itiswellknownthattheseKDEtechniquesdonotscalewellwiththedimension ofthedataspace[67]. 75 PAGE 93 todetermineanOED 3 .Inthisscenario,wechoosetoutilizeradialbasisfunctions RBFstoapproximatelocalresponsesurfaces[70,26].Essentially,thesemethods canoperateonanunstructuredsetofsamplestoconstructalocalsurrogateabouta given 2 anddierentiatethesurrogatetoobtainanapproximationtothegradient,seeAppendixAfordetails.Thisis,inmanyways,themostchallengingscenario forimplementationofthemeasure-theoreticapproachtoOED.Hence,intheensuing examples,weassumewedonothaveanadjointandthatwehave N uniformrandom samplesin ,withcorrespondingdata,forwhichwecanusetodetermineanOED. VI.3HighlySensitiveQoI Inthissection,weexploretheabilityofeachOEDmethodtodistinguishbetween EDscomposedofQoIthatarehighlysensitivetochangesinparameters.Webegin withanexampletoillustrateourndings,andfollowwithanexplanation. RemarkVI.3 Wenotethat,inExampleVI.1below,theproleoftheobserveddensityoneachQoIdierssubstantiallyfromthatchosenforthesamemodelinExampleIV.4.ThischoiceismadetohighlightaspeciclimitationoftheBayesian approachtoOED,specically,thatthemethodhasdicultydistinguishingbetween twohighlysensitiveEDswhenlimitedmodelsolvesareavailable. ExampleVI.1Time-DependentDiusion:LimitedModelSolves Weconsideratime-dependentdiusionproblemwithtwouncertainparametersdeningthe thermalconductivitiesofametalrod.SeeExampleIV.4foracompletedescription ofthemodelandproblemsetup. Recall,theexperimentsinvolveplacingtwocontactthermometersalongthefull rodthatcanrecordtwoseparatetemperaturemeasurementsat t f .Thus,thereisa bijectionbetween and Q sopoints x 0 ;x 1 2 canbeusedtoindexthe EDs.FortheconsistentBayesianapproach,welettheuncertaintyineachQoIbe 3 Or,evenworse,tousethese N samplestobothdetermineanOEDandthensolvetheSIPwith thesamesetof N samples. 76 PAGE 94 describedbyatruncatedGaussianprolewithaxedstandarddeviationof 1 E )]TJ/F15 11.9552 Tf 11.88 0 Td [(04 Thischoiceofuncertaintiessuggeststhemeasurementinstrumenthasahighdegree ofaccuracyrelativetotherangeofthedatathatweexpecttoobservebasedonthe priorinformation,i.e., D supp obs D << D D Here,wefocusontheabilityofeachOEDapproachtocorrectlyclassifytwo specicgroupsofEDs:oneoptimalgroupandonesub-optimalgroup.Inparticular, weareinterestedintheabilitiesofeachmethodasafunctionofthenumberofmodel solvesavailable.WechoosetheoptimalgroupofEDstobe10pointschosenrandomly fromasquarewithsidelength 0 : 04 centeredattheindexingpoint : 97 ; 0 : 03 2 andthesub-optimalgrouptobe10pointschosenrandomlyfromasquarewithside length 0 : 04 centeredattheindexingpoint : 9 ; 0 : 7 2 ,seeFigureVI.1. FigureVI.1: ThetwogroupsofEDsshownintheindexingset .Thegroupin thebottomrightoveraredregionistheoptimalgroupofEDsandthegroupnear theupperrightoverablueregionisthesub-optimalgroup.Forreference,weshow thesetwoclustersplottedontopoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 computedusingall5,000available samplesin asdescribedabove. Results 77 PAGE 95 ForbothOEDmethods,weconsiderapproximatingtheESEandEIGvaluesfor increasingnumbersofmodelsolvesupto N =5000 ,whereeachsetofsamplesis drawnfromtheuniformprior.Weassumethatwedonothaveanadjointproblem availabletoapproximate f J i ;Q j g N i =1 20 j =1 ,therefore,weemployaRBFmethodto obtaintherequiredgradientinformation,seeAppendixAfordetails. Foragivensetof N samples,weapproximatetheESEandEIGvaluesforeach ofthe20EDs.Recall,ourgoalistocorrectlyclassifyeachEDasbelongingtoeither theoptimalorsub-optimalgroup.InFigureVI.2,weshowtheapproximateESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 andEIGvalues,foreachofthe20EDs,asafunctionofthenumberofmodelsolves available.NoticethecleardierenceinESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 betweentheoptimalandsub-optimal clustersofEDsforall N .However,fromthisgure,theEIGvaluesofeachofthe 20EDsappeartobethesameforall N InTableVI.1,wechooseoneoptimalandonesub-optimalEDandshowtheEIG valuesforincreasing N .WeseethattheEIGvaluesforthechosenoptimalandsuboptimalEDsarethesame,outtotwelvesignicantdigits,for N 1000 .Thisposes aconcern,astheESEinformsusthattheoptimalclusterofEDsisapproximatelyten timesmoresensitivetoinformativeoftheparametersthanthesub-optimalcluster, i.e.,weexpectanEDfromtheoptimalclustertoscaleasetinthedataspaceof D measureapproximately650unitstoasetin of -measureoneunitandanEDfrom thesub-optimalclustertoscaleasetinthedataspaceof D -measureapproximately 70unitstoasetin of -measureoneunit,seeFigureV.2inChapterVforan illustrationofthisgeometricinterpretationoftheESE.Thus,usinganEDfromthe optimalclustertosolveaSIPreducesuncertainsetsinthedataspacebyanorder ofmagnitudemorein -measurecomparedtousingoneofthesuboptimalEDs. However,theEIGdoesnotdierentiatebetweenthetwogroupsuntilweusemore than1000modelsolves.Evenfor 1000 N 5000 ,theEIGvaluesamongstthetwo clustersarerelativelysimilar.Consider N =5000 ,therelativedierencebetweenthe 78 PAGE 96 twoEIGvaluesisapproximately 0 : 0041% FigureVI.2: leftTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthe20EDsunderconsideration.Noticetheclear dierenceinESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 valuesbetweentheoptimalgroupredandthesub-optimalgroup blue.Theoptimalgrouplookstobejustasingleline,however,thereareactually10 linesplottedhereandtheysimplyliesoclosetogetherthattheyareindistinguishable. rightTheEIGforthe20EDsunderconsideration.Inthisimage,wedonotseea cleardistinctionbetweenthetwogroupsofEDs.Again,thereare10redlinesand10 blueslineshere,theyarejustnotdistinguishable.SeeTableVI.1foraquantitative descriptionoftheseEIGvalues. LemmaVI.1explainswhytheEIGvaluesfortheoptimalandsub-optimalEDs inExampleVI.1arethesamefor N 1000.Consider N =1000,foranyofthe20 EDs,each producedinAlgorithm4containsonlyasinglesampleandtherefore thecorresponding KL Q computedisequaltologandthenEIG=log 6 : 908.Hence,asweincrease N ,weincreasethemaximumvaluetheEIGcantake anditsabilitytodierentiatebetweenhighlysensitiveEDs.Thissuggeststhatina scenarioinwhichmorethanoneEDhasanEIGofapproximatelylog N ,weshould bewaryoftheEIG'sabilitytodierentiateeectivelybetweenEDs.FigureVI.2 showsthattheESEdoesnotsuerfromthissamelimitation. VI.4UnknownProleoftheObservedDensity SomeapplicationslooktodetermineanOEDtosubsequentlyreduceuncertainties inmodelinputparametersfromuncertainobservationaldata without priorknowledge oftheprolesoftheobserveddensityoneachQoI.Forexample,consideracontami79 PAGE 97 N EIGforoptimalEDEIGforsub-optimalED 102.302585092992.30258509299 503.912023005433.91202300543 1004.605170185994.60517018599 3005.703782474665.70378247466 5006.214608098426.21460809842 10006.907755278986.90775527898 20007.600902459547.60073425065 30008.006367567658.00620455542 40008.29404964018.29361794088 50008.517193191428.51684558669 TableVI.1: TheEIGforasingleEDfromtheoptimalclusterandasingleEDfrom thesub-optimalcluster.TheEIGvaluesareidenticaltotwelvesignicantguresfor N 1000.However,fromtheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 valuesinFigureVI.2,weknowthattheoptimal EDisapproximately10timesmoresensitivetoinformativeoftheinputparameters thanthesub-optimalED. nanttransportmodelinwhichweseektoreduceuncertaintiesinthediusivitypropertiesofthedomainbasedonconcentrationmeasurementatsomespeciedpointsin space-time.SupposewemaymeasureeachQoImultipletimesandusethesemultiplemeasurementstodenetheproleoftheobserveddensityoneachQoI.Priorto gatheringthesemultiplemeasurements,itisunlikelythattheproleoftheobserved densityisknown.ThispresentsapotentialissuefortheconsistentBayesianapproach asthismethodrequiresthepriorknowledgeoftheprolesoftheobserveddensityon eachQoIinordertodetermineanOED. Alternatively,withinthecontextofOED,weoftenmustprescribeanexpected utilitysystematicallytoabroadsetofEDs.ThissetofEDsmayincludeEDswhere thestandarddeviationoftheproleoftheobserveddensityislargerelativetothe rangeofthedataspace.Inthefollowingexample,weconsiderbothofthesescenarios: thedependenceoftheEIGontheprolesoftheobserveddensityoneachQoI,and theimpactofhighlyuncertainobserveddensitiesrelativetothe D -measureofthe dataspace.Aftertheexample,weprovideanexplanationofourndings. ExampleVI.2Time-DependentDiusion:UnknownObservedDensityProle Weconsideratime-dependentdiusionproblemwithtwouncertainparametersden80 PAGE 98 ingthethermalconductivitiesofametalrod.SeeExampleIV.4foracompletedescriptionofthemodelandproblemsetup. Recall,theexperimentsinvolveplacingtwocontactthermometersalongthefull rodthatcanrecordtwoseparatetemperaturemeasurementsat t f .Thus,thereisa bijectionbetween and Q sopoints x 0 ;x 1 2 canbeusedtoindextheEDs. Forthisexample,weconsiderthescenarioinwhichwedonotknowtheuncertainties ineachQoI,i.e., inEq. IV.3 isunknown.Moreover,weassumewearegiven limitedresourcestodetermineanOED,specically,wehavejust N =100 model solvesatasetofi.i.drandomsamplesin Asinthelastexample,wespecifytwogroupsofEDs:oneoptimalgroupandone sub-optimalgroup.Wedeneoptimalandsub-optimalintermsoftheESEvalues approximatedusingthesame N =100 i.i.d.randomsamplesthatareavailablefor theconsistentBayesianapproach.Inparticular,weareinterestedintherobustnessof theconsistentBayesianapproachtovaryingprolesoftheobserveddensityoneach QoI.WechoosetheoptimalgroupofEDstobe50pointschosenrandomlyfroma squarewithsidelength 0 : 04 centeredattheindexingpoint : 97 ; 0 : 03 2 ,and thesub-optimalgrouptobe50pointschosenrandomlyfromasquarewithsidelength 0 : 04 centeredattheindexingpoint : 73 ; 0 : 26 2 ,seeFigureVI.3. Results ForbothOEDmethods,weconsiderapproximatingtheESEandEIGvaluesfor N =100 samples.Weassumethatwedonothaveanadjointproblemavailableto approximate f J i ;Q j g N i =1 100 j =1 ,therefore,weemployaRBFmethodtoobtainthe requiredgradientinformation,seeAppendixA. WeapproximatetheESEandEIGvaluesforeachofthe100EDs.Recall,our goalistocorrectlyclassifyeachEDasbelongingtoeithertheoptimalorsub-optimal group.InFigureVI.4,weshowtheapproximateESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 andEIGasafunctionof thestandarddeviation oftheGaussianuncertaintydenedoneachQoI.Noticethe 81 PAGE 99 FigureVI.3: ThetwogroupsofEDsshownintheindexingset .Thegroup inthebottomrightoveraredregionistheoptimalgroupofEDsandtheother groupoverablueregionisthesub-optimalgroup.Forreference,weshowthesetwo clustersplottedontopoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 computedusing N =100availablesamplesin asdescribedabove. 82 PAGE 100 cleardierenceinESEbetweentheoptimalandsub-optimalclustersofEDsforall .ThisisbecausetheESEis not afunctionoftheproleoftheobserveddensityon eachQoI. However,theEIGvaluesareclearlydependenton .Forsmall ,theEIGvalues forallofthe100EDsareindistinguishableinFigureVI.4.Then,thismethodcorrectly classieseachEDaseithersub-optimaloroptimalfor 1 E 03 1 : 5 .Finally,for > 1 : 5 ,theEIGvaluesfortheoptimalgroupofEDslieinthemiddleoftheEIG valuesforthesub-optimalgroupofEDs. InTableVI.2,wechooseoneoptimalandonesub-optimalEDandshowtheEIG valuesforincreasing .Weseethatthesub-optimalEDhasthesameEIGvalueas theoptimalEDfor 1 E 05 1 E 04 ,asmallerEIGvaluefor 1 E 04 < 1 : 5 andalargerEIGvaluefor > 1 : 5 .Moreover,noticethattheEIGvalueforthe sub-optimalEDactuallybeginstoincreaseas increasesbeyond 1 E + 00 .Given verynoisyobservationaldataand,hence,alargeuncertaintyontheQoIdening anED,theEIGpotentiallymisclassiesoptimalandsub-optimalEDsandproduces non-intuitiveresultsregardingtheincreaseoftheEIGastheuncertaintiesonthedata increase. Toprovideamoreglobalillustrationoftheimpactsthat hasontheEIGvalues fortheentiredesignspace,inFigureVI.5,weshowtheEIGovertheindexingset forvariousvaluesof andscatterplotthetop 100 OEDs.Itisclearfrom theseimagesthattheOEDasdenedbytheEIGisafunctionof .Thetop 100 OEDlocationsbeginas,apparently,uniformrandompointsin ,thenbeginto clustertowardsthebottomcenterof ,thenclusterinthebottomrightof andnallysettleintwoseparateregionsnearthecenterof as becomeslarge. AsdepictedinExampleVI.2,theEIGproducesinconsistentOEDresultsasthe proleoftheobserveddensityoneachQoIvaries.Moreover,insomecases,the EIGvalueincreasesasthestandarddeviationoftheproleoftheobserveddensity 83 PAGE 101 FigureVI.4: leftWeshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 asafunctionof toemphsizethatthe ESEisdenedindependentoftheexpectedproleoftheobserveddensityoneach QoI.rightTheEIGforthe100EDsunderconsiderationasafunctionof .The optimalgroupisshowninredandthesub-optimalgroupisshowninblue.Notethat, as increase,theEIGfortheoptimalclusterbecomessmallerthantheEIGforthe sub-optimalgroup. EIGforoptimalEDEIGforsub-optimalED 1E-054.605170185994.60517018599 1E-044.605170185994.60517018599 5E-04 4.60517018599 4.60514348489 1E-03 4.60517018599 4.59878890151 1E-02 4.58943082345 4.39595820507 1E-01 4.15175649436 2.15023400619 5E-01 1.96082600602 0.463244860459 1E+00 0.942029897225 0.359941128198 1.5E+00 0.46543957244 0.377398140826 2.0E+000.241747553646 0.391255373739 3.0E+000.114709270539 0.404324508399 5.0E+000.0925786430422 0.412147506656 1.0E+010.0976835734117 0.415690505556 1.0E+020.101585686731 0.416892047918 TableVI.2: TheEIGforasingleEDfromtheoptimalgroupandasingleEDfrom thesub-optimalgroup.InboldfontwehighlightlargerEIGvalueineachcolumn. FromtheESEvaluesinFigureVI.4,weknowthattheoptimalEDisapproximately 10timesmoresensitivetoinformativeoftheinputparametersthanthesub-optimal ED.However,theEIGvalueforthesub-optimalEDisthesameforsmall ,smaller for inthemiddleoftherange,andlargerfor > 2 : 0 E +00. 84 PAGE 102 FigureVI.5: TheEIGovertheindexingset andthetop100OEDsfor increasing .Forverysmall =1 E )]TJ/F15 11.9552 Tf 12.13 0 Td [(05,weseethatthetop100OEDsappearto berandompointsin .For0 : 1 1 : 0,thetop100OEDsareallclusteredin thebottomrightof .Then,assigmaincreasebeyond1 : 0,thetop100OEDs moveawayfromthebottomrightof andsettleintworegionsnearthecenter ofthespace. 85 PAGE 103 increases.Inotherwords,increasingtheuncertaintyinthedataspacecanleadusto concludethesameQoIhaveimprovedinformationgains,whichiscounter-intuitive. Weillustratethecomplexitiesofthisissuewiththenonlinearsystemintroducedin SectionIII.5: 1 x 2 1 + x 2 2 =1 x 2 1 )]TJ/F20 11.9552 Tf 11.955 0 Td [( 2 x 2 2 =1 Recall,theparameterrangesaregivenby 1 2 [0 : 79 ; 0 : 99]and 2 2 [1 )]TJ/F15 11.9552 Tf 11.371 0 Td [(4 : 5 p 0 : 1 ; 1+ 4 : 5 p 0 : 1]whicharechosenasin[10]toinduceaninterestingvariationintheQoI.We considertwoEDsdenedbythethefollowingQoI: Q = x 1 ; Q = x 2 : Forclarity,ratherthanconsidertheEIGforeachED,weconsiderthe KL Q value forasingle obs D 2O Q chosensothatthecenteroftheparameterspacemapstothe centerofthe obs D foreachED.Weconsiderthe KL Q valuesforeachEDforvarying InFiguresVI.6{VI.13,weshowtheobserveddensitiesonthedataspacedenedby eachED,thecorrespondingpushforwardoftheprior,andtheassociatedposteriorsfor variousvaluesof .For =1E-07,thesupportof obs D containsjustasinglesamplefor eachED.Therefore,thecorrespondingposteriorsalsohavesupportsthatcontainjust asinglesample.Thisyieldsaninformationgainvalue KL Q log0000 10 : 5966. As increaseto0.03,weseereasonableresults,i.e.,astheuncertaintiesonthe observeddensityincreasetheinformationgainvaluesdecreaseforeachED.However, as goesfrom0.03to0.05,weseethe KL Q valuefor Q increase .Thisisnot intuitive,weexpectthatastheuncertaintiesontheobserveddataincreasethatour 86 PAGE 104 informationgainshoulddecrease.Weseethesamepropertyfor Q as increases from0.1to1.0. Toshedlightonthissituation,werefertoLemmaVI.2,whichstatesthat KL Q prior : = KL Q Q prior D : obs D : Astheuncertaintyproleontheobserveddensityincreases,theobserveddensity approachesauniformdensityover D .Thus,theKLdivergenceof obs D from Q prior D quantieshowdierent Q prior D isfromauniformdensity.ThisisobservedinFigureVI.13,whereboththeobserveddensitieson Q and Q arenearlyuniformand Q prior D isclosertouniformover D Q than Q prior D over D Q FigureVI.6: topleftApproximationof using obs D on Q with =1E-07, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E-07, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q VI.5RedundantData 87 PAGE 105 FigureVI.7: topleftApproximationof using obs D on Q with =1E-03, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E-03, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 88 PAGE 106 FigureVI.8: topleftApproximationof using obs D on Q with =1E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 89 PAGE 107 FigureVI.9: topleftApproximationof using obs D on Q with =2E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =2E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 90 PAGE 108 FigureVI.10: topleftApproximationof using obs D on Q with =3E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =3E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 91 PAGE 109 FigureVI.11: topleftApproximationof using obs D on Q with =5E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =5E-02, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 92 PAGE 110 FigureVI.12: topleftApproximationof using obs D on Q with =1E-01, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E-01, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 93 PAGE 111 FigureVI.13: topleftApproximationof using obs D on Q with =1E+00, toprightacomparisonoftheobserveddensity obs D on Q withthepushforwardof theprioron Q .topleftApproximationof using obs D on Q with =1E+00, toprightacomparisonoftheobserveddensity obs D on Q withthepushforward oftheprioron Q 94 PAGE 112 Inthissection,weconsidertheabilityofeachOEDapproachtodierentiate betweenEDsthatarecomposedofQoIcontainingredundantQoIcomponents,i.e., theQoIcomponentsareeithernotGDorverynearlynotGD.ConsiderthetimedependentdiusionproblemdiscussedindetailinChaptersIVandV.InFigureVI.14,weshowtheplotsoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andtheEIGovertheindexingset ofthedesignspacefor N =5000and =0 : 1. FigureVI.14: SmoothedplotsoftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andtheEIGovertheindexingset forthedesignspace Q IntheimagesofFigureVI.14,theline x 0 = x 1 representstheEDsdenedby placingbothsensorsattheexactsamelocationin.Althoughtheremaybeother combinationsofsensorlocationsthatyieldlocallyorgloballyredundantQoI,wefocus onthisregionofthedesignspaceasitisthemostobviousregionofEDsproducing redundantQoI.FortheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,itappearsthatalloftheEDsneartheline x 0 = x 1 havesimilarESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 valuesandarealldeterminedtobeequallysuboptimal.However, theEIGdierentiatesbetweenregionsontheline x 0 = x 1 ,itsuggestsplacingboth sensorsnearthecenterorbothsensorsnearoneoftheendsoftherodismoreoptimal thatplacingthembothaboutone-thirdofthewayfromeitherendoftherod. TheESEcannotbeusedtodierentiatebetweenEDscomposedofredundant QoIbecausewhenthelocallinearapproximationsarelinearlydependent,theESE isinnitesoESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 iszero.TheEIGontheotherhand,distinguishestheutilityof 95 PAGE 113 thesensorlocationsalongtheline x 0 = x 1 becausetheEIGisbasedonthesolutions toSIPs,and,besidestheadditionalcomputationalcomplexityinvolved,solvingthe SIPwithredundantQoIproducesasimilarsolutionassolvingitwithjusttheGD componentsoftheQoImap. InFigureVI.15,weshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 andtheEIGgivenjustasinglesensorto placewithintogatherdata.Weseeinthiscase,bothoptimizationobjectivesshow cleardierencesinutilityamongthedierentlocationsoftherod.TheEIGproduces asimilardescriptionoftheone-dimensionaldesignspacetothatoftheline x 0 = x 1 inthetwo-dimensionaldesignspace.Thisisaniceproperty,asitmaybedicultto eliminateanyEDswithredundantQoIpriortodetermininganOED. FigureVI.15: SmoothedplotsoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 andtheEIGovertheindexingset forthedesignspace Q TheoptimalapplicationoftheESEtoproblemsinwhichweexpectEDsto becomposedofQoIwithredundantQoIcomponentsisatopicofcurrentresearch. TheESKisapossibleremedytothisproblem,asitdescribestheamountofnew" informationprovidedbyeachQoIcomposinganED.Givenanentiredesignspace thatproducesEDswithredundantQoI,alloftheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 valueswouldbeunusually small,ifnotzero,thus,suggestingthatweshouldconsiderEDsdenedbylowerdimensionaldataspaces.Thiscanbeconsideredbothastrengthandaweakness foreachapproach.AlthoughtheESEcannotdierentiatebetweenredundantQoI,it 96 PAGE 114 doestelltheresearcherthattheyhaveconsideredasetofEDsthatcontainredundant QoI,andtheresearchershouldconsiderusinglessQoIcomponentstodenetheEDs. However,inourexperience,theEIGdierentiatesbetweenredundantdataquitewell, butitdoesnotinformifredundantQoIcomponentsdenetheEDs.Thisisatopic ofcurrentresearchandfurtherpursuitofaremedyisbeyondthescopeofthisthesis. VI.6Summary Thisthischapter,weexaminedtheabilitiesofthetwoSB-OEDapproachesproposedinthreecriticalscenarios:limitedmodelsolvesavailabletodeterminean OEDamongasetofEDscomposedofhighlysensitiveQoI,unknownoruncertainknowledgeoftheprolesoftheobserveddensityoneachQoIcomposingtheEDs underconsiderationandprolesoftheobserveddensitywithsupportsthatextend wellbeyondtherangeoftheQoImap,andEDscomposedofQoIcontaining redundantcomponents. AlthoughweillustratedtheinadequacyoftheEIGindierentiatingbetweentwo highlysensitiveEDswhenrestrictedtofewmodelsolves,thereareapproachesthat mayremedythisissue.Forexample,surrogatemodelscanbeusedtogreatlyreduce thesamplingerrorinUQproblemsandarealikelysolutiontotheinadequaciesof theEIGgivenfewmodelsolves. Whentheprolesoftheobserveddensitiesaredenedappropriately,theEIG producesasimilardescriptionofthedesignspacetotheESE.Insomesense,wemust tunetheproleoftheobserveddensityforeachEDsothattheresultingEIGvalues describeanamountof relative informationgainedcomparedtotheotherEDsunder consideration.Wemaythinkofthisastuning todisplaytherelativesensitivitiesof eachEDasopposedtotheutilityofeachEDgivenaspeciedprolefortheobserved densities.Theissueofthesupportoftheobserveddensityextendingwellbeyondthe rangeoftheQoImapunderconsiderationmightbeaddressedpriortoapproximating theEIG,therebyeliminatingEDsthatshowlittlesensitivitytochangesinthemodel 97 PAGE 115 inputparametersbeforeaSB-OEDapproachisemployed. TheissueofredundantQoImaybethemostpromisingpathforfuturework stemmingfromthischapter.Oftentimes,applicationsstrivetogathermultitudesof dataforsubsequentuseininferringmodelinputparameters.Insomecases,this abundanceofdataislledwithredundantinformation,i.e.,multipleQoIthatare sensitivetothesamedirectionswithintheparameterspace.AlthoughtheEIGappearstodierentiatebetweentheutilitiesofEDscontainingredundantinformation, usingahigherdimensionaldataspacethannecessaryincreasescomputationalcosts andincreasestheriskthatwegathersetsofinfeasibledata. Inthischapter,wehaveprovidedstrongargumentsforsupportingtherobust natureofthemeasure-theoreticapproachtoSB-OED.Thisapproachquantiesnot onlytheESEdiscussedhere,butalsotheESKdiscussedinChapterV.TheESK describesthecomputationalcomplexityofagivenSIPandbyminimizingtheESK weminimizetheexpectederrorsinourapproximationtoagivenSIP.However,we donot eliminate theerrorinourapproximation.AnexactsolutiontoaSIPinvolving solutionstoPDEsis,ingeneral,unattainable,however,wemayimprovethesolution toaspecicgroupofSIPs.Inthefollowingchapter,weproposeareformulationof theSIPforapplicationswithepistemicuncertaintiesinmodelinputparameters. 98 PAGE 116 CHAPTERVII THECOMPUTATIONALCOMPLEXITYOFTHESIP InChapterV,weintroducedtheESKofaQoImapanddiscusseditsrelationship tothecomputationalcomplexityinaccuratelyapproximatingthesolutiontoaSIP [14].AlthoughaQoImapwithsmallESKhasoptimalpropertiesforsetapproximation,ingeneral,thereisstillerrorinanyapproximationtothesolutiontoaSIP.In thischapter,wecontinuetheexplorationofthecomputationalcomplexityofagiven SIP.InSectionVII.1,weconsidertheimpactthiserrorhasinthescenarioofparameteridenticationunderuncertainty.Intheseproblems,thereexistsatrueparameter responsibleforobserveddata,however,duetomeasurementerror,modelinadequacy, limiteddata,etc.,wearelimitedtodescribingsetsof probable parametersresponsible fortheobserveddata.TheaccuracyofthesolutiontoagivenSIPimpactstheability tocorrectlyidentifythistrueparameterwithinaneventofnonzeroprobabilityin WeconsideranalternativeapproachforformulatingandsolvingtheSIPsuchthat, undercertainassumptions,weguaranteethatthetrueparameteriscontainedwithin thesupportofanapproximatedsolutiontotheSIP, V ,i.e., P )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [( true 2 supp V =1 : FornonlinearQoImaps,thisapproachislimitedinitsabilitytoproduceatight coveringoftheinverseimage,however,thedevelopmentofthisapproachprovides insightintotwocriticalareasofthemeasure-theoreticmethodforSB-OED.InSectionVII.2,weprovideacleardenitionofa single-objective optimizationproblem thatconsidersboththeprecisionandaccuracypropertiesofaSIP.Thisobjectiveis, implicitly,afunctionofthesensitivitiesoftheQoImap,thenumberofmodelsolves available,andthedimensionoftheparameterspace.InSectionVII.3,wedescribean alternativecharacterizationofthecomputationalcomplexityofagivenSIPinterms ofaboundontheerroroftheapproximationtoaninverseimage. 99 PAGE 117 VII.1AReformulationoftheSIPforEpistemicUncertainty AsdescribedinChapterIII,oneapproachforapproximatingthesolutiontoaSIP istoutilizeAlgorithm3.Essentially,thisalgorithmdiscretizes withofteni.i.d. randomsamples i N i =1 thatimplicitlydenetheVoronoitessellation fV i g N i =1 Then,probabilitiesonthediscretizeddataspace,determinedbytheobservedprobabilitymeasure,areproportionedouttotheappropriate V i .Weconsiderthesimple caseofalinearQoImapandauniformobserveddensitydenedonsomegeneralized rectangle E 2B D .Inthiscase,theapproximationtothesolutiontotheSIP,asdeterminedbyAlgorithm3,issimplyaprobabilitymeasurethatisconstantoversome unionofVoronoicellsapproximatingtheinverseimageof E ,andzeroeverywhere else.Inotherwords,theSIPreducestoaccuratelyapproximatingthesetdening thesupportoftheinverseimage. Thenitesamplingof maycausethisapproximationtobeapoorapproximationoftheactualinverseimage,seeFigureVII.1left.Suchanapproximationmay notincludethetrueparameterresponsiblefortheobserveddata.Therefore,wedevelopamethodtoproduceacoveroftheexactinverseimage,implicitly,byinating theuncertaintiesoneachobservedQoItoreectthediscretizationerrorexpectedin theapproximatesolutiontotheSIP.Thatis,wedenean ~ E 2B D suchthatthe approximation of Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E covers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E and,thus,isguaranteedtocontainthetrue parameter. VII.1.1CoveringtheExactInverseImage SupposewearegivenalinearQoImap,aVoronoitessellationoftheparameter space fV i g N i =1 denedimplicitlybyasetofi.i.d.randomsamples i N i =1 ,and anapproximationto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E forsomegeneralizedrectangle E 2B D .Asseenin FigureVII.1,thisapproximationmayexcluderegionsoftheexactinverseimage. 100 PAGE 118 Thus,thetrueparameterresponsiblefortheobserveddatamaynotbecontained withinthesupportoftheapproximateinverseimage. FigureVII.1: leftIngreen,weshowtheapproximateinverseimagewiththe givenVoronoitesselationof .Theexactinverseimageisshowninblue.Noticethe trueparameter true inrediscontainedwithintheexactinverseimage,however,itis notcontainedwithintheapproximation.rightInblue,weshowtheevent E 2B D thatdenes Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,thetruedatuminred,andtheobserveddatuminblackthatis responsibleforthedenitionof E 2B D Thisapproximateinverseimage,denoted Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,isdeterminedbyAlgorithm3. Inthiscase,thealgorithmsimplymapseach i N i =1 to D with Q and,if Q i 2 E then V i ispartoftheinverseimage,otherwise V i isnotpartoftheinverseimage. Theunder-approximatedregionsof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E resultfromthecoarsediscretizationof by fV i g N i =1 Tomodify Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E sothatitcoverstheexactinverseimage,wemustensurethat any V i suchthat V i Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E 6 = ; ispartoftheapproximateinverseimage.One approachtoobtainthisgoalistoextendtheboundariesof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E outward,bysome appropriatedistance,sothattheVoronoiapproximationtothisinatedparallelogram covers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,seeFigureVII.2. Thedistanceinwhichweextendtheboundariesof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isdeterminedbythe 101 PAGE 119 maximumradiusoftheVoronoicellsinthetessellation R N =max i =1 ;:::;N radius V i ; VII.1 wheretheradiusofaVoronoicellisdenedbytheradiusoftheminimumcircumscribingballcenteredat i radius V i :=sup 2V i d i ; : Wedenethis inated inverseimageimplicitlybasedonaninationof E 2B D FigureVII.2: Inpurpleweshow Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ,inyellowweshowtheinatedinverse imageresultingfromextendingtheboundariesoutward,andingreenweshowthe approximationtotheinatedinverseimagethatcovers Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Notethat,this inationisdeterminedbyextendingtheboundariesoutwardeachdirectionbya distanceof R N ,themaximumradiusoftheVoronoicellsinthetesselation. VII.1.2Inating E 2B D toImplicitlyDeneaCover Theinationof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ischosensothattheapproximationtothisinationcovers theoriginal Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Here,wediscussanapproachtoimplicitlydenethisination basedoninating E 2B D to ~ E 2B D sothat Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E denesthisexactinationand, thus,theapproximationto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E ,denotedby Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E ,covers Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Recall,a Voronoicellintheapproximationofaninverseimagebelongstotheapproximation 102 PAGE 120 becauseitscenter i ismappedintothegeneralizedrectangleofuncertainty E 2B D i.e., Q i 2 E .Thus,wemayimplicitlydenethisinationbyinatingthe generalizedrectangleofuncertaintyinthedataspace E to ~ E ,sothattheboundaries of Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E areeachmovedoutwardby R N fromtheboundariesof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .We determinehowmuchtoinate E ineachcoordinatedirectionusingtheJacobianof themap Q Recall,weassumethat Q isalinearmapsothereexistsa d n matrix J ,such that Q = J .Letthe i throwof J bedenotedby J i ,then k J i k informsofusthe amount Q i changesas movesinthedirectionofthe n -dimensionalvectordenedby thisrow,moreover,this n -dimensionalvectorisorthogonaltothetwoboundariesof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E denedby Q i .Toextendeachoftheseboundariesoutwardby R N ,weextend thecorrespondingboundariesof E outwardby k J i k R N .Wedothisforeachofthe d componentsoftheQoImaptoobtain ~ E 2B D suchthat Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E covers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,see FigureVII.3. Thisapproachdoesnotproduceaminimumcoverof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E withthegiven Voronoitesselation fV i g N i =1 .IntermsofnonlinearQoImaps,wehavefoundthis methodgenerallyproducesalargecoverof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,solargethat,insomecases,it isnotsubstantiallymoreusefulthansimplypropogatingtheoriginaluncertaintyof theparametersforwardtomakeaprediction.However,thereareotherapproaches availablethatlookpromisinginregardstoproducingamuchtighterandmoreuseful coverof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E fornonlinear Q .Forexample,convolutionscanbeusedinsignal processingtosmoothnoisydata.Inthissameway,itappearsaconvolutioncanbe usedtosmooth Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E carefullysothattheresultingsmoothedinverseimagecovers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E .However,theresultsfromthissectionhavestillprovidedvaluableinsight intothedenitionofasingle-objectiveoptimizationproblemseeSectionVII.2denedintermsoftheQoImap,numberofsamples,anddimensionof .Also,the resultsinthissectionhaveinspiredanalternativedescriptionofthecomputational 103 PAGE 121 FigureVII.3: topleftTheinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E andtheapproximation.top rightThe E 2B D deningtheuncertainitesontheobserveddata.bottomright Theinated ~ E 2B D determinedbytheJacobianof Q and R N .bottomleftThe exactinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,theinatedinverseimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E ,andtheresulting approximationto Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E thatcovers Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E 104 PAGE 122 complexityofagivenSIPintermsofaboundonthesetapproximationerrorofthe resultinginverseimageseeSectionVII.3.Weleavetheexplorationofconvolutions toproducetightcoversof Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E fornonlinearmapsandarbitrary E 2B D tofuture work. VII.2Single-ObjectiveOptimizationProblem InChapterV,wedenedthemeasure-theoreticSB-OEDproblemwithascalarizedmulti-objectiveoptimizationproblemwhereweminimizeaweightedsumofthe ESEandESKtodetermineanOED.However,wedidnotprovideareliablemethod fordetermininganoptimalweight inDenitionV.12.Intuitively,thisoptimal weightshouldbeafunctionofthenumberofsamplesavailableandthedimension oftheparameterspace,asthesequantitieswilllargelyimpactourabilitytoaccuratelyapproximateagiveninverseimage.Here,weusetheapproachintroduced inSectionVII.1toimplicitlyscalarizethemulti-objectiveproblem.Theresulting scalarizationisafunctionoftheQoImap,thenumberofsamplesavailable,andthe dimensionoftheparameterspace. Essentially,weutilizetheobservationsofSectionVII.1toquantifytheutility ofagivenEDintermsofthemeasureoftheinatedinverseimage Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E whose approximationyieldsacoverof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .WenotethatweperformthisSB-OED methodwiththeanticipationofsolvingtheoptimalSIPinthestandardway.In otherwords,oncewedetermineanOED,wedonotinatetheuncertaintieson theproleofthegivenobserveddensitytoensureacoverofthesolutiontotheSIP. Instead,weutilizethisapproachtoquantifylocalgeometricpropertiesofapossibly nonlinearQoImap.ThiscomputationissimilartocomputingtheESE.Weassume that E 2B D isgivenbyageneralizedrectangle,however,wequantifythe ination eect usingtheinated ~ E 2B D introducedabove.Thus,theresultingformforthe localinationeectofagiven Q nowhasa D ~ E inthenumerator,andthis ~ E is determinedbytheJacobianof Q ,andthusdependsonthelocallinearapproximation 105 PAGE 123 oftheQoImapunderconsideration. VII.2.1Ination AssumewearegivenanEDdenedbytheQoImap Q .GiventheJacobianof Q atsome 2 ,denoted J ;Q ,thelengthsofthesidesofthecorresponding ~ E denoted~ w k; ,aredeterminedby ~ w k; = w k +2 R N k J k; k ; wherethe w k arethewidthsoftheoriginalgeneralizedrectangleofuncertainty E 2 B D ,and J k; isthe k throwof J ;Q .TheLebesguemeasureof ~ E isgivenby D ~ E = d Y k =1 ~ w k; = d Y k =1 w k +2 R N k J k; k : VII.2 Notethat, D ~ E isafunctionoftheQoImapunderconsideration, 2 ,and R N denedinEq.VII.1. R N isafunctionofthenumberofsamplesusedtoconstruct thetessellation,andisimplicitlyafunctionofdim ,asweexpectatessellation usingthesamenumberofsamplesinahigherdimensionalspaceto,ingeneral,have alargermaximumradiusoftheVoronoicells.Theabovemotivatesthefollowing DenitionVII.1 Forany Q 2Q and 2 ,wedenethe localinationeect of Q as I Q = D ~ E Q d k =1 k ; where f k g 1 k d arethesingularvaluesoftheJacobian J ;Q and ~ E istheinated generalizedrectangledescribedabove.The expectedinationeectEIE isgiven by H I Q = 1 Z 1 I Q d )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 : AswiththeESEandESK,wegenerallyapproximatetheEIEusingasetofi.i.d. 106 PAGE 124 randomsamples i N i =1 andcomputing H I Q H N I Q := 1 N N X i =1 1 I Q i )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 : VII.3 Wenotethat,thissingle-objectivedoesrelatetotheESEinanotableway.As thenumberofsamplesincreases,themaximumradiusofthevononoicellsgoesto zeroandtheEIEreducestotheESEscaledby D E .Formally TheoremVII.2 Assume Q : !D is B -measurableandGD.Let i N i =1 bea setofi.i.d.samplesinthecompact drawnfromaprobabilitydensity f suchthat f > 0 forall 2 .Then,forany 2 lim N !1 I Q = D E M Q : VII.4 Proof: In[61,39]itisshownthatlim N !1 R N =0.Thus, lim N !1 I Q =lim N !1 D ~ E Q d k =1 k =lim N !1 Q d k =1 w k +2 R N k J k; k Q d k =1 k = Q d k =1 w k Q d k =1 k = D E Q d k =1 k = D E M Q : Thisfollowsintuition,givenascenarioinwhichwecanaordarbitrarilymany modelsolves,wesimplywishtominimizetheESEanddonotcareaboutcomputationalcomplexitybecausewehavemanymodelsolvesavailabletoapproximatethe solutiontotheSIP.InExampleVII.2,weshowtheEIEforvarious N andnotethat, 107 PAGE 125 forsmall N ,therearesimilaritiesbetweentheEIEandtheESK. Wenotethat,unliketheESE,theshapeandmeasureofthegeneralizedrectangle E 2B D doimpacttheEIE.Specically,theproductofthesidelengthsof ~ E implies thenumeratorisnolongeraconstantforeachEDevenwhenthe E 2B D ineach dataspacehasthesameshapeandmeasure.Thismeansthat,inordertousethe EIEasanoptimizationcriterion,wemustspecifytheshapeof E 2B D ,i.e.,wemust havepriorknowledgeoftheproleoftheobserveddensityoneachQoI. ExampleVII.1TwoLinearEDs Inthisexample,weillustratetheinationeffectwithtwolinearmaps Q a and Q b denedby Q a = 2 6 4 10 01 3 7 5 ;Q b = 2 6 4 0 : 5051 : 178 1 : 1780 : 505 3 7 5 ; whereweassumethat E 2B D isgivenbyasquarewithsidelengths0.5.Notethat, inthiscase,theQoImapsarelinear,thus,weneednotapproximateanygradients oraveragethescalingeect,skewness,orinationeectovertheparameterspace. TheseQoImapswerechosensothatthescalingeectof Q b islessthanthescaling eectof Q a ,specically, 0 : 883 M Q b PAGE 126 weshowrepresentativeinverseimagesofachosen E 2B D undereachoftheQoI maps. InFigureVII.5,weshowtheinverseoftheinationeectofeachQoImapasa functionofthenumberofsamplesavailable.Withfewsamplesavailable,theination eectdetermines Q a ismoreoptimalthan Q b ,despitethefactthat M Q b PAGE 127 FigureVII.5: Theinverseoftheinationeect,forboth Q a and Q b ,asa functionofthenumberofsamplesavailable.Although M Q b PAGE 128 computationfordierent N isthevalueof D ~ E whichisgivenbyEq. VII.2 using adierentsetof N samplesfromthoseusedinthecomputationsoftheJacobians.In otherwords,foragiven N ,wegenerate N samplesandapproximate R N .Then,we usethexedsetof ^ N samplesandcompute f Q k i g ^ N i =1 10000 k =1 .Weapproximate f J i ;Q k g ^ N i =1 10000 k =1 usingaRBFinperpolationmethod,seeAppendixAfordetails. Wecalculate f H ^ N I Q k g 10000 k =1 usingEquation VII.3 InFigureVII.6,weshowtheresultingdescriptionsofthedesignspacefor N = 1 E + k for k =0 ; 1 ;:::; 5 .For N =1 E +0 ,theEIE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 resemblestheESK )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 seen inFigureV.5.Wedonotsuggestthatas N 0 theEIEconvergestotheESK becausetheESKisindependentofthemeasureoftheinverseimageandEIEisnot, however,thesimilaritiesarenotable.Asweincreasethenumberofsamplesavailable, theEIE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overthedesignspacechangesnoticeably.For N =1 E +5 ,theEIE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 looksverysimilartotheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,however,itdeterminestheOEDtobeinadierent locationthanthatdeterminedbytheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .Thisillusratesthepointmadeabovethat as N !1 ,theEIEconvergestotheESEmultipliedby D E VII.3ABoundontheErrorofSetApproximations Here,webuildontheideasfromSectionVII.1.Wemayconsiderthecover describedabovetobeanoutersetapproximationoftheregioninwhichsetapproximationerrormayexist.Inthesameway,wemay deate theinverseimageand deneaninnersetapproximationoftheregioninwhichsetapproximationerrormay exist.Thisinspiresthetopicofthissection:determiningaboundontheerrorofan approximationtoaninverseimage. Wefocusentirelyonsetapproximation,i.e.,wedenetheerrorofthesolution toaSIPtobethesymmetricdierencebetweenthesupportoftheexactinverse imageandthesupportoftheapproximateinverseimage.Thus,weassumethe supportoftheexactinverseimagedoesnotspantheentireparameterspace,i.e., 111 PAGE 129 FigureVII.6: TheEIE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingset forthedesignspace Q for variousnumbersofsamples N .Inthetopleft,theEIE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 looksverysimilartothe ESK )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 showninFigureV.5.Asweincreasethenumberofsamples,thelocationof theOEDdoesnotchange,however,thegeneraldescriptionof Q changesnoticably. WebegintoseesimilarfeaturestothatoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 seeninFigureV.4. 112 PAGE 130 Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isproperwhichalsosuggests E D isproper.Wedenetheerrorin anapproximationofasetusingthesymmetricdierence. DenitionVII.3SetApproximationError The error inanapproximation, ^ A ,to A 2B isgivenby Error ^ A = A 4 ^ A ; where istheLebesguemeasureon ,and 4 isthesymmetricdierenceoperator. VII.3.1InnerandOuterErrorBounds Asdescribedabove,assuming Q isalinearmapand E 2B D ageneralizerectangle,wemayinate E 2B D to ~ E 2B D sothatweguarantee Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E seeEq.VII.2.Wemaythinkofthisinatedimage Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E asanoutersetapproximationoftheregionof inwhichsetapproximationerrorto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E mayexist. Similarly,wemay defalate E 2B D to E ~ 2B D sothat Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ isaninnersetapproximationoftheregionof inwhichsetapproximationerrorto Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E maylive. Thus, Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ,andaboundontheerrorisgivenby Error Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E )]TJ/F20 11.9552 Tf 11.956 0 Td [( Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ where Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E istheapproximationto Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E istheinatedinverseimage, and Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ~ inthedeatedinverseimage.Formally, TheoremVII.4 Let R n and D R d ,with d n .Let ; B ; and D ; B D ; D bemeasurespacesusingtheBorel -algebra,where and D arethe Lebesguemeasuresdenedon and D ,respectively.Let Q : !D belinearand E 2B D ageneralizedrectangle.Let i N i =1 beasetofsamplesin implicitly deningtheVoronoitessellation fV i g N i =1 .Then,usingthediscretization fV i g N i =1 and Algorithm3,theerrorintheapproximation Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E to Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E isboundedby Error Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E )]TJ/F20 11.9552 Tf 11.955 0 Td [( Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ : VII.5 113 PAGE 131 Proof: Theboundfollowsfromtheobservationthat Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ~ Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E whichimplies Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E 4 Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ~ E n Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E ~ whichimplies Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 ~ E )]TJ/F20 11.9552 Tf 11.955 0 Td [( Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E ~ : WeillustratethebounddescribedinTheoremVII.4intheexamplebelow. ExampleVII.3Linear ConsiderthelinearQoImap Q denedby Q = 2 6 4 0 : 2490 : 249 0 : 249 )]TJ/F15 11.9552 Tf 9.299 0 Td [(0 : 0415 3 7 5 : First,wefocusontheerrorinapproximationsmadetotheinverseimageofasquare withsidelengths0.1withauniformgridof N samples.InFigureVII.7,weshow theapproximateinverseimage,theexactinverseimage,theinnerandouterset approximationsoftheerror,andtheerror.Intheleftcolumn,theexactinverse imageisshowninblueanditsapproximationingreen.Intherightcolumn,the errorisshowninred,theoutersetapproximationinorange,andtheinnersetapproximationinbeige.Weseethatas N increases,boththeerrorandthebound ontheerrorbecomeconnedtoasmallerregionaroundtheboundaryoftheexact inverseimage.InTableVII.1,weshowtheerrorandtheboundontheerrorfor N =100 ; 192 ; 400 ; 784 ; 1600 samplesonaregulargrid. Next,weconsidertheerrorboundfor N =100 ; 192 ; 400 ; 784 ; 1600 uniformrandomsamples.InFigureVII.8,weshowtheapproximateinverseimage,theexact 114 PAGE 132 NError Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Errorbound 1000.04560.234 1920.03190.168 4000.02380.117 7840.01330.0839 16000.01150.0587 TableVII.1: TheMCapproximationto Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheerrorboundgiven inEq.VII.5usingaregulargridofsamples. inverseimage,theinnerandoutersetapproximationsontheerror,andtheerror. Intheleftcolumn,theexactinverseimageisshowninblueanditsapproximationin green.Intherightcolumn,theerrorisshowninred,theoutersetapproximationin orange,andtheinnersetapproximationinbeige.Weseethatas N increases,both theerrorandtheboundontheerrorbecomeconnedtoasmallerregionaroundthe boundaryoftheexactinverseimage.InTableVII.1,weshowtheerrorandthebound ontheerrorfor N =100 ; 192 ; 400 ; 784 ; 1600 uniformrandomsamples. NError Q )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 E Errorbound 1000.04770.517 1920.03290.388 4000.03090.263 7840.02140.1971 16000.01360.146 TableVII.2: TheMCapproximationto Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheerrorboundgiven inEq.VII.5usinguniformrandomsamples.. VII.4Summary Weconcludethissectionwithseveralpromisingdirectionsforfuturework.First, theinabilityofthecoverdescribedinSectionVII.1tobetightaroundtheboundaryof theinverseimagefornonlinearQoImapssuggestsanalternativeapproachtocovering theseimages.AsdiscussedattheendofSectionVII.1,convolutionsappeartobean appropriatetooltoresolvethisissue.Ineachoftheabovesections,theresultsare 115 PAGE 133 intimatelytiedtotheboundaryoftheinverseimage.Forexample,inSectionVII.3, weobservetheregionthatboundstheerrortoappeartobestronglydetermined bythepropertiesoftheboundaryoftheinverseimage.Thissuggeststheexpected errorinsetapproximationsmaybedeterminedasafunctionofthemeasureofthe boundaryofthesetasalower-dimensionalobject.Thepursuitofthistopicisbeyond thescopeofthisthesis,however,wenotethatinitialexplorationoftheboundaries ofparallelepipedsproducesasimilarscalarizationtothemulti-objectiveoptimization problemdescribedinSectionVII.2. 116 PAGE 134 FigureVII.7: TheMCapproximationto Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheouterandinner setapproximationusingaregulargridofsamplesfor N =192 ; 400 ; 784 ; 1600from toptobottom 117 PAGE 135 FigureVII.8: TheMCapproximationto Error Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E andtheouterandinner setapproximationusinguniformrandomsamplesfor N =192 ; 400 ; 784 ; 1600from toptobottom. 118 PAGE 136 CHAPTERVIII AGREEDYIMPLEMENTATIONFORSB-OED Oftentimes,applicationsyieldhigh-dimensionalparameter,data,anddesign spaces.Thesehigh-dimensionalspacesmakeuncertaintyquanticationdicultin manyways,e.g.,accurateapproximationofasolutiontotheSIP,thepushforward oftheprior,andtheOEDbecomecomputationallyexpensive.Here,wefocusonthe OED.Insomecases,itmaybeinfeasibletoexhaustivelysearchthedesignspaceto determinetheOED.Forexample,givenanapplicationwithabudgetofgathering nineQoImeasurementsfromasetof100possibleQoIyields )]TJ/F18 7.9701 Tf 5.479 -4.379 Td [(100 9 =1 : 9 E +12possibleEDs.Ratherthanacombinatorialsearchofthedesignspace,weproposeagreedy algorithmtoavoidthiscomputationalexpense. VIII.1AGreedyAlgorithm WedescribeagreedyalgorithmforSB-OED.WechoosetofocusontheESE, however,thisalgorithmisappropriateforanyoftheoptimizationcriteriadiscussed above.First,wecomputetheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 foreachoftheEDsdenedbyasingleQoI.The QoIthatdenestheEDwiththegreatestESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 isthentheQoIdeningtheOED forasingleQoI. Given thisOEDforasingleQoI,wethencomputetheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 forthe EDsdenedbyatwo-dimensionalvector-valuedQoIwhereoneofthecomponents istheOEDforasingleQoI.Again,thegreedyoptimalpairofQoIisgivenbythe EDfromthisdesignspacewiththegreatestESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 .Thealgorithmcontinuesin thiswayuntilthedimensionofthevector-valuedQoImapmatchesthedimensionof thedesireddataspace D thatisspeciedatthebeginningofthealgorithm.This issummarizedinAlgorithm5wherewelet Q d denoteadesignspacedenedby vector-valuedQoImapsofdimension d wheretherst d )]TJ/F15 11.9552 Tf 12.041 0 Td [(1componentsarechosen bythepreviousstepsinthealgorithm. SomegreedySB-OEDmethodsgatherthedataforthecurrentED,solvean inverseproblem,andthenusetheresultingsolutiontotheinverseproblemtoinform 119 PAGE 137 theexpectationdeningtheESEforthenextstep.Wenotethat,thealgorithm proposedheredoes not requirethesolutionofaSIPbeforedeterminingthenext optimalQoI.Ateachstep,wecomputetheexpecationintheESEusingthesameset ofi.i.d.samplesin Algorithm5: AGreedyAlgorithmforSB-OED 1.Specifyadesireddim D 2.Let Q denoteallpossibleEDscomposedofasingleQoI. for d =1 ;:::; dim D do Q opt =argmax Q 2Q d ESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 Q if d PAGE 138 where=[ )]TJ/F15 11.9552 Tf 9.299 0 Td [(0 : 5 ; 0 : 5] 2 representsanalloyplate, =1 : 5isthedensityoftheplate, c =1 : 5istheheatcapacity, isthethermalconductivity,and S isaGaussian sourcewiththefollowingsourceparameters: x src =[0 ; 0]isthelocation, A =50 istheamplitude,and w =0 : 05isthewidth.Welet t 0 =0, t f =2,andimpose homogeneousNeumannboundaryconditionsoneachboundaryoftheplate. Supposetheplateismanufacturedbyweldingtogetherninesquareplatesof thesameshapeandofsimilaralloytype,seeFigureVIII.1.However,duetothe manufacturingprocess,theactualalloycompositionsmaydierslightlyleadingto uncertainthermalconductivityproperties ineachofthenineregionsoftheplate, denotedby 1 ;:::; 9 .Thatis,welet =[0 : 01 ; 0 : 2] 9 Giventhethermalconductivityvaluesforeachofthenineregionsoftheplate, weapproximatesolutionstothePDEinEq.VIII.1usingthestate-of-the-artopen sourceniteelementsoftwareFEniCS[3,47]witha40 40triangularmesh,piecewise linearniteelements,andaCrank-Nicolsontimesteppingscheme. FigureVIII.1: Thedomainisconstructedbyweldingtogetherninesquareplates, eachofwhichhasconstant,butuncertain,thermalconductivity VIII.2.2ChooseOneSensor Supposetheexperimentsinvolveplacingninecontactthermometersontheplate thatcanrecordnineseparatetemperaturemeasurementsat t f .Wediscretizeusing 121 PAGE 139 10,000uniformrandompoints, f x k 0 ;x k 1 g 10000 k =1 .Thissetofpointsservesastheset ofpossiblesensorlocationsQoIin,fromwhichwemustchoosenine.TheOED isgivenbytheEDthatmaximizestheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 ,however,giventhisdiscretizationof ,thereare )]TJ/F18 7.9701 Tf 5.48 -4.378 Td [(10 ; 000 9 =2 : 7 E +30possibleEDs.Inordertoavoidthiscombinatorial computationalexpense,weuseAlgorithm5toapproximatetheOEDfromthisset ofEDs. First,weconsider Q ,thespaceofEDsdenedbyplacingasinglecontact thermometerontheplatethatcanrecordthetemperatureat t f .Thus,thereisa bijectionbetweenand Q sopoints x 0 ;x 1 2 canbeusedtoindextheEDs.We let Q k = u )]TJ/F15 11.9552 Tf 5.479 -9.684 Td [( x k 0 ;x k 1 ; t f foreach k .These Q k denoteallofthepossiblecomponents thatmakeuptheQoImapsthatdenetheEDsforthisrstsensorlocationandall subsequentsensorlocations.Thus,foreach Q k 2Q Q k = Q k .Wegenerate 1,000samples, f i g 5000 i =1 ,in andapproximate f J i ;Q k g 1000 i =1 10000 k =1 usinganRBF interpolationmethod,seeAppendixAfordetails.Wenotethat,theseJacobiansare actuallygradientsanddonotneedtobecomputedagainafterthis.Wecalculate theESEforeach Q z 2Q usingEq.V.17. InFigureVIII.2,weshowtheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingsetforthecurrent designspace Q .WeobservethattheOEDislocatedatthecenterorthedomain, nearthelocationofthesource.Thechosenmanufacturingofthemetalplateby weldingtogetherninesquareplatesisvisiblewiththissimulateddata.Thus,wehave determinedtheoptimallocationforasinglesensorandwexthisdecisionthroughout therestofthenextsection.DenotethisEDas Q opt 2Q VIII.2.3ChooseKSensors Now,weconsider Q ,thespaceofEDsdenedbyplacingtwocontactthermometersontheplatethatcanrecordthetemperatureat t f ,wheretherstcontactthermometerisplacedattheoptimalsinglesensorlocationgivenaboveand thesecondcontactthermometerisplacedatanyofthe10,000possiblesensorlo122 PAGE 140 FigureVIII.2: TheESE )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 overtheindexingsetforthedesignspace.Noticethe OEDisshowninwhite,nearthecenterofthedomainwherethesourceislocated. 123 PAGE 141 cations.Thus,thereisagainabijectionbetweenand Q .Welet Q = Q opt 2Q ;Q z : Q z 2Q .WecalculatetheESEforeach Q z 2Q using Eq.V.17. InFigureVIII.3top,weshowtheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 overtheindexingsetforthecurrent designspace Q .WeobservethattheOEDisapairofsensors,onedenedby Q opt 2Q ,theotherlocatedtotheleftofthecenterofthedomainnearthe boundaryoftwooftheplatesthatwereweldedtogether.Wenotethat,itappears therearefourlocationsthatwouldproducesimilarESEvalues:totheleft,top,right, andbottomoflocationofthesource.Thisisnotsurprisingduetothesymmetryof theproblem.ItisinterestingthattherearelocalminimumintheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 aswemove fromthecenterofthedomainoutwardtowardstheboundaries.Wehavedetermined thelocationfortwosensorsandwexthisdecisionthroughouttherestofthissection. DenotethisEDas Q opt 2Q Inthisway,wecontinuetoplacethenextoptimalsensorin,seeFiguresVIII.3 {VIII.6.Ineachgure,thenextoptimalsensorlocationisshowninwhite,and thepreviouslychosensensorlocationsareshowninblack.Noticethatafterasensor inplacedinaregion,inthesubsequentgures,theregionaroundthatsensorhas relativelylowESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 values. 124 PAGE 142 FigureVIII.3: topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensors areshowninblack,andtheoptimallocationforthenextsensorisshowninwhite atthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthe previouslyplacedsensorsareshowninblack,andtheoptimallocationforthenext sensorisshowninwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 125 PAGE 143 FigureVIII.4: topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensors areshowninblack,andtheoptimallocationforthenextsensorisshowninwhite atthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthe previouslyplacedsensorsareshowninblack,andtheoptimallocationforthenext sensorisshowninwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 126 PAGE 144 FigureVIII.5: topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensors areshowninblack,andtheoptimallocationforthenextsensorisshowninwhite atthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthe previouslyplacedsensorsareshowninblack,andtheoptimallocationforthenext sensorisshowninwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 127 PAGE 145 FigureVIII.6: topTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthepreviouslyplacedsensors areshowninblack,andtheoptimallocationforthenextsensorisshowninwhite atthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 .bottomTheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 for Q .Noticethatthe previouslyplacedsensorsareshowninblack,andtheoptimallocationforthenext sensorisshowninwhiteatthemaximumoftheESE )]TJ/F18 7.9701 Tf 6.587 0 Td [(1 128 PAGE 146 CHAPTERIX CONCLUSIONS IX.1Summary Thisthesishasdevelopedabroadsetoftoolstoemploytoecientlydetermine anOEDinthecontextofSB-OED.Webeganwithacontributiontothecurrentstate ofBayesianSB-OEDinwhichweemploytherecentlydevelopedconsistentBayesian approachtosolvingSIPstoecientlyintegrateovertheobservableQoIspaceduring computationoftheEIGofagivenED.Thedevelopmentofthismethodrequiredthe adaptationofanobserveddensityspacetoconformtotheassumptionrequiredinthe consistentBayesianformulation,anissuethatisavoidedintheclassicalBayesianSBOEDformulation.WefollowedthiswithanovelperspectiveonSB-OEDinspiredby theset-basedframeworkofthemeasure-theoreticapproachtoSIPs.Thisperspective proposedadenitionoftheutilityofagivenEDbasednotonsolutionstoSIPS, butonlocalgeometricpropertiesinherenttotheQoImapdenedbytheED.This perspectiveallowsforaquanticationoftheutilityofagivenEDthatisindependent ofboththemethodologyemployedtosolvethechosenSIPandthespecicationofa probabilitymeasureonthespaceofobservableQoI. ThesimilaritiesoftheresultsfromthesetwoSB-OEDapproachesencouraged anextensivecomparisonbetweenthetwoapproachesfocusedonahandfulofchallengingSB-OEDscenarios.Fromthiscomparison,weconcludedthat,ingeneral, themeasure-theoreticapproachallowsforamorerobustandsystematicapproachto SB-OEDinwhichthedescriptionsofagivendesignspacearesubstantiallymoreconsistentinthissetofchallengingSB-OEDscenarios.ThedependenceofthemeasuretheoreticapproachtotheinherentlocalpropertiesofagivenQoImapprovidethis robustinformationinthesedicultsettings,whereasthedependenceoftheconsistentBayesianapproachonactualsolutionstoSIPsprovedtoobscureresultsin challengingscenarios. 129 PAGE 147 Althoughwedeterminedthemeasure-theoreticapproachtoprovideamoresystematicmethodforSB-OED,weneglectedtoexploretheimpactoftheESKon comparisonswiththeEIG.Weconcludedthisthesiswithaseriesofresultsregarding approximationerrorsandcomputationalcomplexityofagivenSIP.Thisexploration providedacleardenitionofasingle-objectiveoptimizationproblemdenedimplicitlyintermsofthesensitivitiesoftheQoImap,thedimensionoftheparameterspace, andthenumberofmodelsolvesavailabletoultimatelyapproximatethesolutiontoa chosenSIP.Thisnalresultsprovidesaclearpathforimplementationofthismethod toalargeclassofapplicationsinwhichmodelsolvesareexpensive,andtherefore limited,resultingincoarseapproximationstothesolutionofagivenSIP. IX.2FutureWorkandOpenProblems Throughoutthisthesis,weeludedtoseveralripeopportunitiesforfuturework. Below,weconsolidatethoseideas. IX.2.1ContinuousOptimization Firstandforemost,thepursuitofacomputationallyecientapproachforcouplingtheSB-OEDmethodsdevelopedwithinthisthesiswithcontinuousoptimization techniquesisanintriguingtopicforfuturework.Wehaverestrictedourselvestorelativelylowdimensionaldesignspacesinthisthesis,theextensionofthisworkto high-dimensionaldesignspaceswoulddemandecientmethodsincontinuousoptimizationasanexhaustivesearchofthedesignspacewouldbecomeintractable. IX.2.2SurrogatestoAccelerateandImproveApproximationoftheEIG WeillustratedadependenceoftheEIGonthenumberofmodelsolvesavailable tocomputerelevantapproximations.Thepursuitofsurrogatemodelingtoboth acceleratethecomputationoftheEIGandincreasetheaccuracyinthescenarioof fewmodelsolvesavailableisaninterestingtopicoffutureresearch. IX.2.3OptimalDesignSpace Thedeterminationofthe optimal designspacetoconsiderinanygivenapplicationisofgreatinterest.Asdiscussedseveraltimesthroughoutthisthesis,notall 130 PAGE 148 observableQoIareequallyuseful,and,worseyet,someQoImaynotbeusefulas all.Wemayencounterscenariosinwhichwesimplycannotlearnusefulinformation aboutoneormanyuncertainmodelinputparametersgiventhespecieddesignspace. Iftheapplicationdoesnotallowforabroadeningofthedesignspace,wemayneedto considerdeterminationoftheoptimaldimensionofthedesignspace,i.e.,theoptimal numberofQoItousetosolvetheresultingSIPsothatresourcesarenotwastedin theprocess.Weoftenblindlyencouragetheovergatheringofobservabledataanddo notaddressthespecicutilityofeachpiece,orset,ofobservabledatagathered.The determinationoftheoptimaldesignspacedimensionwouldcertainlysaveresources intheeldandsubsequentlyreducecomputationalcostofboththeSB-OEDmethod employedandthesolutionoftheresultingSIP. IX.2.4RBFsforGradientApproximation Themeasure-theoreticmethodforSB-OEDrequiresgradientinformationandoftentimeswearesimplygivenasetofsamplesfromapriorfromwhichtoapproximate thesegradients.InAppendixA,weprovidedabriefdiscussionofouruseofRBFs tolocallyapproximatearesponsesurfaceandcomputegradientinformation.This topicisofgreatinterestduetoitsabilitytoallowimplementationofthemeasuretheoreticmethodforSB-OEDinavarietyofresearchscenarios.Althoughthistopic issummarizedbelow,athoroughexplorationoftheseRBFmethodsandtheirstabilityandconvergence,withrespecttogradientapproximations,onsetsofunstructured samplesiswarranted. IX.2.5ConvolutionsandGaussianFiltersforEpistemicUncertainty AseludedtoinSectionVII.1,theproposedmethodforcoveringasoughtinverse imageislimitedinutilityastheQoImapsdeviatefromlinear.However,thereare otherapproachesavailablethatlookpromisinginregardstoproducingamuchtighter coverof Q )]TJ/F18 7.9701 Tf 6.586 0 Td [(1 E .Forexample,convolutions,similartoGaussianlters,canbeusedin signalprocessingtosmoothnoisydata.Inthissameway,itappearsaconvolutioncan 131 PAGE 149 beusedtosmoothagivenapproximateinverseimagesothattheresultingsmoothed inverseimagecoversthesoughtaftersolution. IX.2.6ImproveSystematicSB-OEDFormulation ThecontinuedimprovementoftheSB-OEDmethodsdescribedinthisthesisis anobviousprojecttopursue.Asweencountermoreapplicationsandmoredicult SB-OEDscenarios,wewillundoubtedlybeencouragedtomodifyandimproveupon thecurrentmethodswithinthisthesis.Theultimategoalofacompletelyhands-o approachtoSB-OEDapplicabletonearlyallmodelingscenariosofinterestislikely unrealistic,however,themoresystematicandrobustthecurrentsmethodsbecomeis ofgreatusetothescienticmodelingcommunity. 132 PAGE 150 REFERENCES [1]AlenAlexanderian,NoemiPetra,GeorgStadler,andOmarGhattas.A-optimal designofexperimentsforinnite-dimensionalbayesianlinearinverseproblems withregularized l 0 -sparsication. SIAMJournalonScienticComputing 36:A2122{A2148,2014. [2]AlenAlexandrian,NoemiPetra,GeorgStadler,andOmarGhattas.Afast andscalablemethodfora-optimaldesignofexperimentsforinnite-dimensional bayesiannonlinearinverseproblems. SIAMJournalonScienticComputing 38:A243{A272,2016. [3]MartinAlns,JanBlechta,JohanHake,AugustJohansson,BenjaminKehlet, AndersLogg,ChrisRichardson,JohannesRing,MarieRognes,andGarthWells. TheFEniCSProjectVersion1.5. ArchiveofNumericalSoftware ,3,2015. [4]BillyAmzal,FrdricYBois,EricParent,andChristianPRobert.Bayesianoptimaldesignviainteractingparticlesystems. JournaloftheAmericanStatisticalAssociation ,101:773{785,2006. [5]A.Atkinson,A.Donev,andR.Tobias. OptimumExperimentalDesigns,with SAS .OxfordUniversityPress,2007. [6]AnthonyCAtkinsonandAlexanderNDonev.Theconstructionofexactdoptimumexperimentaldesignswithapplicationtoblockingresponsesurfacedesigns. Biometrika ,pages515{526,1989. [7]VictorBayona,MiguelMoscoso,ManuelCarretero,andManuelKindelan.Rbffdformulasandconvergenceproperties. JournalofComputationalPhysics 229:8281{8295,2010. [8]JosM.Bernardo.Expectedinformationasexpectedutility. TheAnnalsof Statistics ,7:686{690,1979. [9]F.Bisetti,D.Kim,O.Knio,Q.Long,andR.Tempone.Optimalbayesian experimentaldesignforpriorsofcompactsupportwithapplicationtoshock-tube experimentsforcombustionkinetics. Int.J.NumericalMethodsinEngineering 108:136{155,2016. [10]J.Breidt,T.Butler,andD.Estep.AMeasure-TheoreticComputationalMethod forInverseSensitivityProblemsI:MethodandAnalysis. SIAMJournalon NumericalAnalysis ,49:1836{1859,2011. [11]MartinDBuhmann. Radialbasisfunctions:theoryandimplementations ,volume12.Cambridgeuniversitypress,2003. [12]T.Butler,D.Estep,andJ.Sandelin.AComputationalMeasureTheoretic ApproachtoInverseSensitivityProblemsII:APosterioriErrorAnalysis. SIAM JournalonNumericalAnalysis ,50:22{45,2012. 133 PAGE 151 [13]T.Butler,D.Estep,S.Tavener,C.Dawson,andJ.J.Westerink.AMeasureTheoreticComputationalMethodForInverseSensitivityProblemsIII:Multiple QuantitiesofInterest. SIAMJournalonUncertaintyQuantication ,pages1{27, 2014. [14]T.Butler,L.Graham,D.Estep,C.Dawson,andJ.J.Westerink.Denitionand solutionofastochasticinverseproblemforthemanning'snparametereldin hydrodynamicmodels. AdvancesinWaterResources ,78:60{79,2015. [15]T.Butler,L.Graham,S.Mattis,andS.Walsh.Ameasure-theoreticinterpretationofsamplebasednumericalintegrationwithapplicationstoinverseand predictionproblemsunderuncertainty.SubmittedtoSIAMJ.Sci.Comput., 2016. [16]T.Butler,J.Jakeman,andT.Wildey.Aconsistentbayesianformulationfor stochasticinverseproblemsbasedonpush-forwardmeasures.Submittedto SIAMJ.Sci.Comput.,2016. [17]T.Butler,M.Pilosov,andS.Walsh.Simulation-basedoptimalexperimental design:Ameasure-theoreticperspective.InReview,2016. [18]KathrynChaloner.Anoteonoptimalbayesiandesignfornonlinearproblems. JournalofStatisticalPlanningandInference ,37:229{235,1993. [19]KathrynChalonerandKinleyLarntz.Optimalbayesiandesignappliedtologistic regressionexperiments. JournalofStatisticalPlanningandInference ,21:191{ 208,1989. [20]KathrynChalonerandIsabellaVerdinelli.Bayesianexperimentaldesign:A review. StatisticalScience ,10:273{304,1995. [21]MerliseAClyde. Bayesianoptimaldesignsforapproximatenormality .PhD thesis,UniversityofMinnesota,1993. [22]T.A.CoverandJ.A.Thomas. ElementsofInformationTheory .JohnWiley& Sons,2006. [23]M.CrampinandF.A.E.Pirani. ApplicableDierentialGeometry .Cambridge UniversityPress,1987. [24]C.DellacherieandP.A.Meyer. ProbabilitiesandPotential .North-HollandPublishingCo.,Amsterdam,1978. [25]HolgerDetteandLindaMHaines.E-optimaldesignsforlinearandnonlinear modelswithtwoparameters. Biometrika ,81:739{754,1994. [26]GregoryFasshauerandMichaelMcCourt. Kernel-basedapproximationmethods usingMatlab ,volume19.WorldScienticPublishingCoInc,2015. 134 PAGE 152 [27]GregoryEFasshauer. MeshfreeapproximationmethodswithMATLAB ,volume6.WorldScientic,2007. [28]GregoryEFasshauerandJackGZhang.Onchoosingoptimal"shapeparametersforrbfapproximation. NumericalAlgorithms ,45-4:345{368,2007. [29]R.A.Fisher. TheDesignofExperiments .OliverandBoyd,Edinburgh,5th edition,1949. [30]IanFord,D.M.Titterington,andChristosP.Kitsos.Recentadvancesinnonlinearexperimentaldesign. Technometrics ,31:49{60x,1989. [31]BengtFornbergandNatashaFlyer.Accuracyofradialbasisfunctioninterpolationandderivativeapproximationson1-dinnitegrids. AdvancesinComputationalMathematics ,23-2:5{20,2005. [32]LindleyGraham,StevenMattis,ScottWalsh,TroyButler,MichaelPilosov,and DamonMcDougall.BET:Butler,Estep,TavenerMethodv2.0.0,Aug.2016. https://doi.org/10.5281/zenodo.59964. [33]MaxGunzburger.Adjointequation-basedmethodsforcontrolproblemsinincompressible,viscousows. Flow,TurbulenceandCombustion ,65:249{272, 2000. [34]RollandLHardy.Multiquadricequationsoftopographyandotherirregular surfaces. Journalofgeophysicalresearch ,76:1905{1915,1971. [35]X.HuanandT.M.Marzouk.Simulation-basedoptimalBayesianexperimental designfornonlinearsystems. JournalofComputationalPhysics ,232:288{317, 2013. [36]XunHuan. NumericalApproachesforSequentialBayesianOptimalExperimentalDesign .PhDthesis,MassachusettsInstituteofTechnology,Cambridge,MA, September2015. [37]EJKansaandYCHon.Circumventingtheill-conditioningproblemwithmultiquadricradialbasisfunctions:applicationstoellipticpartialdierentialequations. Computers&Mathematicswithapplications ,39-8:123{137,2000. [38]MarcC.KennedyandAnthonyO'Hagan.Bayesiancalibrationofcomputermodels. JournaloftheRoyalStatisticalSociety:SeriesBStatisticalMethodology 63:425{464,2001. [39]EstateKhmaladzeandNToronjadze.Onthealmostsurecoveragepropertyof voronoitessellation:the1case. AdvancesinAppliedProbability ,33:756{ 764,2001. [40]AIKhuri.Anoteond-optimaldesignsforpartiallynonlinearregressionmodels. Technometrics ,26:59{61,1984. 135 PAGE 153 [41]AndreIKhuriandSiuliMukhopadhyay.Responsesurfacemethodology. Wiley InterdisciplinaryReviews:ComputationalStatistics ,2:128{149,2010. [42]J.Kiefer.Optimumexperimentaldesigns. JournaloftheRoyalStatisticalSociety.SeriesBMethodological ,21:272{319,1959. [43]JackKieferandJacobWolfowitz.Optimumdesignsinregressionproblems. The AnnalsofMathematicalStatistics ,pages271{294,1959. [44]S.KullbackandR.A.Leibler.Oninformationandsuciency. TheAnnalsof MathematicalStatistics ,22:79{86,1951. [45]ElisabethLarssonandBengtFornberg.Anumericalstudyofsomeradialbasis functionbasedsolutionmethodsforellipticpdes. Computers&Mathematics withApplications ,46:891{902,2003. [46]D.V.Lindley.Onameasureoftheinformationprovidedbyanexperiment. The AnnalsofMathematicalStatistics ,27:986{1005,1956. [47]AndersLogg,Kent-AndreMardal,andGarthWells,editors. AutomatedSolutionofDierentialEquationsbytheFiniteElementMethod .SpringerBerlin Heidelberg,2012. [48]QuanLong,MohammadMotamed,andRaulTempone.Alaplacemethodfor under-ddeterminedbayesianoptimalexperimentaldesigns. ComputerMethods inAppliedMechanicsandEngineering ,285:849{876,2015. [49]QuanLong,MarcoScavino,RaulTempone,andSuojinWang.Fastestimationofexpectedinformationgainsforbayesianexperimentaldesignsbasedon laplaceapproximations. ComputerMethodsinAppliedMechanicsandEngineering ,259:24{39,2013. [50]QuanLong,MarcoScavino,RaulTempone,andSuojinWang.Fastbayesian optimalexperimentaldesignforseismicsourceinversion. ComputerMethodsin AppliedMechanicsandEngineering ,291:123{145,2015. [51]RichardALuettichJr,JohannesJWesterink,andNormanWSchener.Adcirc: Anadvancedthree-dimensionalcirculationmodelforshelves,coasts,andestuaries.report1.theoryandmethodologyofadcirc-2ddiandadcirc-3dl.Technical report,DTICDocument,1992. [52]CharlesAMicchelli.Interpolationofscattereddata:distancematricesand conditionallypositivedenitefunctions.In Approximationtheoryandspline functions ,pages143{145.Springer,1984. [53]AnnaM.MichalakandPeterK.Kitanidis.Estimationofhistoricalgroundwater contaminantdistributionusingtheadjointstatemethodappliedtogeostatistical inversemodeling. WaterResourcesResearch ,40:n/a{n/a,2004.W08302. 136 PAGE 154 [54]PeterMuller,DonA.Berry,AndyP.Grieve,MichaelSmith,andMichaelKrams. Simulation-basedsequentialbayesiandesign. JournalofStatisticalPlanningand Inference ,137:3140{3150,2007. [55]PeterMller,BrunoSans,andMariaDeIorio.Optimalbayesiandesignbyinhomogeneousmarkovchainsimulation. JournaloftheAmericanStatisticalAssociation ,99:788{798,2004. [56]R.-E.Plessix.Areviewoftheadjoint-statemethodforcomputingthegradient ofafunctionalwithgeophysicalapplications. GeophysicalJournalInternational 167:495,2006. [57]KennethJ.Ryan.Estimatingexpectedinformationgainsforexperimentaldesignswithapplicationstotherandomfatigue-limitmodel. JournalofComputationalandGraphicalStatistics ,12:585{603,2003. [58]K.Sargsyan,H.N.Najm,andR.Ghanem.Onthestatisticalcalibrationof physicalmodels. InternationalJournalofChemicalKinetics ,47:246{276, 2015. [59]A.M.Stuart.Inverseproblems:ABayesianperspective. ActaNumerica 19:451{559,2010. [60]Ne-ZhengSunandWilliamW.-G.Yeh.Coupledinverseproblemsingroundwater modeling:1.sensitivityanalysisandparameteridentication. WaterResources Research ,26:2507{2525,1990. [61]S.TavenerT.WildeyC.DawsonT.Butler,D.EstepandL.Graham.Solving stochasticinverseproblemsusingsigma-algebrasoncontourmaps. [62]AlbertTarantola. InverseProblemTheoryandMethodsforModelParameter Estimation .siam,2005. [63]JojannekevanDenBerg,AndrewCurtis,andJeannotTrampert.Optimalnonlinearbayesianexperimentaldesign:anapplicationtoamplitudeversusoset experiments. GeophysicalJournalInternational ,155:411,2003. [64]T.vanErvenandP.Harremoes.Renyidivergenceandkullback-leiblerdivergence. IEEETransactionsonInformationTheory ,60:3797{3820,2014. [65]IsabellaVerdinelli.Advancesinbayesianexperimentaldesign. BayesianStatistics ,4:467{481,1992. [66]S.Walsh,T.Wildey,andJ.Jakeman.Optimalexperimentaldesignusinga consistentbayesianapproach.SubmittedtoASCE-ASMEJournalofRiskand UncertaintyinEngineeringSystems,PartB:MechanicalEngineering,2017. [67]MPWandandMCJones.Multivariateplug-inbandwidthselection. ComputationalStatistics ,9:97{116,1994. 137 PAGE 155 [68]JGWangandGRLiu.Ontheoptimalshapeparametersofradialbasisfunctions usedfor2-dmeshlessmethods. Computermethodsinappliedmechanicsand engineering ,191:2611{2630,2002. [69]JohnL.WilsonandDouglasE.Metcalfe.Illustrationandvericationofadjoint sensitivitytheoryforsteadystategroundwaterow. WaterResourcesResearch 21:1602{1610,1985. [70]GradyBarrettWright.Radialbasisfunctioninterpolation:numericalandanalyticaldevelopments.2003. [71]SitaoWuandTommyWSChow.Inductionmachinefaultdetectionusing som-basedrbfneuralnetworks. IEEETransactionsonIndustrialElectronics 51:183{194,2004. [72]XiongZhang,KangZhuSong,MingWanLu,andXLiu.Meshlessmethods basedoncollocationwithradialbasisfunctions. Computationalmechanics 26:333{343,2000. 138 PAGE 156 APPENDIXA.RADIALBASISFUNCTIONSFORDERIVIATIVE APPROXIMATIONS TheuseofradialbasisfunctionsRBFstointerpolatedataonanunstructured setofsamplesisawell-studiedtopic[70,26,34,52,27].Thesemethodshavefound awiderangeofapplicationsfromapproximatingsolutionstoPDEs[45,37,72,11] tomachinelearningalgorithms[71,26]toapproximatinggradients[31,7].In[7], theconvergenceoftheRBFmethod,withrespecttoapproximatingdierentialopperators,isstudiedforagivensetofequispacedsamples.Here,weconsiderthe convergenceoftheRBFmethod,withrespecttoapproximatinggradients,onan arbitrarysetofuniformrandomsamples. InSectionA.1,wereviewthewell-knownRBFmethodforinterpolatingdataon anunstructuredsetofsamples.InSectionA.2,wedisplaytheconvergenceofthe methodforapproximatinggradientsontwooscillatorytestfunctions. A.1UsingRBFstoInterpolateUnstructuredData Supposewearegivenasetoffunctionevaluations f f ;:::;f N g ,where ;:::; N 2 aredistinct,forsomefunction f : R n R .Then,an interpolantisgivenby[70], s N = N X i =1 i )]TJ 7.472 0.479 Td [( )]TJ/F20 11.9552 Tf 11.955 0 Td [( i ; where issomeRBFandthe i aredeterminedbysolvingthelinearsystem A = f where a j;k = )]TJ 7.472 0.478 Td [( j )]TJ/F20 11.9552 Tf 11.955 0 Td [( k = 1 ;:::; N ,and f = )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(f ;:::;f N .This linearsystemenforcestheinterpolationconditions s N i = f i for i =1 :::N Whenweareinterestedinapproximatingthegradientof f atsomepoint 2 thenwemaychoosetoconstructonlyalocalinterpolantwith K PAGE 157 Oftentimes,thepartialderivativesof ,denoted @ j for j =1 ;:::;n =dim ,are easilydeterminedanalytically.Inwhichcase,thepartialderivatives @ j s K of s K are givenby, @ j s K = K X k =1 i @ j )]TJ 7.472 0.478 Td [( )]TJ/F20 11.9552 Tf 11.956 0 Td [( k for j =1 ;:::;n: InAlgorithm6,weillustratethestepsinvolvedinapproximating r f using thisRBFmethod.Computationally,themostsignicantexpensescomefromthe nearestneighborsearchandsolutionofthelinearsystemtodeterminethecoecients k Algorithm6: RBFGradientApproximation 1.Givenasetofsamples: i 2 ;i =1 ;:::;N ; 2.Given f i ;i =1 ;:::;N ; 3.Given = i forsome i 2f 1 ;:::;N g ; 4.Findthe K nearestneighborsof : k ;k =1 ;:::;K ; 5.Determinetheinterpolationweights k bysolvingthelinearsystemdenedby theinterpolationconditions s K k = f k for k =1 ;:::;K ; 6.For j =1 ;:::;n =dim ,derivethepartialderivativesof s K @ j s K = K X k =1 k @ j )]TJ 7.472 0.479 Td [( )]TJ/F20 11.9552 Tf 11.955 0 Td [( k 7.For j =1 ;:::;n =dim ,evaluate @ j s K 8.Construct r s K r s K = @ 1 s K ;:::;@ n s K Therearemanypossiblechoicesfor [70,26].Here,wechoose tobeamultivariateGaussianoftheform, r = e )]TJ/F21 7.9701 Tf 6.587 0 Td [(r ; 140 PAGE 158 where istheso-calledshapeparameterand r isthedistancefromthecenterofthe Gaussianto .Wenotethat,acurrenttopicofresearchisthecarefuldeterminiation ofanoptimal [7,68,28],however,adiscussionofthistopicisbeyondthescopeof thiswork.Thechoiceoftheoptimalnumberofnearestneighborstoconsider K is alsoatunableparameter.Forsimplicity,inallofthecomputationsthatfollow,we choose =1and K =10 3 dim )]TJ/F18 7.9701 Tf 6.586 0 Td [(2 A.2ConvergenceoftheExpectedRelativeError Here,weconsidertheconvergenceofthismethodasafunctionofthenumber ofuniformrandomsamplesavailablein .First,weintroduceanoscillatorytest function f : =[0 ; 1] 2 R denedby, f =sin 1 sin 2 ; r f = )]TJ/F15 11.9552 Tf 5.48 -9.684 Td [(2 cos 1 sin 2 ; 2 sin 1 cos 2 : A.1 Weconsiderapproximationsof r f atthepoint = : 200 ; 0 : 249 2 ,where r f : 942 ; 0 : 038.For N =20 ; 40 ; 80 ;:::; 20480,weapproximate r f usingAlgorithm6with K =10,i.e.,weusethe10nearestneighborstoapproximate thegradient.Wecomputeeachapproximation1,000timesusingdierentsetsof N uniformrandomsamplesandcomputetheexpectedrelativeerror.InFigureA.1, weshowtheconvergenceoftheexpectedrelativeerroroftheapproximationofeach componentof r f andthe O 1 N convegencerateforreference.Although,for each N ,theexpectedrelativeerrorfor @ 2 f isordersofmagnitudelargerthan thatof @ 1 f ,bothconvergeatarateofapproximately O 1 N .Notethat,we expectthismethod,oranymethod,tohavedicultyapproximating @ 2 f because thiscomponentisveryclosetozerowhichcausesrelativeerrorstobeamplied. 141 PAGE 159 FigureA.1: Theconvergenceoftheexpectedrelativeerrorforeachcomponentof Eq.A.1.Noticeeachcomponentof r f convergesatapproximatelytherate O 1 N Next,weconsidertheoscillatoryfunction f : =[0 ; 1] 4 R denedby, f =sin 1 sin 2 sin 3 sin 4 ; r f = )]TJ/F15 11.9552 Tf 5.479 -9.683 Td [(2 cos 1 sin 2 sin 3 sin 4 ; A.2 2 sin 1 cos 2 sin 3 sin 4 ; 2 sin 1 sin 2 cos 3 sin 4 ; 2 sin 1 sin 2 sin 3 cos 4 : Weconsiderapproximationsof r f atthepoint = : 200 ; 0 : 249 ; 0 : 100 ; 0 : 150 2 where r f : 923 ; 0 : 018 ; 3 : 911 ; 2 : 065.For N =160 ; 320 ; 640 ;:::; 20480,we approximate r f usingAlgorithm6with K =90,i.e.,weusethe90nearest neighborstoapproximatethegradient.Wecomputeeachapproximation1,000times usingdierentsetsof N uniformrandomsamplesandcomputetheexpectedrelative error.InFigureA.2,weshowtheconvergenceoftheexpectedrelativeerrorofthe 142 PAGE 160 approximationofeachcomponentof r f andthe O 1 N convegencerateforreference.Although,foreach N ,theexpectedrelativeerrorfor @ 2 f isordersof magnitudelargerthanthatoftheotherpartials,allconvergeatarateofapproximately O 1 N .Notethat,weexpectthismethod,oranymethod,tohavediculty approximating @ 2 f becausethiscomponentisveryclosetozerowhichcauses relativeerrorstobeamplied. FigureA.2: Theconvergenceoftheexpectedrelativeerrorforeachcomponentof Eq.A.2.Noticeeachconvergesatapproximatelytherate O 1 N RemarkA.1 InChaptersV,VIandVII,weapproximategradientsofQoImapsdenedbylinearfunctionalsofthesolutiontoastationaryconvection-difussionproblem withanuncertainsourceamplitude,andatime-dependentdiusionproblemwithuncertaindiusioncoecients.ThoseapproximationsarecomputedusingAlgorithm6 with K =10 InChapterVIII,weapproximategradientsofQoImapsdenedbylinearfunctionalsofthesolutiontoatime-dependentdiusionproblemwithuncertaindiusion 143 PAGE 161 coecients.ThoseapproximationsarecomputedusingAlgorithm6with K =20 144 PAGE 162 Glossary BET Butler,Estep,Tavener.27 ED ExperimentalDesign.2 EIE ExpectedInationEect.106 EIG ExpectedInformationGain.7 ESE ExpectedScalingEect.62 ESK ExpectedSkewness.60 GD GeometricallyDistinct.12 i.i.d. independentidenticallydistributed.28 KDE KernelDensityEstimator.21 KL Kullback-Leibler.7 MCMC MarkovchainMonteCarlo.7 OED OptimalExperimentalDesign.3,7 QoI QuantityofInterest.1 SB-OED Simulation-BasedOptimalExperimentalDesign.3,7 SIP StochasticInverseProblem.1 145 |