Citation
Simulation-based optimal experimental design: a measure-theoretic perspective

Material Information

Title:
Simulation-based optimal experimental design: a measure-theoretic perspective
Creator:
Butler, Troy
Pilosov, M.
Walsh, S.
Physical Description:
Journal Article preprint

Notes

Abstract:
A new approach for simulation-based optimal experimental design is developed based on a measure-theoretic perspective for computational deterministic models. Here, an experimental design refers to an experiment defining a particular map from the space of model inputs to the space of observable model outputs. The term optimal experimental design then refers to the choice of a map from model inputs to observable model outputs with optimal geometric properties. The quantifiable geometric properties considered here are the skewness and scaling effect of a map, which are related to the accuracy and precision of solving a stochastic inverse problem. We prove efficient computable approximations of average skewness and scaling effects based on singular value decompositions of sampled Jacobian matrices of the proposed maps. A recently developed measure-theoretic framework is subsequently used to formulate and solve stochastic inverse problems requiring no additional assumptions. Several examples illustrate the various concepts throughout this work. A more detailed numerical example showing a full end-to-end quantification and reduction of uncertainties is also provided demonstrating the process of solving stochastic inverse problems using optimally chosen maps in order to inform predictions.
Acquisition:
Collected for the Auraria Institutional Repository by the AIR Self-Submittal tool. Submitted by Troy Butler.
Publication Status:
Unpublished

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
Copyright Troy Butler. Permission granted to University of Colorado Denver to digitize and display this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.

Downloads

This item is only available as the following downloads:


Full Text

PAGE 1

DRAFT SIAMJ.S CI. C OMPUT. c r xxxxSocietyforIndustrialandAppliedMathematics Vol.xx,pp.x x{x Simulation-BasedOptimalExperimentalDesign:AMeasureTheoretic Perspective T.Butler M.Pilosov y and S.Walsh z Abstract. Anewapproachforsimulation-basedoptimalexperimentald esignisdevelopedbasedonameasuretheoreticperspectiveforcomputationaldeterministicmo dels.Here,anexperimentaldesignrefersto anexperimentdeningaparticularmapfromthespaceofmode linputstothespaceofobservable modeloutputs.Thetermoptimalexperimentaldesignthenre ferstothechoiceofamapfrom modelinputstoobservablemodeloutputswithoptimalgeome tricproperties.Thequantiable geometricpropertiesconsideredherearetheskewnessands calingeectofamap,whicharerelated totheaccuracyandprecisionofsolvingastochasticinvers eproblem.Weproveecientcomputable approximationsofaverageskewnessandscalingeectsbase donsingularvaluedecompositionsof sampledJacobianmatricesoftheproposedmaps.Arecentlyd evelopedmeasure-theoreticframework issubsequentlyusedtoformulateandsolvestochasticinve rseproblemsrequiringnoadditional assumptions.Severalexamplesillustratethevariousconc eptsthroughoutthiswork.Amoredetailed numericalexampleshowingafullend-to-endquantication andreductionofuncertaintiesisalso provideddemonstratingtheprocessofsolvingstochastici nverseproblemsusingoptimallychosen mapsinordertoinformpredictions. Keywords. optimalexperimentaldesign,uncertaintyquantication, inverseproblems 1.Introduction.1.1.OverviewandMotivation. Acriticalchallengetothecommunityofcomputational mathematiciansistodevelopanend-to-endalgorithmicfra meworkandanalysisforthequanticationandreductionofuncertaintiesincomputational modelsofphysicalsystems.Asa rststep,modelinputsareoftendescribedasbelongingtoa setofphysicallyplausiblevalues usingeitherengineeringordomain-specicknowledge.The rangeofmodeloutputsmaythen beapproximatedbyanensembleofdatausingthemodelandsam plesofmodelinputs.When theoutputdataaresensitivetoperturbationsinmodelinpu ts,thepredictivecapabilitiesof eventhemostsophisticatedcomputationalmodelsmaybesev erelylimitedbysuchacoarse descriptionofuncertaintiesinthemodelinputs. Specicationofaprobabilitymeasureonthespaceofmodeli nputsinordertoemploya forwarduncertaintyquantication(UQ)methodisoftenthe nextimportantstepinenhancingthequantitativecapabilitiesofthemodel.Forexample ,samplingaprobabilitymeasure onthespaceofmodelinputsandevaluatingthemodelonthese samplesgeneratesastatisticalensembleofpredicteddatafromwhichcondenceinter vals/boundscanbecomputed. Determiningaphysicallymeaningfulprobabilitymeasureo nthespaceofmodelinputsoften requirestheformulationandsolutionofastochasticinver seproblemusingwhatever(uncerDepartmentofMathematicalandStatisticalSciences,Univ ersityofColoradoDenver,Denver,CO80202 ( Troy.Butler@ucdenver.edu ). y DepartmentofMathematicalandStatisticalSciences,Univ ersityofColoradoDenver,Denver,CO80202 ( Michael.Pilosov@ucdenver.edu ). z DepartmentofMathematicalandStatisticalSciences,Univ ersityofColoradoDenver,Denver,CO80202 ( Scott.Walsh@ucdenver.edu ). 1

PAGE 2

DRAFT 2 T.Butleret.al. tain)dataareavailableonobservablemodeloutputs.Insom ecases,wemaybeabletodene certainaspectsofanexperimentthatdenethetypesofobse rvablemodeloutputsforwhich wecollectdatausedinthesolutiontothestochasticinvers eproblem.Thechoiceofobservable modeloutputs,whichwerefertoasquantitiesofinterest(Q oI),denesaparticularQoImap frommodelinputstomodeloutputs.TwodierentQoImaps,cor respondingtotwodierent experiments,mayleadtosignicantlydierentsolutionsto astochasticinverseproblem.This workfocusesonquantifyingspecicgeometricpropertieso finvertingsetsthroughtheQoI mapsinordertodeneageneraloptimalexperimentaldesign (OED)problemdenedover thespaceofallpossibleQoI.Thisapproachisrootedinmeas uretheoryandthereforerequires veryfewassumptionsonthespacesandtheQoImapsconsidere d. 1.2.RelationshiptoPriorandOtherWork. Theworkof[ 10 ]rstintroducedthegeometricconceptofskewnessinaQoImap,butwaslimitedinsev eralwaysthatareremedied andexpandeduponinthiswork.First,nocomputationallye cientwaytocomputeskewness wasprovidedotherthanbyinspectionofshapesoflower-dim ensionaldatamanifolds(which isnotalwayspossible)orbyrepeatedapplicationofGram-S chmidtalgorithmstogradient vectorsofaQoImap.Second,noconceptofhowaQoImapscales inverseeventsinmeasure wasconsideredwhenchoosinganoptimalQoImap.Finally,th eideaofskewnessinamap waspresentedin[ 10 ]asintimatelyconnectedtotheuseofameasure-theoreticU Qframework fortheformulationandsolutionofastochasticinversepro blem. Inthiswork,weproveecientcomputablemethodsforapprox imatingbothskewnessand scalingeectsusingsingularvaluedecompositionsofsampl edJacobiansofaQoImap.A generalmulticriteriaoptimizationproblemisdenedwhos esolutiondeterminestheoptimal QoImapbasedonthesequantiedgeometricpropertiesandis completelyindependentof anyparticularUQframeworkormethod.Thereareinfactmany UQmethodsthatrequire dierentassumptionsandsolvedierentstochasticinversep roblems.Forexample,there aremanymethodsbasedontheBayesianparadigm(e.g.,see[ 11 33 32 ]),dataassimilation methodsthatarebasedeitherontheKalmanFilterorvariati onaltechniques(e.g.,see[ 22 15 14 ]formethodsbasedonthefamousEnsembleKalmanFilter,and see[ 31 29 ]formethodsthat exploitadjointbasedtechniquesandarerelatedtoregular izationandleast-squarestechniques [ 27 ]),andrecentlyameasure-theoreticframeworkdevelopedi n[ 6 7 8 ].Afullexpositionand analysisoftheeectoftheoptimallychosenQoIonsolutions todierenttypesofstochastic inverseproblems,formulatedandsolvedwithdierentUQmet hods,isbeyondthescopeofthis work.Wethereforelimitthediscussionandnumericalexamp lestousingthemeasure-theoretic frameworkforformulatingandsolvingstochasticinversep roblems,andleaveapplicationto otherUQmethodstofuturework. ItisworthnotingthatOEDcanmeanagreatmanythingsdepend ingonthecontext. Thetermexperimentaldesign(sometimesreferredtoasdesi gnofexperiments),waslikely rstpopularizedwithinthestatisticalcommunity,e.g.,s ee[ 23 ]andthereferencesthereinfor earlierreferencesonthetopic.Thegeneralprocessofdete rminingtheoptimaldesigniscarried outbyanalyzingsomecollectionofhypothesizedstatistic almodelsthatrelatethepredictors (modelinputparameters)totheresponsevariables(QoI).T heoptimaldesignischosenwith respecttoastatisticalcriterion,e.g.,thestatisticalm odelthatminimizesvariance.The interestedreadercanreferto[ 1 ]orthemorethorough[ 3 ]formorerecentreferencesonthe

PAGE 3

DRAFT ExperimentalDesignwithMeasureTheory 3 topic.Weemphasizethatdevelopingappropriatestatistic almodelsandevaluationcriteria arethekeystepsanddiersubstantiallyfromtheapproachpr oposedhereforphysics-based computationalmodels.Theapproachweproposerequiresaco mputationalmodelwhereany choiceofQoIdenestherelationshipbetweenthe\predicto r"and\response"variables,and theevaluationcriteriaareintermsofgeometricpropertie softhisQoImap.Itmaybe possibletodeneaframeworkthatutilizesthesedierentpe rspectivesonOED,e.g.,inorder todeterminethebestsurrogatemodelforsomeproposedseto fQoI,butthisisalsoleftfor futurework. Morerecently,teamsofresearchershaveconsideredasimul ation-basedBayesianOED approach[ 21 5 ].Theresultsappearquitepromising.However,thattypeof approachis ofteninspiredmoredirectlyfromthestatisticalliteratu reandisalsointimatelytiedtothe UQmethodproposedforthesolutionofastochasticinversep roblemtoidentifyparameters. Itisalsosimilartoapproachestakeninthegeoscienceandh ydrologycommunities[ 24 30 ]. Additionally,thattypeofsimulation-basedapproachmayr equiremultiplesolutionsofthe actualstatisticalinverse/inferenceprobleminordertod etermineanoptimaldesignthatleads tomaximuminformationgain,whichmayresultinalargercom putationaloverhead.This makesadirectcomparisontowhatweproposeratherarbitrar yastheactualstochasticinverse problem,UQmethodsused,andalgorithmicproceduresimple mentedaredistinctlydierent. 1.3.Outline. Therestofthispaperisoutlinedasfollows.Section 2 describesthespaces andassumptionsformingthefoundationofthiswork.InSect ion 3 ,wedenethefundamental geometricaspectsofsetinversionthatarequantiedandde scribetherelationtoaccuracyand precisionofsolutionstothestochasticinverseproblem.T heOEDproblemissubsequently denedinSection 4 .Webrieryreviewthemeasure-theoreticframeworkforthef ormulation andsolutionofastochasticinverseprobleminSection 5 .InSection 6 ,wediscussthecomputationalconsiderationsusingareal-worldapplicationof stormsurgemodelingthatappears in[ 19 ]wheresomeoftheseideaswereappliedtochooseoptimalbuo ylocationsforsimulated HurricaneGustavdata.Whileseveralshortexamplesarepro videdthroughouttoillustrate specicconceptsandissues,wepresentafullend-to-endqu anticationandreductionofuncertaintiesexampleinSection 7 thatusesacanonicalmodelofheattransferinmanufactured thinmetalplatestodemonstrateafullunicationoftheide aspresentedherein.Weendwith concludingremarksandsomepossiblefutureresearchdirec tionscurrentlybeinginvestigated inSection 8 2.SpacesofModelInputs,ModelOutputs,andQoIMaps.2.1.BasicAssumptionsandTerminology. Forsimplicity,werefertoallmodelinputsas eithermodelparametersorjustparameterswhenthecontext isclear,andthespaceofmodel inputsisthenreferredtoastheparameterspace,whichisde notedby .Then,aQoImapis a(possiblyvector-valued)functionofparametersandofte ndenotedby Q ( ).Theoutputsof theQoImaparereferredtoasthedata,andtherangeofaQoIma piscalledthedataspace, whichisoftendenotedby D := Q ( ). Weassumethat R n and D R d andthat( ; B ; )and( D ; B D ; D )aremeasure spacesusingtheBorel -algebras.Themeasures and D arereferredtoasvolumemeasures eventhoughtheytechnicallymaynotbeLebesguemeasuresal thoughtheyareanobvious

PAGE 4

DRAFT 4 T.Butleret.al. choicewhichweoftenuseundercertainassumptions.Thepur poseofthesemeasuresisto describethesizesofsets,whichweusetoevaluatethepreci sionofaQoImapinsolving stochasticinverseproblemsasdiscussedinSection 3 Itisworthnotingthatunlesssomespecicconditionsarepl acedonthecomponentsof theQoImap Q D mayexistasalower-dimensionalmanifoldembeddedin R d .Insucha case,theusual d -dimensionalLebesguemeasureassignszeromeasuretoalle ventsin B D ,i.e., alleventsin B D aredeemedashavingthesame\size".Thisissueistechnical lydicultto dealwithwhendescribingprobabilitymeasuresintermsofd ensities,andonewaytoavoid thisdicultyistolet D bedenedbythepushforwardmeasureof Welet Q denotethespaceofalllocallydierentiableQoImapsunderc onsiderationby themodelerforcollectingdatatosolveastochasticinvers eproblem.Forexample,suppose themodelerisconsideringtwodierentexperimentaldesign sthatleadtoQoImaps, Q ( a ) : !D ( a ) ; or Q ( b ) : !D ( b ) : Then, Q = n Q ( a ) ;Q ( b ) o : Foranymap Q 2Q ,weareinterestedinhowthismapaectsthelocalgeometricp ropertiesrelatedtotheshape(specicallytheskewness)andsca lingofthevolumemeasureofthe inverseofanoutputevent E 2B D .Conceptually,theskewnessof Q describesageneralincreaseingeometriccomplexityof E under Q 1 ,whichquantitativelyisrelatedtothenumber ofsamplesin requiredtoaccuratelyestimate ( Q 1 ( E )).Thelocalscalingeectof E by themap Q 1 describestheprecisionofusingthemap Q 2Q toidentifyparametersthatmap to E .InSection 3 ,weshowthatboththelocalskewnessof Q 1 ( E )andthelocalscalingof E bythemap Q 1 (asdeterminedbythe -measure)canbedescribedintermsofsingular valuesofaJacobianof Q .Welet J ;Q 2 R d n denotetheJacobianof Q atapoint 2 andwhentheQoImapbeingconsideredisclearfromcontext,w esimplyuse J Afewremarksareinorder.Forsimplicityindescribingsets ,wewillignoreanyboundaries of or D .However,inpractice,weassumethat iscompact,which,bytheassumed smoothnessofany Q 2Q ,givesthat D isalsocompact.Inotherwords,weassumethat ( )and D ( D )arenite,whichisoftenthecaseinpracticebytheintrodu ctionofknown orassumedboundsofparameteranddatavalues.Second,sinc etheskewnessandscaling propertiesof Q mayvarysignicantlythroughout when Q isnonlinear,wemustaccount forthisvariabilityindeterminingoptimalQoI. 2.2.Dening Q Here,weprovideajusticationforrestrictingthedimensi onalityofQoI mapsthatdene Q .Sinceweassumeevery Q 2Q issmooth,werstsimplifythediscussion byconsideringthecasewhere Q islinearandthengeneralizetononlinearbut(piecewise) dierentiable Q .Since Q isassumedlinear,then Q ( )= J where J 2 R d n .If d>n ,then thereareatmost n rowsof J thatarelinearlyindependent.If d n ,thentherearemost d rowsof J thatarelinearlyindependent.Ineithercase,let m n denotethenumberofrowsof J thatarelinearlyindependent.Basicresultsfromlinearal gebraimplythat D = Q ( ) R d isdenedbythecolumnspaceof J andisgivenbyavectorsubspaceofdimension m .Thus, D isdenedbyan m -dimensionalhyperplaneembeddedin R d ,and B D isaBorel -algebra

PAGE 5

DRAFT ExperimentalDesignwithMeasureTheory 5 thatcanbeconstructedby m -dimensionalgeneralizedrectanglesdenedwithrespectt oan orthogonalsetof m vectorsin R d Let ^ J denoteanysubmatrixof J denedbychoosingany m linearlyindependentrowsof J ,anddenoteby ^ Q ( )themapinto R m denedby ^ J .Wesimilarlydenote ^ D = ^ Q ( ) R m and ^ B D astheBorel algebraof m -dimensionalsetsinan m -dimensionalspace.Let P denote a m d projectionmatrixfrom D to ^ D .Weabusenotationandlet PE denotetheactionof thematrix P onallvectorsin E 2B D ,i.e., PE 2 ^ B D istheprojectionoftheevent E onto ^ D .Thus,forany E 2B D ,thereisaunique ^ E = PE 2 ^ B D representingtheprojectionof E onto ^ D .Theimplicationisthatforany E 2B D Q 1 ( E )= ^ Q 1 ( ^ E ) 2B : Thisimplicationleadstoaninterestinginterpretationof thefollowingLemmasummarizedin Remark 2.2 Lemma2.1. Suppose Q 2Q isa d -dimensionalmap, J ;Q hasconstantrank m in isaproductmeasure,and E isanysetin B D .Forevery > 0 ,thereexistsapiecewiselinear m -dimensionalmap ^ Q : ^ D ,andanitenumber K of m -dimensionalgeneralized rectangles, n ^ E k o 1 k K ^ B D ,suchthatif D denotesthesymmetricdierenceof Q 1 ( E ) and [ 1 k K ^ Q 1 ( ^ E k ) ,then ( D ) < Proof Since Q isassumeddierentiable,itismeasurable,and Q 1 ( E ) 2B .Let > 0. Astandardresultinmeasuretheoryisthatforany A 2B ,thereexistsanitenumber K ofgeneralizedrectangles, f A k g 1 k K ,suchthat ( A 4[ 1 k K A k ) <; e.g.,seeTheorem2.40in[ 16 ].Let A = Q 1 ( E ) andapplysucharesult.Foreach1 k K ,choose 2 A k anddenoteitby ( k ) ,andlet Q ( k ) denotetheanemapon A k givenby Q ( ( k ) )+ J ( k ) ;Q : Bytheconstantrankassumption,dene ^ Q asthe m -dimensionalpiecewiselinearmapconstructedbytaking m n submatricesof J ( k ) ;Q andthesame m rowsof Q ( ( k ) )foreach 1 k K .Finally,let ^ E k = ^ Q ( A k ) ; foreach1 k K: Theconclusionfollowsbythechoiceof A k andthepropertiesof ^ Q WhiletheaboveLemmaprovidesatheoreticalmotivationfor developingandusingsurrogatepiecewiselinearmaps,itdoesnotprovideacomputat ionalapproachforconstructing thegeneralizedrectangles f A k g 1 k K usedintheproof.Moreover,itisnotnecessarilythe casethatsuchasurrogateisapointwiseaccurateapproxima tiontotheexactQoImapsince thediametersofanyindividual A k usedintheproofcanbearbitrarilylargeevenif ( A k ) issmall.Thisimpliesthatextrapolationerrorsinthesurr ogatecanbelarge.Additionally,

PAGE 6

DRAFT 6 T.Butleret.al. thereisnoguaranteethatwecanobservedatarelatedtothis surrogateawayfromthepoints Q ( ( k ) )duetothepossibleextrapolationerrorsdiscussedabove. Remark2.1 Weingeneraldonotknowwhichevents E 2B D areimportanttoconsider priortocollectingdataforaQoI.Lemma 2.1 employsthefactthatany E 2B D canbe approximatedarbitrarilywellusinganitenumberofgener alizedrectangles.Itisalsocommonfor -algebrasonhigherdimensionalspacestobegeneratedfrom generalizedrectangles. Therefore,intheremainderofthispaper,unlessexplicitl ystatedotherwise,wewillassume thatanyevent E 2B D isageneralizedrectangle. Remark2.2 Lemma 2.1 hasaninterestingconceptualimplicationintheOEDcontex t. Specically,iftheJacobianmatrixof Q hasconstantrank m in ,thenthelocalapproximationof Q 1 ( E )in -measurecannotbesignicantlyimproveduponbyusingmore than m n geometricallydistinct modelobservablestodene Q [ 8 ].Here,by geometricallydistinct ,wesimplymeanthat J ;Q hasfullrankforall 2 WithLemma 2.1 andRemark 2.2 inmind,wehereinafteradditionallyassumethatany Q 2Q isavector-valueddierentiablefunctionofgeometrically distinctcomponentmaps, whichimpliesthat d n 3.Accuracy,Precision,andSVDofJacobians. Webeginwithatechnicallemmaand ensuingcorollaryusedtodescribethesizesof d -dimensionalparallelepipedsembeddedin n dimensionalspaces.Weareinterestedintwocases: d -dimensionalparallelepipedsdened bytherowsofagivenmatrix J ,and d -dimensionalparallelepipedsdeterminedbythecross sectionsof n -dimensionalcylindersthataredenedbythepre-imageofa d -dimensionalunit cubeunder J Lemma3.1. Let J beafullrank d n matrixwith d n ,and Pa ( J ) denotethe d dimensionalparallelepipeddenedbythe d rowsof J .TheLebesguemeasure d in R d of Pa ( J ) isgivenbytheproductofthe d singularvalues f k g dk =1 of J ,i.e., d ( Pa ( J ))= d Y k =1 k : (3.1) Proof Thesingularvaluesof J areequaltothesingularvaluesof J > .Considerthe reducedQRfactorizationof J > J > = ~ QR; (3.2) where ~ Q is n d and R is d d .BythepropertiesoftheQRfactorization,weknowthe singularvaluesof R arethesameasthesingularvaluesof J > .Let x 2 R d ,then jj ~ Q x jj 2 =( ~ Q x ) > ( ~ Q x )= x > ~ Q > ~ Q x = x > x = jj x jj 2 ; (3.3) so ~ Q isanisometry.ThisimpliestheLebesguemeasureofthepara llelepipeddenedbythe rowsof R isequaltotheLebesguemeasureoftheparallelepipeddene dbythecolumnsof

PAGE 7

DRAFT ExperimentalDesignwithMeasureTheory 7 J > ,ortherowsof J d ( Pa ( J ))= d ( Pa ( R ))= d Y k =1 r k = d Y k =1 k ; (3.4) where f r k g 1 k d arethesingularvaluesof R and f k g 1 k d arethesingularvaluesof J Wenowturnourattentiontothesecondcaseofdescribingthe sizeofa d -dimensional parallelepipeddeterminedbythecrosssectionofan n -dimensionalcylinderdenedbythe pre-imageofa d -dimensionalunitcubeunder J .Inthiscase,weconsiderthepseudo-inverse of J J + = J > ( JJ > ) 1 .Asisevidentfromtheformulaofthepseudo-inverse,thera ngeof J + isequaltotherowspaceof J .Thisimpliesthatthe d -measureofthecross-sectionofthe pre-imageofaunitcubeunder J isequaltothethe d -measureoftheparallelepipeddened bythecolumnsof J + Corollary3.2. Let J beafullrank d n matrixwith d n .Then Pa (( J + ) > ) isa d dimensionalparallelepipeddeningacross-sectionofthe pre-imageofa d -dimensionalunit cubeunder J anditsLebesguemeasure d isgivenbytheinverseoftheproductofthe d singularvalues f k g dk =1 of J ,i.e., d ( Pa (( J + ) > ))= d Y k =1 k 1 : (3.5) Proof Considerthepseudo-inverseof J J + = J > ( JJ > ) 1 : Fromthisequation,itisclearthatthecolumnspaceof J + isequaltotherowspaceof J Therowspaceof J denesasubspaceorthogonaltothe n -dimensionalcylinderthatisthe pre-imageofaunitcubeunder J .Therefore,thecolumnspaceof J + isorthogonaltothe pre-imagecylinderand Pa (( J + ) > )isa d -dimensionalparallelepipeddeninganorthogonal cross-sectionofthiscylinder. Frombasicresultsinlinearalgebra,thesingularvaluesof ( J + ) > areequaltothoseof J + Then,frompropertiesofthepseudo-inverse,thesingularv aluesof J + aretheinverseofthe singularvaluesof J .Finally,fromLemma 3.1 ,itfollowsthat d ( Pa (( J + ) > ))= d Y k =1 k 1 ; (3.6) where f k g dk =1 arethesingularvaluesof J 3.1.SkewnessandAccuracy. Supposewecanchoosebetweentwoexperimentsleading totwodistinctchoicesofQoImapssothat Q = Q ( a ) ;Q ( b ) .Wegenerallyapproximate solutionstostochasticinverseproblems,e.g.,usingnit erandomsampling,andweexpect thatcertainchoicesofQoImapsmayleadtomoreaccuratesol utionsundertheconstraintsof axedcomputationalbudget.Below,wedenetheskewnessof aQoImap,provideameans

PAGE 8

DRAFT 8 T.Butleret.al. forcomputingtheexpectedskewness,anddescribetherelat ionshiptoaccurateapproximation ofinversesetswithnitesampling. First,assumethat Q islinearand E 2B D ageneralizedrectangle.When d = n Q 1 ( E ) isa d -dimensionalparallelepipedin (ignoringanyaectsfrompossibleintersectionswith theboundaryof )thatisin1-to-1correspondencewith E .If d
PAGE 9

DRAFT ExperimentalDesignwithMeasureTheory 9 Conceptually, S Q ( J ; j k )describestheamountof redundant informationpresentinthe k th componentoftheQoImapcomparedtowhatispresentintheoth er d 1componentswhen invertingnearthepoint 2 .Thesmallestvalue S Q ( )canbeisone.Thereisnolargest valuesincethereexistsmaps Q thathavegeometricallydistinctcomponentmaps,butthe conditionoftheJacobianmaybearbitrarilylarge.IftheJa cobianweretoeverfailtobefull rank,then S Q ( )wouldbeinnite.However,theassumptionofgeometricall ydistinctQoI preventsthisfromoccurring.Remark3.1 In[ 10 ],thefundamentaldecompositionresultofTheorem 3.3 wasusedtoshow thatthenumberofsamplesdeningregulargrids(andthusal souniformi.i.d.setsofsamples) in requiredtoobtainaccurateapproximationsin -measureof Q 1 ( E )isproportionalto sup 2 ( S Q ( )) d 1 : However,inthatwork,therewasnodiscussionofhowtocompu te S Q ( ) otherthananimplicitassumptionofrepeatedapplications ofaGram-Schmidtprocesstoobtain j j ?k j ,andinspectionoflowdimensionaldataspaceswastheprima rytooltorankproposed QoImapsbytheirskewness.Below,weexploitecientSVDcom putationstocomputeboth localandexpectedskewness,andweprovidenumericalresul tsdemonstratinghowexpected valuesofskewnessrelatetoan expected accuracyininversesetapproximations. ThefundamentaldecompositionofTheorem 3.3 alongwithLemma 3.1 providesaconvenientmethodfordeterminingtheskewnessintermsofthe d -dimensionalparallelepipeds describedby Q Corollary3.5. Forany Q 2Q S Q ( ) canbecompletelydeterminedbythethenormsof n -dimensionalvectorsandproductsofsingularvaluesofthe JacobianofQmapsofdimenions d 1 and d S Q ( )=max 1 k d j j k j d 1 ( Pa ( J k; )) d ( Pa ( J )) : (3.9) Proof S Q ( )=max 1 k d S Q ( J ; j k )=max 1 k d j j k j j j ?k j =max 1 k d j j k j d 1 ( Pa ( J k; )) d ( Pa ( J )) ; (3.10) thenapplyingLemma 3.1 wehave max 1 k d j j k j d 1 ( Pa ( J k; )) d ( Pa ( J )) =max 1 k d j j k j Q d 1 r =1 k;r Q dr =1 r : (3.11) where f r g 1 r d arethesingularvaluesof J and f k;r g 1 r d 1 arethesingularvaluesof J k; Corollary 3.5 impliesthatwecanexploitecientsingularvaluedecompos itionstoalgorithmicallyapproximate S Q ( )atanypoint 2 .Since S Q ( )mayvarysubstantiallyover ,wemustquantifythisvariabilityinordertooptimallycho ose Q 2Q Denition3.6. Forany Q 2Q ,wedenethe average (or expected )skewnessas S Q = 1 ( ) Z S Q ( ) d ; (3.12)

PAGE 10

DRAFT 10 T.Butleret.al. Generally,weapproximate S Q usingMonteCarlotechniquestogenerateasetofindependentidenticallydistributed(i.i.d.)samples ( i ) Ni =1 andcompute S Q S Q;N := 1 N N X i =1 S Q ( ( i )) : (3.13) Remark3.2 Noticethedescriptionsofskewnessareindependentofgene ralizedrectangle E 2B D .Inotherwords,skewnessisapropertyinherenttothemap Q itselfanddescribes thewayinwhichthegeometryof E 2B D ischangedbyapplying Q 1 Whilelinearmapscanoftenbeusedtodirectlyconstruct,or atleastdescribeexactly, theinversesofsets,weconsidertheuseofnitesamplingin toapproximatetheinverse sets.Thisismotivatedbyseveralcomputationalconsidera tionsofmanyUQmethodsandthe typesofmaps Q 2Q weconsiderasdescribedbelow. ItiscommonforUQmethodstousenitesamplingtodescribet heapproximatesolutions. Wewouldprefertore-useanycomputationsfromthesolution totheOEDproblem(e.g.,in approximatingminimalvaluesofexpectedskewnessusingEq .( 3.13 ))forsolvingthestochastic inverseproblem.SeeSection 6 formorediscussiononcomputationalconsiderations. 3.2.ScalingandPrecision. Tomotivatewhatfollows,considerthesimpleproblemwhere wemustchoosebetweentwodierentexperimentsleadingtotw odistinctQoImaps Q ( a ) and Q ( b ) sothat Q = Q ( a ) ;Q ( b ) .Let E ( a ) and E ( b ) representthesetofallprobableobservations fromusingeither Q ( a ) or Q ( b ) ,respectively.Then,dependingonwhichexperimentweobse rve, wewouldconcludethateithertheparametersbelongto Q ( a ) ; 1 ( E ( a ) )or Q ( b ) ; 1 ( E ( b ) )almost surely.Suppose ( Q ( a ) ; 1 ( E ( a ) )) ( Q ( b ) ; 1 ( E ( b ) )) ; thenwegenerallyexpectthatensemblesofparametersample sgeneratedfromresultsbasedon theexperimentleadingto Q ( a ) willhavesmallervariance(whichisadescriptionofprecis ion instatisticalterms)thanthosebasedontheexperimentlea dingto Q ( b ) .Thismotivatesa generalmeasure-theoreticgoalfordesigningexperiments whereeventsofhighprobabilityon adataspacearemade\smallinvolume"ontheparameterspace byinvertingtheQoImap. Webeginwithasimplifyingassumptionthat Q 2Q islinear,withgeometricallydistinct componentmaps.Then,thereexistsa d n matrix J ,suchthat Q ( )= J .If d = n and = R n ,itiseasilyshownfromstandardresultsinmeasuretheorya ndlinearalgebrathat ( Q 1 ( E ))= D ( E )det( J 1 )= D ( E ) d Y k =1 k 1 ; (3.14) where f k g 1 k d arethesingularvaluesof J .Notethatif R n isproper,thentheabove equationisnotnecessarilytrueas Q 1 ( E )mayintersecttheboundaryof .Weneglect suchboundaryeectsinthecomputations,andsimplynotetha tincertaincasestheymay playanimportantrolealthoughthisisnotthetypicalcasei nourexperience.If d
PAGE 11

DRAFT ExperimentalDesignwithMeasureTheory 11 recallthat Q 1 ( E )isdenedbyacylinderin withcrosssectionsgivenby d -dimensional parallelepipeds.WecanuseCorollary 3.2 tocomputethemeasureofthese d -dimensional parallelepipeds.Thismotivatesthefollowing Denition3.7. Forany Q 2Q and 2 ,wedenethe local( -measure)scaling eect of Q as M Q ( )= d Y k =1 k 1 ; (3.15) where f k g 1 k d arethesingularvaluesoftheJacobian J ;Q .The average (or expected )) ( -measure)scalingeect isgivenby M Q = 1 ( ) Z M Q ( ) d : (3.16) Aswithaverageskewnessof Q ,wegenerallyapproximate M Q usingasetofi.i.d.random samples ( i ) Ni =1 andcomputing M Q M Q;N := 1 N N X i =1 M Q ( ( i )) : (3.17) Wesummarizetheaboveresultsintothefollowing Corollary3.8. Forany Q 2Q ,thelocalskewness,localscalingeect,averageskewness ,and averagescalingeectof Q canbeestimatedusingnormsofrow-vectorsandsingularval uesof theJacobian J ( i ) ;Q atanitesetofpoints ( i ) 1 i N Remark3.3 Analternativeto M Q thataccountsforpossiblydierent D ( E )istousesimple multiplicationof D ( E )assuggestedbyEq.( 3.14 ). 4.OptimalExperimentalDesign. Webeginwithasimplemotivatingexamplethatshows howtheskewnessinaQoImapaectsexpectederrorsin -measuresofinversesetsapproximatedbyanitesetofuniformi.i.d.randomsamples.While wecouldusesimpleMonte Carloestimatesofthe -measureofinversesets,weinsteadoptfortheVoronoicell sdened byasetofsamplesinordertobetterillustratetheerrorsin estimating -measureofinverse sets(e.g.,seeFigure 4.1 ). Example1. Suppose =[0 ; 1] 2 ,andwehaveachoicebetweentwoexperimentsleadingto twodistincttwo-dimensionallinearQoImaps Q ( a ) and Q ( b ) sothat Q = Q ( a ) ;Q ( b ) .Map Q ( a ) isdescribedbya 2 2 matrixwithorthogonalrow-vectorssothat S Q =1 andmap Q ( b ) isdescribedbya 2 2 matrixwithnon-orthogonalrowvectorssothat S Q 4 .Wechoose E ( a ) 2B ( a ) D and E ( b ) 2B ( b ) D astworectangleschosensothat Q ( a ) ; 1 ( E ( a ) ) and Q ( b ) ; 1 ( E ( b ) ) haveexactlythesame -measure.InFigure 4.1 ,weshowsomerepresentativeplotsofthe approximationsoftheseinverseeventsusingdierentnumb ersofuniformi.i.d.randomsamplesin .Theerrorsintheseapproximations,givenbythesymmetricd ierences,arevisually illustratedbythedarkestshadedregionanddecreasewitha dditionalsamples.InFigure 4.2

PAGE 12

DRAFT 12 T.Butleret.al. andTable 4.1 ,weshowtheconvergenceofthemeasureofthesymmetricdie rencetobenear theknownconvergencerateofMCmethods, N 1 = 2 .Inotherwords,weexpectconvergenceat thesameasymptoticratenomatterhowskewedthemap.Howeve r,theresultsfortheskewed map Q ( b ) generallyrequireapproximatelyfourtimesthenumberofra ndomsamplesinorderto obtainthesamelevelofexpectedaccuracyin -measureasseenusing Q ( a ) (seeTable 4.1 ). Figure4.1:Theexactinversesets Q ( a ) ; 1 ( E ( a ) )(leftcolumn)and Q ( b ) ; 1 ( E ( b ) )(rightcolumn) aretheintersectionofthedierentregionsineachplotandh avetheexactsame -measure. Theerrorin -measureoftheapproximationtotheseeventsisgivenbythe symmetric dierenceoftheexactinversesetswiththeimplicitlydene dVoronoicellsassociatedwith niterandomsamplingoftheseeventsandisillustratedbyt hedarkestshadedregion.We showthisusing50samples(top),and3200samples(bottom). Intheaboveexample,weonlyconsideredtheimpactofskewne ss.Forsucientlylarge numbersofsamples,theerrorin -measureofapproximatinginversesetsfromusingeither QoImapcanbemadearbitrarilysmall.Whentherearenolimit sonthecomputational budget,skewnessfromaQoImapmaynolongerbeaconcern,and wewouldgenerallychoose theQoImapthatleadstooptimalprecision.However,facedw iththepracticalrealities ofnitecomputationalresources,weassumethatthegenera lgoalistocomputeaccurate estimatesofinversesetsthatalsoimprovetheprecisionof predictions.Insomecases,the choiceistrivial.Forexample,if S Q ( a ) < S Q ( b ) and M Q ( a ) < M Q ( b ) ,thenweclearlychoosethe experimentleadingto Q ( a ) .However,ifwechangeoneofthoseinequalities,thenthech oice wemakebecomesambiguous.Below,wedescribeoneapproachf orquantitativelyassessing whichQoImapisoptimalinsuchcases.

PAGE 13

DRAFT ExperimentalDesignwithMeasureTheory 13 r r n Figure4.2: Convergenceplot(log-log scale)forthemeansymmetricdier-enceshowninTable 4.1 Num.Samples Q ( a ) Q ( b ) 50 4 : 66 E 2 8 : 01 E 2 200 2 : 37 E 2 4 : 45 E 2 800 1 : 22 E 2 2 : 33 E 2 3200 6 : 13 E 3 1 : 22 E 2 Table4.1: Meanerror(computedover100 trialsforeachnumberofsamples)givenbysymmetricdierencein -measureapproximationsofinversesetsusingnon-skewedandskewedmaps, Q ( a ) and Q ( b ) ,respectively. 4.1.AMulticriteriaOptimizationProblem. Welet S R denotethesetofallpossible valuesof S Q and M R denotethesetofallpossiblevaluesof M Q forall Q belongingtoa specied Q .Clearly, S isboundedbelowby1,whichrepresentsthecaseof\optimalg lobal skewness"inaQoImap Q .Similarly, M isboundedbelowby0,whichrepresentsthecase whereasetofoutputdataexactlyidentiesaparticulargen eralizedcontourofparameters responsibleforthedata. For Q ,wecanclearlyorderall Q 2Q accordingtoeitherthevaluesof S Q or M Q .In otherwords,thevaluesof S Q and M Q separatelydescribeanorderingindexonthespace ofallpossibleQoIgivenby Q .However,themapping Q 7! ( S Q ; M Q )describesadouble indexingwithnonaturalordering.Wedenemetricsinorder toquantifythedistancetothe optimalpointof(1 ; 0)intheCartesianproductspacedescribedbythepairs( S Q ; M Q ).While therearemanyoptionsfordeningmetrics,athoroughinves tigationontheeectofdierent metricsisbeyondthescopeofthisworkwhichisfocusedonth eexpositionofthisgeneral approach.Below,wechooseaparticularformforthemetrics on S and M thatminimizesthe eectofthepossiblydisparaterangesofvalueswemayobserv ein S and M onthesolution tothemulticriteriaoptimizationproblem. Wedene( S ;d S )and( M ;d M )usingthemetrics d S ( x;y )= j x y j 1+ j x y j forall x;y 2S ; (4.1) and d M ( x;y )= j x y j 1+ j x y j forall x;y 2M : (4.2) Choose 2 (0 ; 1)andlet Y denotetheCartesianproductspace SM ,withmetricdened by d Y ( x;y )= !d S ( x 1 ;y 1 )+(1 ) d M ( x 2 ;y 2 )forall x;y 2 Y : (4.3) Notethat determinestherelativeimportanceweplaceoneitherpreci sionoraccuracy.

PAGE 14

DRAFT 14 T.Butleret.al. Choosing =0implieswedisregardskewnessinthecriterionwhereasch oosing =1implies wedisregardthescalingeect.Remark4.1 Choosinganoptimal givenacomputationalbudgetisthetopicofcurrent research.Below,weprovidesomesimpleexampleswith =0 ; 0 : 5 ; and1toillustratehow dierentchoicesof inruencestheselectionoftheoptimalQoI. Denition4.1. Given Q and 2 (0 ; 1) ,the optimalexperimentaldesignproblem (OEDproblem) isdenedbythemulticriteriaoptimizationproblem min Q 2Q d Y ( p;y ) ; (4.4) where p =(1 ; 0) isthe idealpoint and y =( S Q ; M Q ) for Q 2Q 4.2.DiscreteOptimization. When Q denesasetwithnitecardinality,thesolutionto theminimizationproblemdenedbyEq.( 4.4 )canbefoundbyanexhaustivesearch.This problemcanclearlybecomecomputationallyexpensiveasth enumberofpossiblemapsgets large,anditmaybenecessarytoinitiallylter Q toasmallersetofpossibilities,e.g.,using domainspecicinformation.However,itiscompletelystra ightforwardtoimplementandis embarrassinglyparallel. Example2. Let R 2 andsupposewecanchoosefromacongurationofexperiments thatleadtoachoiceofanytwomeasurementsfrom f Q 1 ( ) ;Q 2 ( ) ;Q 3 ( ) g ; where Q 1 ( )=0 : 5 1 +0 : 5 2 ;Q 2 ( )=2 : 5 1 +0 : 5 2 ;Q 3 ( )= 0 : 2 1 +0 : 3 2 : Inotherwords, Q = Q ( a ) ;Q ( b ) ;Q ( c ) wherewehave Q ( a ) =( Q 1 ;Q 2 ) Q ( b ) =( Q 1 ;Q 3 ) ,and Q ( c ) =( Q 2 ;Q 3 ) .Thelinearityofthemapsmakescomputationsof S Q and M Q trivial.Wesee inFigure 4.3 andTable 4.2 theoptimalchoiceofpairofQoItouseintheinverseproblem for threedierentoptimizationcriteria;minimize M Q (choosing =0 ),minimize S Q (choosing =1 ),andminimizeEq. ( 4.3 ) with =0 : 5 .Itisvisuallyevidentfromtheleftplotof Figure 4.3 thattheQoIthatminimizes M Q alsoproducesaninversesetwiththesharpest corners,i.e.,thisisthemapwiththemostskewness.Inthemi ddleplotofFigure 4.3 we observetheoppositeeect,theQoIthatminimizes S Q maximizes M Q .Intherightplot ofFigure 4.3 ,weobserveaQoImapthatservesasthebest\compromise"and produces reasonablysmallvaluesof S Q and M Q simultaneously. 4.3.ContinuousOptimization. Inmanyphysicalapplications,whilewemaybelimited inplacingatotalof d measurementdevicescorrespondingto d geometricallydistinctobservablemodeloutputs,theactualcongurationofthedevicesm ayonlybelimitedtobelonging toaparticularspace-timecylinder.Forexample,wemaybea bletoplacetwocontactthermometersanywherealongathinrodinordertorecordthetemp eratureataparticularpoint intime,buttheremaybenoadditionalrestrictions.Insuch acase,thespace Q isdened implicitlyandrepresentsan uncountable set.Itmaybepossibletouse r Q S Q and r Q M Q (andpossiblyHessiansof S Q and M Q on Q ),toemploycontinuousoptimizationmethodsto determinelocallyoptimalsolutionstotheminimizationpr oblemdenedbyEq.( 4.4 ).While

PAGE 15

DRAFT ExperimentalDesignwithMeasureTheory 15 Figure4.3:TheinversesetsfromthethreedierentQoImapsd ening Q aregivenbythe intersectionoftheindividualcomponentmapinversesets. Fromlefttorightweshowthe eectofusing Q ( a ) =( Q 1 ;Q 2 ), Q ( b ) =( Q 1 ;Q 3 ),and Q ( c ) =( Q 2 ;Q 3 ). Opt.Criteria Optimal Q 2Q M Q S Q d Y ( p;y ), =0 : 5 Minimize M Q Q ( a ) 4 : 0 E 2 1 : 80 E +0 4 : 84 E 1 Minimize S Q Q ( b ) 1 : 6 E 1 1 : 02 E +0 1 : 57 E 1 Eq.( 4.3 ), =0 : 5 Q ( c ) 4 : 7 E 2 1 : 08 E +0 1 : 20 E 1 Table4.2:Weenumerate Q (rstcolumn)andshow M Q S Q and d Y (0 ;y )for =0 : 5inthe second,thirdandfourthcolumns,respectively.thisisbeyondthescopeofthiswork,weprovideaconcreteco nceptualexampletoillustrate thegeneralcomplexity.Furtherinvestigationofthispart iculartypeofoptimizationproblem islefttofuturework. Example3. Considertheproblemwhereweusetheheatequationtopredict temperatures onathinmetalrodofunitlength.Let n=[0 ; 1] denotethespatialdomain.Supposetherod ismanufacturedbyweldingtogethertwothinrodsofequalle ngth 0 : 5 andofsimilaralloytype together.However,duetothemanufacturingprocess,suppo setheactualalloycompositions maydierslightlyleadingtouncertainthermaldiusivity propertiesonthelefthalfandright halfoftherod,denotedby 1 and 2 ,respectively.Supposetheexperimentswecanruninvolve placingtwocontactthermometersalongthefullrodthatcan recordtwoseparatetemperature measurementsatthesametimewhentherodissubjecttoapart icularlowtemperaturesource. ThegoaloftheOEDistodeterminewheretoplacethetwocontac tthermometersinorderto quantifyandreduceuncertaintiesinthethermaldiusivit ypropertiesonbothsidesoftherod. Usingknowledgeofthetypesofalloysandimpuritiesintrod ucedbythemanufacturing process,supposethatforanyrod =( 1 ; 2 ) 2 =[0 : 01 ; 0 : 2] 2 .Thecongurationofthe contactthermometersonaroddenesacoordinate,whichwed enoteby ( x 1 ;x 2 2 n n ,where x and y denotethepositionsoftherstandsecondcontactthermome tersalong n ,respectively. Apoint ( x 1 ;x 2 ) thencorrespondstoaparticular Q 2Q mapping toa D .Wecantherefore plot S Q M Q and d Y ( p;y ) (foranychoiceof )asfunctionsover n n .Weshowsuchplots inFigure 4.4 .Intheseplots,lightercolorscorrespondtomoreoptimalva luesanddarker colorsarelessoptimal.Observethesymmetryalongtheline x 1 = x 2 ,whichcorrespondstoa renamingofcontactthermometeroneascontactthermometer twoandviceversa.Placingthe

PAGE 16

DRAFT 16 T.Butleret.al. 0.00 0.25 0.50 0.75 1.00 x 1 0.00 0.25 0.50 0.75 1.00x 2 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.00 0.25 0.50 0.75 1.00 x 1 0.00 0.25 0.50 0.75 1.00x 2 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.00 0.25 0.50 0.75 1.00 x 1 0.00 0.25 0.50 0.75 1.00x 2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 Figure4.4:Fromlefttoright:Plotsof M Q S Q andvalueof d Y ( p;y )with =0 : 5over Q 2Q denedbythespatiallocationoftherstmeasurementonthe horizontalaxisandthe spatiallocationofthesecondmeasurementontheverticala xis.Darkercolorsindicateless optimalchoicesof Q andlightercolorsindicatemoreoptimalchoicesof Q contactthermometerssymmetricallyawayfromthemidpoint |neartheendsoftherod|can leadtooptimalQoImapsforallofthedierentoptimization criteria,whichmatchesphysical intuition(seethelowerrightandupperleftcornersofthep lotsinFigure 4.4 ).However,there aregenerallyseverallocalmaximaandminimaintheseplots .Thus,givenanyinitialguess foranoptimalQoImap,andanyparticularoptimizationcrit eria,continuousoptimization algorithmswillingeneralproduceonlyalocally,notgloba lly,optimalQoImap. 5.FormulationandSolutionofaStochasticInverseProblem. Webrierysummarizethe measure-theoreticframeworkfortheformulationandsolut ionofastochasticinverseproblem. Formoredetailsonthetheoryandcomputationalalgorithms ,wedirecttheinterestedreader to[ 6 7 8 9 ].InSection 7 ,weshowtheeectofoptimallychoosingtheQoIonsolutionst o thestochasticinverseproblem. GivenaQoImap Q : !D andaprobabilitymeasure P D on( D ; B D ),thestochastic inverseproblemistodetermineaprobabilitymeasure P on( ; B ).When Q hasgeometricallydistinctcomponentmapsand d = n ,then P isuniquebyasimplechangeofvariables formula.However,when d
PAGE 17

DRAFT ExperimentalDesignwithMeasureTheory 17 oftheinverseproblem, Q denesabijectionbetween L and D ,sothemeasure P D uniquely determines P L butcannotprovideanyinformationabouttheprobabilities alongthegeneralizedcontours.Employinganansatz[ 8 ]forthefamilyofconditionalprobabilitymeasures denesaprobabilitymeasure P on( ; B )thatisuniquefortheparticularansatz.Following[ 8 ],weusethestandardansatzbasedontheuniquedisintegrat ionofvolumemeasures thateectivelyproportionsprobabilityalongcontoureven tsaccordingtotheirrelativesize. Theresultingprobabilitymeasuresexploittheavailableg eometricinformationresultingin non-parameteric(andoftenmulti-modal)solutionstothes tochasticinverseproblem.These solutionsaretypicallyapproximatedusingrandomsamplin g,see[ 8 9 ]formoreinformation anddetailsabouttheapproximationsandcomputationalalg orithms.Theinterestedreader mayalsosee[ 10 28 ]formoreengineeringorienteddiscussionsofthecomputat ionalalgorithm inthecontextofparticularapplicationsinvolvingshallo wwaterequationsandsubsurface contaminanttransport.Below,weprovideasimpleexamples howinghowtheskewnessina QoImapeectstheconvergencetothesolutionofthestochast icinverseproblem. Example4. Let =[0 ; 1] 2 R 2 andconsidertwolineartwo-dimensionalQoImaps Q ( a ) and Q ( b ) with S Q ( a ) =1 and S Q ( b ) =2 ,butchosensothat M Q ( a ) = M Q ( b ) =1 .Toillustrate howskewnessaectstheaccuracyinapproximatingtheinver sionofaprobabilitymeasureusing astraightforwardi.i.d.uniformrandomsamplingalgorith m[ 8 9 ],weconsiderthesimple problemofinvertingauniformdistributiondenedonrecta ngles E ( a ) D ( a ) and E ( b ) D ( b ) where E ( a ) and E ( b ) arechosensothat ( Q ( a ) ; 1 ( E ( a ) ))= ( Q ( b ) ; 1 ( E ( b ) ) ,andboth Q ( a ) ; 1 ( E ( a ) ) and Q ( b ) ; 1 ( E ( b ) ) areinteriorto .Withthesechoices,asimplechangeof variablesformulashowsthatthesolutiontothestochastic inverseproblemisgivenbyauniform distributiondenedeitheron Q ( a ) ; 1 ( E ( a ) ) or Q ( b ) ; 1 ( E ( b ) ) ,whichwecomputeonaregular uniformgridof 200 200 pointsin asareferencesolutionforeachcase. Notethattheprobabilityofanyofthei.i.d.randomsamples belongingtoeither Q ( a ) ; 1 ( E ( a ) ) and Q ( b ) ; 1 ( E ( b ) ) isexactlythesamesince ( Q ( a ) ; 1 ( E ( a ) ))= ( Q ( b ) ; 1 ( E ( b ) ) .However, theskewnessof Q ( b ) aectstheset Q ( b ) ; 1 ( E ( b ) ) sothatthereismoreapproximationerrorof theuniformprobabilitymeasureonthiseventinawaysimila rtoasshowninExample 1 .We usetheHellingermetric[ 17 ]tocomputethedistancefromarandomsampleapproximation tothereferenceprobabilitymeasureassociatedtoeitherm ap.InFigure 5.1 andTable 5.1 ,we showthemeanHellingerdistancevaluescomputedbyrepeati ngeachrandomsettrial50times. Observethatapproximatelytwiceasmanyrandomsamplesare requiredtoachieveasimilar levelofexpectedaccuracyintheapproximateprobabilitym easurefromusing Q ( b ) comparedto using Q ( a ) 6.ComputationalConsiderations. AsmentionedinSection 4 Q maydeneeithera niteorinnitesetofpossibleQoImaps.When Q isnite,wemayuseengineeringor domain-specicknowledgetoreducethesetfurther.Thisma yalsobedonewhen Q isinnite. Example5. Here,wesummarizeanexamplestudiedin[ 19 ]wheresomeofthecomputationalideasfromthisworkwereappliedtoarealapplicatio n.Theproblemistoidentifythe spatialcongurationoffourbuoysrecordingsimulatedmax imumstormsurgeforHurricane GustavinBaySt.Louisinordertoquantifyuncertaintiesin thespatiallyheterogeneousManning'scoecientoffriction.Allowingbuoystobelocateda tanypointinthedomainleads toanuncountablyinnitesetofcongurationsanddenesac ontinuousOEDproblem.In-

PAGE 18

DRAFT 18 T.Butleret.al. n r Figure5.1: Convergenceplot(log-log scale)usingmeanHellingerdistancesofapproximationstoareferenceprob-abilitymeasureasseeninTable 5.1 Num.Samples Q ( a ) Q ( b ) 200 1 : 35 E 01 2 : 01 E 01 400 9 : 93 E 02 1 : 45 E 01 800 6 : 95 E 02 1 : 03 E 01 1600 5 : 07 E 02 7 : 52 E 02 3200 3 : 68 E 02 5 : 39 E 02 6400 2 : 60 E 02 3 : 99 E 02 Table5.1: MeanHellingerdistance(computed over50trialsforeachnumberofsamples)ofrandomsampleapproximationtoreferenceprobabilitymeasuresolvedusing Q ( a ) and Q ( b ) where S Q ( a ) =1 and S Q ( b ) =2 stead,weappliedknowledgeofthemodelanddomaintoavoidc ongurationsthatleadtobuoys exhibitingeitherlittleorsimilarsensitivitiestotheMa nning'scoecientoffrictionvalues. Usingthisknowledge,weidentied194possiblelocations( seetheleftmostplotinthediagramofFigure 6.1 ).Thiscreateda Q denedby 194 4 =57 ; 211 ; 376 possiblecongurations ofthebuoys.Simulationswerecomputedusingthestate-ofthe-artADvancedCIRCulation (ADCIRC)model[ 26 4 18 ]requiringsignicantcomputationalresources,whichlim itedthe numberofsampleswecouldusein toafewhundredincludinganymodelsimulationsused tocomputeJacobiansoftheQoImaps.Therefore,wechosetoop timizeonlyforskewness. WeshowtheeectusingtheoptimalQoImapcomparedtoaQoIma pwithnon-optimalskewnessinFigure 6.1 wheretheinitiallteringoflocationsforsensitivitiesa lsoleadstogreater precisioninthemarginals.Forafulldiscussionoftheresu lts,see[ 19 ]. Theaboveexampleillustratessomeofthecomputationalcon siderationsinoptimizing Q undertheconstraintsofacomputationalbudget.Specical ly,wegenerallymustalways answerthefollowingquestions. Canevery Q 2Q besavedtoleforeachsimulationofthemodel? HowaretheJacobiansofeach Q computedtoapproximate S Q and M Q ? Canwere-useanymodelsimulationsinsolvingthestochasti cinverseproblem? WediscusssomeofthesequestionsinmoredetailinSection 8 .Withrespecttotheabove example,wewereabletostorethemaximumstormsurgeatall1 94proposedbuoylocationsfor eachsampleofManning'scoecientoffrictionvalues.Weus ednitedierencingcenteredat randomsamplestoapproximatetheJacobiansandcouldre-us ethesesamplesinthemeasuretheoreticinversion. Insomecases,itmaybepossibletosavepartsofasimulation inaspatial-temporalmodel inordertoapproximatethe Q 2Q .Forexample,wecouldselectasetoftimenodesatwhich thefullspatialsolutionissavedanda(piecewise)polynom ialtintimeisusedtosimulate thepossibleQoImapsatanypointinspace-time.Thiscreate sacomputationaloverheadthat aectstheoverallcomputationalbudget,butisoutsidethes copeofthiswork.

PAGE 19

DRAFT ExperimentalDesignwithMeasureTheory 19 0 09 0 100 110 120 130 140 1510 14 0 16 0 18 0 20 0 2220 40 80 120 160 200 240 280 320 360 1 ,2 (Lebesgue) Optimal Sub-Optimal UQ UQ 0 09 0 100 110 120 130 140 1510 14 0 16 0 18 0 20 0 2220 80 160 240 320 400 480 560 640 1 ,2 (Lebesgue) Figure6.1:Left:Thedotsidentifythe194proposedbuoysta tionsforrecordingmaximum surgeelevationinBaySt.Louis.Thecolorindicatesthedie renceinmaximumwaterelevationsovertheentirespaceofManning'scoecientoffricti onfordominantlandclassications. Middle:Optimalandsuboptimalbuoylocationsdeterminedb yrankingskewness.Right:Representative2Dmarginal( 1 ; 3 )probabilityplotsofdierentManning'scoecientoffrict ion valuescomputedfrommeasure-theoreticinversion.Thewhi tedotisthereferenceparameter usedtosimulatedataofHurricaneGustav.Figuresareadopt edfrom[ 19 ]. 7.NumericalExample:OEDandEnd-to-EndUQ.7.1.TheModeland Q Considerthesimpletimedependentdiusionmodelforheat transferinathinmetalplatewithuncertaindiusioncoeci entsthatbothsummarizesresults andmatchesphysicalintuition.Themodel,andparticularl oadusedduringexperiments,are describedbythepartialdierentialequation c @T @t = r ( r T )+ f;x 2 n ;t 2 ( t 0 ;t f ) ; where 8>><>>: f ( x )= AH ( t t s )exp ( x 1 ) 2 +( x 2 ) 2 w T ( x ;0)=0 ; @T @n =0 ;x 2 @ n ; and x =( x 1 ;x 2 ) 2 n=[ 1 2 ; 1 2 ] [ 1 2 ; 1 2 ]representsthedimensionsofthethinmetalplate.We let =1 : 5denotethedensityoftheplate, c =1 : 5theheatcapacity,and istheuncertain thermalconductivity.Theforcingfunction f representsanexternalheatsourcedescribedby aGaussianshapewithmaximumamplitudeatthecenterofthep late,andwelet A =50 and w =0 : 05.Thefunction H ( t t s )istheHeavisidefunctionindicatingthattheexternal sourceisturnedoattime t s = t f = 2,wherewetake t f =5 : 0.ThehomogeneousNeumann boundaryconditionsmodelperfectinsulationaroundthebo undary.

PAGE 20

DRAFT 20 T.Butleret.al. SimilartoExample 3 ,weassumethatthemetalplateismanufacturedbyweldingto getheralongtheverticalline x 1 =0tworectangularplatesofthesamesizeandalloytype. However,duetoimperfectionsintheoriginalmanufacturin gprocess,weassumethatthealloy compositioncanvarysignicantlyacrossthecenterlineof theplatedenedby x 1 =0.We let =( 1 ; 2 ) 2 =[0 : 01 ; 0 : 2] 2 denotethethermalconductivityoftheplatewhere 1 and 2 denotethethermalconductivitywhen x 1 < 0andwhen x 1 > 0(theleftandrighthalves), respectively. Forthenumericalsolutionofthemodel,weusethestateofth eartopen-sourcenite elementsoftwareFEniCS[ 2 25 ].Thespatialdiscretizationisgivenbya40 40triangular meshusingpiecewiselinearniteelements.Weusedaunifor mmeshtodiscretize[0 ;t f ]with 500totaltimestepsandaCrankNicolsontimesteppingschem e. Weassumeduringanexperimentthattwocontactthermometer scanbeusedtorecord thetemperatureattwoseparatepointsinspace-time.Thech oicesoftwospace-timepointsto recordthetemperaturedeterminesthespace Q .Thegoalistodeterminewhich Q 2Q helps usobtainthe\best"inferencesaboutthethermalconductiv itiesoneachsideoftheplate. Intuitively,weexpecttotakeonetemperaturemeasurement oneachsideoftheplate,butit isnotclearatwhattimestepseachtemperaturemeasurement shouldberecordedorexactly wheretoplacethesensorsoneachsideoftheplate. Forsimplicity,welimit Q toadiscretesetofpointsinspace-time.First,weselected 20 pointsontheplateforplacingthethermometers,seethepoi ntslabeled p i for1 i 20in theleftplotofFigure 7.1 .Sincevaluesoftemperatureataxedpointinspacearesimi lar inconsecutivetimesteps,weassumedthetemperaturewasre cordedatevery25 th timestep resultinginatotalof20dierentpointsintimeforrecordin gtemperature.Thisresultedin 400possibledierentspace-timelocationstoplaceacontac tthermometer,andchoosingany twodenesaQoImap.Inotherwords, Q isdenedby 400 2 =79 ; 800possibleQoImaps. Thesensorsofthecontactthermometersrecordingdataarem athematicallymodeledby takingtheaveragetemperatureonasmalldiscaboutapoint p i 2 nataxedtime.Specically,let B r ( p i )denotethesmalldisccenteredatpoint p i ,theneachQoIismodeledas Q ( p i ; t j ) := 1 n ( B r ( p i )) Z n T ( x ; t j ) B r ( p i ) dx: Here,thesubscriptgivenby( p i ; t j )isanindexingdescribingthepointinspace p i andtime t j ofthemeasurement.Inthecomputations,wetake r =0 : 05. 7.2.OEDandanInverseProblem. Thestochasticinverseproblemsformulatedbelow aresolvedusingtheopensourceBET[ 20 ]Pythonpackage,andthisexampleisavailable withinthedocumentedexamplesoftheBETpackage.Werefert heinterestedreaderto theBETpackageanddocumentationthereinformoredetailso nthecomputationsdiscussed below. Foreach Q 2Q ,weapproximate S Q and M Q usingEqs.( 3.13 )and( 3.17 ),respectively,to compute S Q;N and M Q;N with N =5000i.i.d.uniformrandomsamplesin .Wedetermine threeoptimalQoImapscorrespondingtothreedierentoptim izationcriteria;minimize M Q;N minimize S Q;N ,andminimizethedistancedenedinEq.( 4.3 )with =0 : 5.Let Q ( M ) Q ( S )

PAGE 21

DRAFT ExperimentalDesignwithMeasureTheory 21 Figure7.1:Left:TheQoIlocationsinthedomainnlabeled p i for1 i 20.Right:Semi-log scatterplotofpointsin Y givenby( S Q;N ; M Q;N )foreach Q 2Q .AlongtheParetofrontier fromtoplefttobottomright,thelargercirclesindicateth eQoImapswhichminimize S Q;N solveEq.( 4.3 )with =0 : 5,andminimize M Q;N ,respectively. Opt.Criteria OptimalQoImaps M Q;N S Q;N d Y ( p;y ) Minimize M Q;N Q ( M ) =( Q ( p 18 ;t 14 ) ;Q ( p 17 ;t 15 ) ) 1 : 72 E 05 1 : 46 E +00 3 : 31 E 01 Minimize S Q;N Q ( S ) =( Q ( p 1 ;t 1 ) ;Q ( p 12 ;t 1 ) ) 3 : 76 E +03 1 : 00 E +00 9 : 99 E 01 Eq.( 4.3 ), =0 : 5 Q ( Y ) =( Q ( p 17 ;t 4 ) ;Q ( p 18 ;t 4 ) ) 6 : 26 E 04 1 : 0004 E +00 9 : 84 E 04 Table7.1:Thedierentoptimizationcriteriaforchoosingo ptimalQoIandvaluesof ( M Q;N ; S Q;N )showninFigure 7.2 andthevalueof d Y ( p;y )usedtodetermineoptimalQoI mapbasedondierentoptimizationcriteria.and Q ( Y ) denotethesolutionstotheseoptimizationproblems,respe ctively(seeTable 7.1 andtherightplotofFigure 7.1 ).Noticethatfollowingthescatterofpointsintherightpl ot ofFigure 7.1 thatasmoothcurvecanapparentlybettedtotheboundaryof thepointsin Y nearestthecoordinateaxes.Inmulticriteriaoptimizatio nthisisreferredtoasthePareto frontier,andweseeinallthreecasesthattheoptimalQoIma pproducesapointin Y thatlies veryneartheParetofrontier.Specically,minimizing M Q;N producesapointatthebottom ofthisfrontier,minimizing S Q;N producesapointattheleftofthisfrontier,andminimizing Y producesapointnearthebottom\corner"ofthisfrontier(t hecornerissomewhatshifted duetothesemi-lognatureoftheplot).Thisdemonstratesat ypeoftrade-othatiscommon inotherareasofmodeldesign,e.g.,thebias/variancetrad e-othatoccursinsemiparametric regression.Asintuitivelyexpected,eachQoImapisdened bycontactthermometerslocated onoppositesidesoftheplate.However,therearenotabledi erencesinspace-timelocations, describedbytheindicesofthecomponentmaps,ofeachoptim alQoImap.Forexample,the QoImapminimizing M Q;N isproducedbymeasurementsmadeneartheendofthesimulati on. Bycomparison,theQoImapminimizing S Q;N isproducedbymeasurementsmadeattherst

PAGE 22

DRAFT 22 T.Butleret.al. timestep. Wenowformulateandsolveastochasticinverseproblemfore achoftheQoImapssolving thedierentoptimizationproblems.First,wesimulateanob served\true"datumforallthree casesbyselectingarandomlychosenpoint,denotedby ref 2 (seethelargerblackcircular dotinthetopplotsofFigure 7.2 ),andcomputing Q ( M ) ( ref ), Q ( S ) ( ref ),and Q ( Y ) ( ref ). Weassumetheuncertaintyinthisobserveddatumisafunctio nofthemeasurementdeviceand nottheactualmap,i.e.,themeasurementisuncertaindueto imperfectionsinthecontact thermometersbutarenotinruencedbythespace-timechoice deningtheQoImap.We choosethisuncertaintytobe0 : 5degrees,andassumethatbyrepeatedexperimentswiththe sameQoImapthatalldatainthe2-dimensionalsquarewithsi delengthsof0 : 5centeredat thetruedatumareequallylikelytooccur.Let E ( M ) E ( S ) ,and E ( Y ) denotethissquarein thedataspacesdenedby Q ( M ) Q ( S ) ,and Q ( Y ) ,respectively. Theassumptionsonthedataspacesimplythattheobservedpr obabilitymeasuretoinvert fromeachofthedataspacesisuniformoneachoftheevents E ( M ) E ( S ) ,and E ( Y ) .Thesupportoftheassociatedinversedensitiessolvingthestocha sticinverseproblemsarethengiven by Q ( M ) ; 1 ( E ( M ) ), Q ( S ) ; 1 ( E ( S ) ),and Q ( Y ) ; 1 ( E ( Y ) )approximatedbytheshadedareasin thetoprowofplotsofFigure 7.2 usingthesame N =5000i.i.d.uniformrandomsamples usedtocompute S Q;N and M Q;N (i.e.,were-usethesamplestocomputethesolutiontothe stochasticinverseproblem).Theapproximatedensitieswi thintheseapproximatedeventsare computedusingtheBETpackageandshowninthebottomrowofp lotsinFigure 7.2 InthetopmiddleplotofFigure 7.2 ,weseethat Q ( S ) ; 1 ( E ( S ) )= .Thus,whilethis inverseimagecanbewellapproximatedbyrelativelyfewsam ples(orinfactnotusingany samplesatall),itisuselessduetothelackofprecisiontha tgivesnonewinformationabout thelocationoftheparametersthatproducedtheobservedda tumotherthantheybelongto theoriginalparameterspace. 7.3.TheImpactonPredictionsandComputationalComplexity. Forsimplicity,we considerapredictionproblemforasimilarQoIusingasimil armodel.Wedenotetheprediction QoIas Q ( p ) whichrepresentstheaveragetemperaturealongtherightbo undaryoftheplateat thenaltimestepwiththeGaussianshapedexternalsourcen owcenteredat( 1 = 2 ; 1 = 2) 2 n,i.e.,centeredatthebottomleftcorneroftheplate. Weapproximatethepredicted(pushforward)densitiesof Q ( p ) associatedwiththedierentinversedensitiesshowninFigure 7.2 usingstandardMonteCarlosamplingandkernel densityestimation.TheresultsaresummarizedinFigure 7.3 .Notethesimilaritiesfrom propagatingtheinversedensitiesassociatedwith Q ( Y ) and Q ( M ) ,andthedierencesofthese propagateddensitiesfromthepropagationoftheinversede nsityassociatedwith Q ( S ) .Specifically,thepropagateddensitiesassociatedwith Q ( Y ) or Q ( M ) producepredictionintervals (denedbythesupportofthepropagateddensities)between 4to5timessmallercompared tothepredictionintervalobtainedbypropagatingtheinve rsedensityassociatedwith Q ( S ) Removingtheeectsofthekerneldensityestimation,thepre dictionintervalsassociatedwith Q ( M ) and Q ( Y ) havelengthsofabout5 : 7and6 : 2,respectively.Thisisconsistentwiththe generalexpectationthatpredictionsassociatedwith Q ( M ) aretypicallythemostprecisedue totheoverallreductioninvarianceduetothesmallersuppo rtoftheinversedensity(ascan bevisuallyobservedinFigure 7.2 ).However,thisslightimprovementinprecisioncompared

PAGE 23

DRAFT ExperimentalDesignwithMeasureTheory 23 n r n r n n r Figure7.2: (top):Randomsampleapproximationof Q ( M ) ; 1 ( E ( M ) )) Q ( S ) ; 1 ( E ( S ) ) ,and Q ( Y ) ( E ( Y ) ) .(bottom):Theapproximateinversedensitieswhosesupport scorrespondtothe topplotsusingthedierentoptimalQoImaps. Figure7.3:Approximatedensitiesof Q ( p ) obtainedbypropagatingtheinversedensitiesshown inFigure 7.2 .Thedierentdensityapproximationsof Q ( p ) arelabeledaccordingtotheoptimal QoImapusedtoconstructtheassociatedinversedensitypro pagatedto Q ( p ) topredictionresultsassociatedwith Q ( Y ) comesatanincreasedcomputationalcomplexity toachievethesameexpectedlevelsofaccuracysince S Q ( M ) ;N > S Q ( Y ) ;N accordingtoTable 7.1 .Infact,itiseasilyseenfromthetoprightplotofFigure 7.2 thatthesupportofthe inversedensityassociatedwithusing Q ( Y ) canbewellapproximatedbyasinglerectanglein ,whichisnottrueforthesupportoftheinversedensityasso ciatedwith Q ( M )

PAGE 24

DRAFT 24 T.Butleret.al. 8.ConclusionsandFutureWork. Wedevelopedageneralcomputationalframework usingecientsingularvaluedecompositionsofsampledJac obianstoquantifythegeometric propertiesofaverageskewnessandscalingeectsofmapsina space Q .Ageneralmulticriteria optimizationproblemwasusedtodenetheOEDproblem,ands everalexamplesillustrated thevariousconceptspresented.Ameasure-theoreticUQmet hodwasusedtoformulateand solveanend-to-endquanticationandreductionofuncerta intiesexamplebasedondierent solutionstotheOEDproblem. Therearemanypossibledirectionsoffutureresearch,andw ecommentonaselectfew thatarethetopicsofcurrentresearch.Onesuchtopicisthe analysisofhowthisOEDapproachimpactsnonmeasure-theoreticUQmethodsusedtofor mulateandsolvestochastic inverseproblems,e.g.,usingBayesianapproaches,andtoc omparethisOEDapproachother approachesbasedondierentconceptualideasofoptimality formulatedforspecicUQmethods.Wearealsoactivelyresearchingasinglecriteriaopti mizationproblemthatexploitsthe skewnessimplicitlyandfocusesentirelyonprecisionbase donquanticationof -scalingof coversofinverseeventsbysimplegeometricobjects. Thereisalsoafundamentalquestionregardingtheuseof d>n QoIincaseswheredata collectionischeap,e.g.,whentherearemanyexistingsens orsintheeldproducingpossibly veryredundantdata.Thisisatypeof\datarich,informatio npoor"problem,andweare studyingapproachesforutilizingalltheavailableQoIdat ainsuchcases.Onemethodthat appearspromisingistopartition anddenelocallyoptimalQoImapsthatreducesolutionof theglobalinverseproblemintoasolutionofonlyasubsetof locallyoptimalinverseproblems. Anothertopicofcurrentinteresttotheauthorsinvolvesad escriptionoftheimprovement orchangeinoptimalityfromusingadditionalQoItoincreas ethedimensionoftheQoImap. Thisrequiresquantifyingboththeinformationgainandinc rease/decreaseincomputational complexityfromusingmore/lessQoI. 9.Acknowledgments. ThismaterialisbaseduponworksupportedbytheU.S.Depart mentofEnergyOceofScience,OceofAdvancedScienticCo mputingResearch,Applied MathematicsprogramunderAwardNumberDE-SC00009279aspa rtoftheDiaMonDMultifacetedMathematicsIntegratedCapabilityCenter.T.Bu tlerandS.Walsh'sworkisalso supportedinpartbytheNationalScienceFoundation(DMS-1 228206). TheauthorswouldalsoliketoacknowledgeDr.LindleyGraha mandDr.StevenMattis fortheirproductiveconversationsandfeedbackonthiswor k,aswellastheirhelpwiththe BET[ 20 ]Pythonpackagewhichleadtothedevelopmentandimplement ationofthesensitivity subpackagebyS.Walshwhichwasusedforquantifyingskewne ssandscalingeectsofQoI mapsinallexamplesshowninthiswork. REFERENCES [1] R.D.CookA.C.Atkinson D-optimumdesignsforheteroscedasticlinearmodels ,Journalofthe AmericanStatisticalAssociation,90(1995),pp.204{212. [2] MartinAlns,JanBlechta,JohanHake,AugustJohansson,Ben jaminKehlet,AndersLogg, ChrisRichardson,JohannesRing,MarieRognes,andGarthWe lls TheFEniCSProject Version1.5 ,ArchiveofNumericalSoftware,3(2015).

PAGE 25

DRAFT ExperimentalDesignwithMeasureTheory 25 [3] A.Atkinson,A.Donev,andR.Tobias OptimumExperimentalDesigns,withSAS ,OxfordUniversity Press,2007. [4] J.Baugh,A.Altuntas,T.Dyer,andJ.Simon SubdomainModelinginADCIRCVersion50 ,2013. [5] F.Bisetti,D.Kim,O.Knio,Q.Long,andR.Tempone Optimalbayesianexperimentaldesignfor priorsofcompactsupportwithapplicationtoshock-tubeex perimentsforcombustionkinetics ,Int.J. NumericalMethodsinEngineering,108(2016),pp.136{155. [6] J.Breidt,T.Butler,andD.Estep AMeasure-TheoreticComputationalMethodforInverseSens itivityProblemsI:MethodandAnalysis ,SIAMJournalonNumericalAnalysis,49(2011),pp.1836{18 59. [7] T.Butler,D.Estep,andJ.Sandelin AComputationalMeasureTheoreticApproachtoInverse SensitivityProblemsII:APosterioriErrorAnalysis ,SIAMJournalonNumericalAnalysis,50(2012), pp.22{45. [8] T.Butler,D.Estep,S.Tavener,C.Dawson,andJ.J.Westerin k AMeasure-TheoreticComputationalMethodForInverseSensitivityProblemsIII:Mult ipleQuantitiesofInterest ,SIAMJournal onUncertaintyQuantication,(2014),pp.1{27. [9] T.Butler,D.Estep,S.Tavener,T.Wildey,C.Dawson,andL.G raham SolvingStochastic InverseProblemsusingSigma-AlgebrasonContourMaps .arXiv:1407.3851,2014. [10] T.Butler,L.Graham,D.Estep,C.Dawson,andJ.J.Westerink Denitionandsolutionofa stochasticinverseproblemforthemanning'snparametere ldinhydrodynamicmodels ,Advancesin WaterResources,78(2015),pp.60{79. [11] S.L.Cotter,M.Dashti,andA.M.Stuart Approximationofbayesianinverseproblems ,SIAM JournalofNumericalAnalysis,48(2010),pp.322{345. [12] M.CrampinandF.A.E.Pirani ApplicableDierentialGeometry ,CambridgeUniversityPress,1987. [13] C.DellacherieandP.A.Meyer ProbabilitiesandPotential ,North-HollandPublishingCo.,Amsterdam,1978. [14] GeirEvensen Sequentialdataassimilationwithanonlinearquasi-geost rophicmodelusingmontecarlo methodstoforecasterrorstatistics ,JournalofGeophysicalResearch:Oceans(1978{2012),99( 1994), pp.10143{10162. [15] GeirEvensen TheensembleKalmanlter:theoreticalformulationandpra cticalimplementation ,Ocean Dynamics,53(2003),pp.343{367. [16] GeraldB.Folland RealAnalysis:ModernTechniquesandTheirApplications ,Wiley,1999. [17] AlisonL.GibbsandFrancisEdwardSu Onchoosingandboundingprobabilitymetrics ,INTERNAT. STATIST.REV.,(2002),pp.419{435. [18] LindleyGraham PolyADCIRCVersion0.3.0 .online,Jan2016. [ https://github.com/UT-CHG/PolyADCIRC ]. [19] L.Graham,T.Butler,S.Walsh,C.Dawson,D.Estep,andJ.J.W esterink AMeasureTheoreticAlgorithmforEstimatingBottomFrictioninaCoa stalInlet:CaseStudyofBaySt.Louis duringHurricaneGustav(2008) ,(2016).Inreview. [20] LindleyGraham,StevenMattis,ScottWalsh,TroyButler,Mi chaelPilosov, andDamonMcDougall BET:Butler,Estep,TavenerMethodv2.0.0 ,Aug.2016. https://doi.org/10.5281/zenodo.59964. [21] X.HuanandT.M.Marzouk Simulation-basedoptimalBayesianexperimentaldesignfo rnonlinear systems ,JournalofComputationalPhysics,232(2013),pp.288{317 [22] RudolphEmilKalmanetal. Anewapproachtolinearlteringandpredictionproblems ,Journalof basicEngineering,82(1960),pp.35{45. [23] J.Kiefer Optimumexperimentaldesigns ,JournaloftheRoyalStatisticalSociety.SeriesB(Method ological),21(1959),pp.272{319. [24] P.C.Leube,A.Geiges,andW.Nowak Bayesianassessmentoftheexpecteddataimpactonpredicti on condenceinoptimalsamplingdesign ,WaterResourcesResearch,48(2012),pp.n/a{n/a. [25] AndersLogg,Kent-AndreMardal,andGarthWells ,eds., AutomatedSolutionofDierential EquationsbytheFiniteElementMethod ,SpringerBerlinHeidelberg,2012. [26] RALuettichandJJWesterink ADCIRCUserManual:A(Parallel)AdvancedCirculationMode lfor Oceanic,CoastalANDEstuarineWaters ,UniversityofNorthCarolinaatChapelHillandUniversity ofNotreDame,version49ed.,April12010. [27] DonaldWMarquardt Analgorithmforleast-squaresestimationofnonlinearpar ameters ,Journalof

PAGE 26

DRAFT 26 T.Butleret.al. theSocietyforIndustrial&AppliedMathematics,11(1963) ,pp.431{441. [28] S.A.Mattis,T.D.Butler,C.N.Dawson,D.Estep,andV.V.Ves selinov Parameterestimation andpredictionforgroundwatercontaminationbasedonmeas uretheory ,WaterResourcesResearch, 51(2015),pp.7608{7629. [29] IMNavon Practicalandtheoreticalaspectsofadjointparameterest imationandidentiabilityinmeteorologyandoceanography ,DynamicsofAtmospheresandOceans,27(1998),pp.55{79. [30] W.Nowak,F.P.J.deBarros,andY.Rubin Bayesiangeostatisticaldesign:Task-drivenoptimal siteinvestigationwhenthegeostatisticalmodelisuncert ain ,WaterResourcesResearch,46(2010), pp.n/a{n/a. [31] VishwasRaoandAdrianSandu Aposteriorierrorestimatesforthesolutionofvariationa linverse problems ,SIAM/ASAJournalonUncertaintyQuantication,3(2015), pp.737{761. [32] A.M.Stuart Inverseproblems:ABayesianperspective ,ActaNumerica,19(2010),pp.451{559. [33] AlbertTarantola InverseProblemTheoryandMethodsforModelParameterEsti mation ,siam,2005.