Citation
Simulation-based optimal experimental design: a measure-theoretic perspective

Material Information

Title:
Simulation-based optimal experimental design: a measure-theoretic perspective
Creator:
Butler, Troy
Pilosov, M.
Walsh, S.
Physical Description:
Journal Article preprint

Notes

Abstract:
A new approach for simulation-based optimal experimental design is developed based on a measure-theoretic perspective for computational deterministic models. Here, an experimental design refers to an experiment defining a particular map from the space of model inputs to the space of observable model outputs. The term optimal experimental design then refers to the choice of a map from model inputs to observable model outputs with optimal geometric properties. The quantifiable geometric properties considered here are the skewness and scaling effect of a map, which are related to the accuracy and precision of solving a stochastic inverse problem. We prove efficient computable approximations of average skewness and scaling effects based on singular value decompositions of sampled Jacobian matrices of the proposed maps. A recently developed measure-theoretic framework is subsequently used to formulate and solve stochastic inverse problems requiring no additional assumptions. Several examples illustrate the various concepts throughout this work. A more detailed numerical example showing a full end-to-end quantification and reduction of uncertainties is also provided demonstrating the process of solving stochastic inverse problems using optimally chosen maps in order to inform predictions.
Acquisition:
Collected for the Auraria Institutional Repository by the AIR Self-Submittal tool. Submitted by Troy Butler.
Publication Status:
Unpublished

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
Copyright Troy Butler. Permission granted to University of Colorado Denver to digitize and display this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.

Auraria Membership

Aggregations:
Auraria Library
University of Colorado Denver

Downloads

This item is only available as the following downloads:


Full Text

PAGE 1

DRAFT SIAMJ.S CI. C OMPUT. c r xxxxSocietyforIndustrialandAppliedMathematics Vol.xx,pp.x x{x Simulation-BasedOptimalExperimentalDesign:AMeasureTheoretic Perspective T.Butler M.Pilosov y and S.Walsh z Abstract. Anewapproachforsimulation-basedoptimalexperimentald esignisdevelopedbasedonameasuretheoreticperspectiveforcomputationaldeterministicmo dels.Here,anexperimentaldesignrefersto anexperimentdeningaparticularmapfromthespaceofmode linputstothespaceofobservable modeloutputs.Thetermoptimalexperimentaldesignthenre ferstothechoiceofamapfrom modelinputstoobservablemodeloutputswithoptimalgeome tricproperties.Thequantiable geometricpropertiesconsideredherearetheskewnessands calingeectofamap,whicharerelated totheaccuracyandprecisionofsolvingastochasticinvers eproblem.Weproveecientcomputable approximationsofaverageskewnessandscalingeectsbase donsingularvaluedecompositionsof sampledJacobianmatricesoftheproposedmaps.Arecentlyd evelopedmeasure-theoreticframework issubsequentlyusedtoformulateandsolvestochasticinve rseproblemsrequiringnoadditional assumptions.Severalexamplesillustratethevariousconc eptsthroughoutthiswork.Amoredetailed numericalexampleshowingafullend-to-endquantication andreductionofuncertaintiesisalso provideddemonstratingtheprocessofsolvingstochastici nverseproblemsusingoptimallychosen mapsinordertoinformpredictions. Keywords. optimalexperimentaldesign,uncertaintyquantication, inverseproblems 1.Introduction.1.1.OverviewandMotivation. Acriticalchallengetothecommunityofcomputational mathematiciansistodevelopanend-to-endalgorithmicfra meworkandanalysisforthequanticationandreductionofuncertaintiesincomputational modelsofphysicalsystems.Asa rststep,modelinputsareoftendescribedasbelongingtoa setofphysicallyplausiblevalues usingeitherengineeringordomain-specicknowledge.The rangeofmodeloutputsmaythen beapproximatedbyanensembleofdatausingthemodelandsam plesofmodelinputs.When theoutputdataaresensitivetoperturbationsinmodelinpu ts,thepredictivecapabilitiesof eventhemostsophisticatedcomputationalmodelsmaybesev erelylimitedbysuchacoarse descriptionofuncertaintiesinthemodelinputs. Specicationofaprobabilitymeasureonthespaceofmodeli nputsinordertoemploya forwarduncertaintyquantication(UQ)methodisoftenthe nextimportantstepinenhancingthequantitativecapabilitiesofthemodel.Forexample ,samplingaprobabilitymeasure onthespaceofmodelinputsandevaluatingthemodelonthese samplesgeneratesastatisticalensembleofpredicteddatafromwhichcondenceinter vals/boundscanbecomputed. Determiningaphysicallymeaningfulprobabilitymeasureo nthespaceofmodelinputsoften requirestheformulationandsolutionofastochasticinver seproblemusingwhatever(uncerDepartmentofMathematicalandStatisticalSciences,Univ ersityofColoradoDenver,Denver,CO80202 ( Troy.Butler@ucdenver.edu ). y DepartmentofMathematicalandStatisticalSciences,Univ ersityofColoradoDenver,Denver,CO80202 ( Michael.Pilosov@ucdenver.edu ). z DepartmentofMathematicalandStatisticalSciences,Univ ersityofColoradoDenver,Denver,CO80202 ( Scott.Walsh@ucdenver.edu ). 1

PAGE 2

DRAFT 2 T.Butleret.al. tain)dataareavailableonobservablemodeloutputs.Insom ecases,wemaybeabletodene certainaspectsofanexperimentthatdenethetypesofobse rvablemodeloutputsforwhich wecollectdatausedinthesolutiontothestochasticinvers eproblem.Thechoiceofobservable modeloutputs,whichwerefertoasquantitiesofinterest(Q oI),denesaparticularQoImap frommodelinputstomodeloutputs.TwodierentQoImaps,cor respondingtotwodierent experiments,mayleadtosignicantlydierentsolutionsto astochasticinverseproblem.This workfocusesonquantifyingspecicgeometricpropertieso finvertingsetsthroughtheQoI mapsinordertodeneageneraloptimalexperimentaldesign (OED)problemdenedover thespaceofallpossibleQoI.Thisapproachisrootedinmeas uretheoryandthereforerequires veryfewassumptionsonthespacesandtheQoImapsconsidere d. 1.2.RelationshiptoPriorandOtherWork. Theworkof[ 10 ]rstintroducedthegeometricconceptofskewnessinaQoImap,butwaslimitedinsev eralwaysthatareremedied andexpandeduponinthiswork.First,nocomputationallye cientwaytocomputeskewness wasprovidedotherthanbyinspectionofshapesoflower-dim ensionaldatamanifolds(which isnotalwayspossible)orbyrepeatedapplicationofGram-S chmidtalgorithmstogradient vectorsofaQoImap.Second,noconceptofhowaQoImapscales inverseeventsinmeasure wasconsideredwhenchoosinganoptimalQoImap.Finally,th eideaofskewnessinamap waspresentedin[ 10 ]asintimatelyconnectedtotheuseofameasure-theoreticU Qframework fortheformulationandsolutionofastochasticinversepro blem. Inthiswork,weproveecientcomputablemethodsforapprox imatingbothskewnessand scalingeectsusingsingularvaluedecompositionsofsampl edJacobiansofaQoImap.A generalmulticriteriaoptimizationproblemisdenedwhos esolutiondeterminestheoptimal QoImapbasedonthesequantiedgeometricpropertiesandis completelyindependentof anyparticularUQframeworkormethod.Thereareinfactmany UQmethodsthatrequire dierentassumptionsandsolvedierentstochasticinversep roblems.Forexample,there aremanymethodsbasedontheBayesianparadigm(e.g.,see[ 11 33 32 ]),dataassimilation methodsthatarebasedeitherontheKalmanFilterorvariati onaltechniques(e.g.,see[ 22 15 14 ]formethodsbasedonthefamousEnsembleKalmanFilter,and see[ 31 29 ]formethodsthat exploitadjointbasedtechniquesandarerelatedtoregular izationandleast-squarestechniques [ 27 ]),andrecentlyameasure-theoreticframeworkdevelopedi n[ 6 7 8 ].Afullexpositionand analysisoftheeectoftheoptimallychosenQoIonsolutions todierenttypesofstochastic inverseproblems,formulatedandsolvedwithdierentUQmet hods,isbeyondthescopeofthis work.Wethereforelimitthediscussionandnumericalexamp lestousingthemeasure-theoretic frameworkforformulatingandsolvingstochasticinversep roblems,andleaveapplicationto otherUQmethodstofuturework. ItisworthnotingthatOEDcanmeanagreatmanythingsdepend ingonthecontext. Thetermexperimentaldesign(sometimesreferredtoasdesi gnofexperiments),waslikely rstpopularizedwithinthestatisticalcommunity,e.g.,s ee[ 23 ]andthereferencesthereinfor earlierreferencesonthetopic.Thegeneralprocessofdete rminingtheoptimaldesigniscarried outbyanalyzingsomecollectionofhypothesizedstatistic almodelsthatrelatethepredictors (modelinputparameters)totheresponsevariables(QoI).T heoptimaldesignischosenwith respecttoastatisticalcriterion,e.g.,thestatisticalm odelthatminimizesvariance.The interestedreadercanreferto[ 1 ]orthemorethorough[ 3 ]formorerecentreferencesonthe

PAGE 3

DRAFT ExperimentalDesignwithMeasureTheory 3 topic.Weemphasizethatdevelopingappropriatestatistic almodelsandevaluationcriteria arethekeystepsanddiersubstantiallyfromtheapproachpr oposedhereforphysics-based computationalmodels.Theapproachweproposerequiresaco mputationalmodelwhereany choiceofQoIdenestherelationshipbetweenthe\predicto r"and\response"variables,and theevaluationcriteriaareintermsofgeometricpropertie softhisQoImap.Itmaybe possibletodeneaframeworkthatutilizesthesedierentpe rspectivesonOED,e.g.,inorder todeterminethebestsurrogatemodelforsomeproposedseto fQoI,butthisisalsoleftfor futurework. Morerecently,teamsofresearchershaveconsideredasimul ation-basedBayesianOED approach[ 21 5 ].Theresultsappearquitepromising.However,thattypeof approachis ofteninspiredmoredirectlyfromthestatisticalliteratu reandisalsointimatelytiedtothe UQmethodproposedforthesolutionofastochasticinversep roblemtoidentifyparameters. Itisalsosimilartoapproachestakeninthegeoscienceandh ydrologycommunities[ 24 30 ]. Additionally,thattypeofsimulation-basedapproachmayr equiremultiplesolutionsofthe actualstatisticalinverse/inferenceprobleminordertod etermineanoptimaldesignthatleads tomaximuminformationgain,whichmayresultinalargercom putationaloverhead.This makesadirectcomparisontowhatweproposeratherarbitrar yastheactualstochasticinverse problem,UQmethodsused,andalgorithmicproceduresimple mentedaredistinctlydierent. 1.3.Outline. Therestofthispaperisoutlinedasfollows.Section 2 describesthespaces andassumptionsformingthefoundationofthiswork.InSect ion 3 ,wedenethefundamental geometricaspectsofsetinversionthatarequantiedandde scribetherelationtoaccuracyand precisionofsolutionstothestochasticinverseproblem.T heOEDproblemissubsequently denedinSection 4 .Webrieryreviewthemeasure-theoreticframeworkforthef ormulation andsolutionofastochasticinverseprobleminSection 5 .InSection 6 ,wediscussthecomputationalconsiderationsusingareal-worldapplicationof stormsurgemodelingthatappears in[ 19 ]wheresomeoftheseideaswereappliedtochooseoptimalbuo ylocationsforsimulated HurricaneGustavdata.Whileseveralshortexamplesarepro videdthroughouttoillustrate specicconceptsandissues,wepresentafullend-to-endqu anticationandreductionofuncertaintiesexampleinSection 7 thatusesacanonicalmodelofheattransferinmanufactured thinmetalplatestodemonstrateafullunicationoftheide aspresentedherein.Weendwith concludingremarksandsomepossiblefutureresearchdirec tionscurrentlybeinginvestigated inSection 8 2.SpacesofModelInputs,ModelOutputs,andQoIMaps.2.1.BasicAssumptionsandTerminology. Forsimplicity,werefertoallmodelinputsas eithermodelparametersorjustparameterswhenthecontext isclear,andthespaceofmodel inputsisthenreferredtoastheparameterspace,whichisde notedby .Then,aQoImapis a(possiblyvector-valued)functionofparametersandofte ndenotedby Q ( ).Theoutputsof theQoImaparereferredtoasthedata,andtherangeofaQoIma piscalledthedataspace, whichisoftendenotedby D := Q ( ). Weassumethat R n and D R d andthat( ; B ; )and( D ; B D ; D )aremeasure spacesusingtheBorel -algebras.Themeasures and D arereferredtoasvolumemeasures eventhoughtheytechnicallymaynotbeLebesguemeasuresal thoughtheyareanobvious

PAGE 4

DRAFT 4 T.Butleret.al. choicewhichweoftenuseundercertainassumptions.Thepur poseofthesemeasuresisto describethesizesofsets,whichweusetoevaluatethepreci sionofaQoImapinsolving stochasticinverseproblemsasdiscussedinSection 3 Itisworthnotingthatunlesssomespecicconditionsarepl acedonthecomponentsof theQoImap Q D mayexistasalower-dimensionalmanifoldembeddedin R d .Insucha case,theusual d -dimensionalLebesguemeasureassignszeromeasuretoalle ventsin B D ,i.e., alleventsin B D aredeemedashavingthesame\size".Thisissueistechnical lydicultto dealwithwhendescribingprobabilitymeasuresintermsofd ensities,andonewaytoavoid thisdicultyistolet D bedenedbythepushforwardmeasureof Welet Q denotethespaceofalllocallydierentiableQoImapsunderc onsiderationby themodelerforcollectingdatatosolveastochasticinvers eproblem.Forexample,suppose themodelerisconsideringtwodierentexperimentaldesign sthatleadtoQoImaps, Q ( a ) : !D ( a ) ; or Q ( b ) : !D ( b ) : Then, Q = n Q ( a ) ;Q ( b ) o : Foranymap Q 2Q ,weareinterestedinhowthismapaectsthelocalgeometricp ropertiesrelatedtotheshape(specicallytheskewness)andsca lingofthevolumemeasureofthe inverseofanoutputevent E 2B D .Conceptually,theskewnessof Q describesageneralincreaseingeometriccomplexityof E under Q 1 ,whichquantitativelyisrelatedtothenumber ofsamplesin requiredtoaccuratelyestimate ( Q 1 ( E )).Thelocalscalingeectof E by themap Q 1 describestheprecisionofusingthemap Q 2Q toidentifyparametersthatmap to E .InSection 3 ,weshowthatboththelocalskewnessof Q 1 ( E )andthelocalscalingof E bythemap Q 1 (asdeterminedbythe -measure)canbedescribedintermsofsingular valuesofaJacobianof Q .Welet J ;Q 2 R d n denotetheJacobianof Q atapoint 2 andwhentheQoImapbeingconsideredisclearfromcontext,w esimplyuse J Afewremarksareinorder.Forsimplicityindescribingsets ,wewillignoreanyboundaries of or D .However,inpractice,weassumethat iscompact,which,bytheassumed smoothnessofany Q 2Q ,givesthat D isalsocompact.Inotherwords,weassumethat ( )and D ( D )arenite,whichisoftenthecaseinpracticebytheintrodu ctionofknown orassumedboundsofparameteranddatavalues.Second,sinc etheskewnessandscaling propertiesof Q mayvarysignicantlythroughout when Q isnonlinear,wemustaccount forthisvariabilityindeterminingoptimalQoI. 2.2.Dening Q Here,weprovideajusticationforrestrictingthedimensi onalityofQoI mapsthatdene Q .Sinceweassumeevery Q 2Q issmooth,werstsimplifythediscussion byconsideringthecasewhere Q islinearandthengeneralizetononlinearbut(piecewise) dierentiable Q .Since Q isassumedlinear,then Q ( )= J where J 2 R d n .If d>n ,then thereareatmost n rowsof J thatarelinearlyindependent.If d n ,thentherearemost d rowsof J thatarelinearlyindependent.Ineithercase,let m n denotethenumberofrowsof J thatarelinearlyindependent.Basicresultsfromlinearal gebraimplythat D = Q ( ) R d isdenedbythecolumnspaceof J andisgivenbyavectorsubspaceofdimension m .Thus, D isdenedbyan m -dimensionalhyperplaneembeddedin R d ,and B D isaBorel -algebra

PAGE 5

DRAFT ExperimentalDesignwithMeasureTheory 5 thatcanbeconstructedby m -dimensionalgeneralizedrectanglesdenedwithrespectt oan orthogonalsetof m vectorsin R d Let ^ J denoteanysubmatrixof J denedbychoosingany m linearlyindependentrowsof J ,anddenoteby ^ Q ( )themapinto R m denedby ^ J .Wesimilarlydenote ^ D = ^ Q ( ) R m and ^ B D astheBorel algebraof m -dimensionalsetsinan m -dimensionalspace.Let P denote a m d projectionmatrixfrom D to ^ D .Weabusenotationandlet PE denotetheactionof thematrix P onallvectorsin E 2B D ,i.e., PE 2 ^ B D istheprojectionoftheevent E onto ^ D .Thus,forany E 2B D ,thereisaunique ^ E = PE 2 ^ B D representingtheprojectionof E onto ^ D .Theimplicationisthatforany E 2B D Q 1 ( E )= ^ Q 1 ( ^ E ) 2B : Thisimplicationleadstoaninterestinginterpretationof thefollowingLemmasummarizedin Remark 2.2 Lemma2.1. Suppose Q 2Q isa d -dimensionalmap, J ;Q hasconstantrank m in isaproductmeasure,and E isanysetin B D .Forevery > 0 ,thereexistsapiecewiselinear m -dimensionalmap ^ Q : ^ D ,andanitenumber K of m -dimensionalgeneralized rectangles, n ^ E k o 1 k K ^ B D ,suchthatif D denotesthesymmetricdierenceof Q 1 ( E ) and [ 1 k K ^ Q 1 ( ^ E k ) ,then ( D ) < Proof Since Q isassumeddierentiable,itismeasurable,and Q 1 ( E ) 2B .Let > 0. Astandardresultinmeasuretheoryisthatforany A 2B ,thereexistsanitenumber K ofgeneralizedrectangles, f A k g 1 k K ,suchthat ( A 4[ 1 k K A k ) <; e.g.,seeTheorem2.40in[ 16 ].Let A = Q 1 ( E ) andapplysucharesult.Foreach1 k K ,choose 2 A k anddenoteitby ( k ) ,andlet Q ( k ) denotetheanemapon A k givenby Q ( ( k ) )+ J ( k ) ;Q : Bytheconstantrankassumption,dene ^ Q asthe m -dimensionalpiecewiselinearmapconstructedbytaking m n submatricesof J ( k ) ;Q andthesame m rowsof Q ( ( k ) )foreach 1 k K .Finally,let ^ E k = ^ Q ( A k ) ; foreach1 k K: Theconclusionfollowsbythechoiceof A k andthepropertiesof ^ Q WhiletheaboveLemmaprovidesatheoreticalmotivationfor developingandusingsurrogatepiecewiselinearmaps,itdoesnotprovideacomputat ionalapproachforconstructing thegeneralizedrectangles f A k g 1 k K usedintheproof.Moreover,itisnotnecessarilythe casethatsuchasurrogateisapointwiseaccurateapproxima tiontotheexactQoImapsince thediametersofanyindividual A k usedintheproofcanbearbitrarilylargeevenif ( A k ) issmall.Thisimpliesthatextrapolationerrorsinthesurr ogatecanbelarge.Additionally,

PAGE 6

DRAFT 6 T.Butleret.al. thereisnoguaranteethatwecanobservedatarelatedtothis surrogateawayfromthepoints Q ( ( k ) )duetothepossibleextrapolationerrorsdiscussedabove. Remark2.1 Weingeneraldonotknowwhichevents E 2B D areimportanttoconsider priortocollectingdataforaQoI.Lemma 2.1 employsthefactthatany E 2B D canbe approximatedarbitrarilywellusinganitenumberofgener alizedrectangles.Itisalsocommonfor -algebrasonhigherdimensionalspacestobegeneratedfrom generalizedrectangles. Therefore,intheremainderofthispaper,unlessexplicitl ystatedotherwise,wewillassume thatanyevent E 2B D isageneralizedrectangle. Remark2.2 Lemma 2.1 hasaninterestingconceptualimplicationintheOEDcontex t. Specically,iftheJacobianmatrixof Q hasconstantrank m in ,thenthelocalapproximationof Q 1 ( E )in -measurecannotbesignicantlyimproveduponbyusingmore than m n geometricallydistinct modelobservablestodene Q [ 8 ].Here,by geometricallydistinct ,wesimplymeanthat J ;Q hasfullrankforall 2 WithLemma 2.1 andRemark 2.2 inmind,wehereinafteradditionallyassumethatany Q 2Q isavector-valueddierentiablefunctionofgeometrically distinctcomponentmaps, whichimpliesthat d n 3.Accuracy,Precision,andSVDofJacobians. Webeginwithatechnicallemmaand ensuingcorollaryusedtodescribethesizesof d -dimensionalparallelepipedsembeddedin n dimensionalspaces.Weareinterestedintwocases: d -dimensionalparallelepipedsdened bytherowsofagivenmatrix J ,and d -dimensionalparallelepipedsdeterminedbythecross sectionsof n -dimensionalcylindersthataredenedbythepre-imageofa d -dimensionalunit cubeunder J Lemma3.1. Let J beafullrank d n matrixwith d n ,and Pa ( J ) denotethe d dimensionalparallelepipeddenedbythe d rowsof J .TheLebesguemeasure d in R d of Pa ( J ) isgivenbytheproductofthe d singularvalues f k g dk =1 of J ,i.e., d ( Pa ( J ))= d Y k =1 k : (3.1) Proof Thesingularvaluesof J areequaltothesingularvaluesof J > .Considerthe reducedQRfactorizationof J > J > = ~ QR; (3.2) where ~ Q is n d and R is d d .BythepropertiesoftheQRfactorization,weknowthe singularvaluesof R arethesameasthesingularvaluesof J > .Let x 2 R d ,then jj ~ Q x jj 2 =( ~ Q x ) > ( ~ Q x )= x > ~ Q > ~ Q x = x > x = jj x jj 2 ; (3.3) so ~ Q isanisometry.ThisimpliestheLebesguemeasureofthepara llelepipeddenedbythe rowsof R isequaltotheLebesguemeasureoftheparallelepipeddene dbythecolumnsof

PAGE 7

DRAFT ExperimentalDesignwithMeasureTheory 7 J > ,ortherowsof J d ( Pa ( J ))= d ( Pa ( R ))= d Y k =1 r k = d Y k =1 k ; (3.4) where f r k g 1 k d arethesingularvaluesof R and f k g 1 k d arethesingularvaluesof J Wenowturnourattentiontothesecondcaseofdescribingthe sizeofa d -dimensional parallelepipeddeterminedbythecrosssectionofan n -dimensionalcylinderdenedbythe pre-imageofa d -dimensionalunitcubeunder J .Inthiscase,weconsiderthepseudo-inverse of J J + = J > ( JJ > ) 1 .Asisevidentfromtheformulaofthepseudo-inverse,thera ngeof J + isequaltotherowspaceof J .Thisimpliesthatthe d -measureofthecross-sectionofthe pre-imageofaunitcubeunder J isequaltothethe d -measureoftheparallelepipeddened bythecolumnsof J + Corollary3.2. Let J beafullrank d n matrixwith d n .Then Pa (( J + ) > ) isa d dimensionalparallelepipeddeningacross-sectionofthe pre-imageofa d -dimensionalunit cubeunder J anditsLebesguemeasure d isgivenbytheinverseoftheproductofthe d singularvalues f k g dk =1 of J ,i.e., d ( Pa (( J + ) > ))= d Y k =1 k 1 : (3.5) Proof Considerthepseudo-inverseof J J + = J > ( JJ > ) 1 : Fromthisequation,itisclearthatthecolumnspaceof J + isequaltotherowspaceof J Therowspaceof J denesasubspaceorthogonaltothe n -dimensionalcylinderthatisthe pre-imageofaunitcubeunder J .Therefore,thecolumnspaceof J + isorthogonaltothe pre-imagecylinderand Pa (( J + ) > )isa d -dimensionalparallelepipeddeninganorthogonal cross-sectionofthiscylinder. Frombasicresultsinlinearalgebra,thesingularvaluesof ( J + ) > areequaltothoseof J + Then,frompropertiesofthepseudo-inverse,thesingularv aluesof J + aretheinverseofthe singularvaluesof J .Finally,fromLemma 3.1 ,itfollowsthat d ( Pa (( J + ) > ))= d Y k =1 k 1 ; (3.6) where f k g dk =1 arethesingularvaluesof J 3.1.SkewnessandAccuracy. Supposewecanchoosebetweentwoexperimentsleading totwodistinctchoicesofQoImapssothat Q = Q ( a ) ;Q ( b ) .Wegenerallyapproximate solutionstostochasticinverseproblems,e.g.,usingnit erandomsampling,andweexpect thatcertainchoicesofQoImapsmayleadtomoreaccuratesol utionsundertheconstraintsof axedcomputationalbudget.Below,wedenetheskewnessof aQoImap,provideameans

PAGE 8

DRAFT 8 T.Butleret.al. forcomputingtheexpectedskewness,anddescribetherelat ionshiptoaccurateapproximation ofinversesetswithnitesampling. First,assumethat Q islinearand E 2B D ageneralizedrectangle.When d = n Q 1 ( E ) isa d -dimensionalparallelepipedin (ignoringanyaectsfrompossibleintersectionswith theboundaryof )thatisin1-to-1correspondencewith E .If d
PAGE 9

DRAFT ExperimentalDesignwithMeasureTheory 9 Conceptually, S Q ( J ; j k )describestheamountof redundant informationpresentinthe k th componentoftheQoImapcomparedtowhatispresentintheoth er d 1componentswhen invertingnearthepoint 2 .Thesmallestvalue S Q ( )canbeisone.Thereisnolargest valuesincethereexistsmaps Q thathavegeometricallydistinctcomponentmaps,butthe conditionoftheJacobianmaybearbitrarilylarge.IftheJa cobianweretoeverfailtobefull rank,then S Q ( )wouldbeinnite.However,theassumptionofgeometricall ydistinctQoI preventsthisfromoccurring.Remark3.1 In[ 10 ],thefundamentaldecompositionresultofTheorem 3.3 wasusedtoshow thatthenumberofsamplesdeningregulargrids(andthusal souniformi.i.d.setsofsamples) in requiredtoobtainaccurateapproximationsin -measureof Q 1 ( E )isproportionalto sup 2 ( S Q ( )) d 1 : However,inthatwork,therewasnodiscussionofhowtocompu te S Q ( ) otherthananimplicitassumptionofrepeatedapplications ofaGram-Schmidtprocesstoobtain j j ?k j ,andinspectionoflowdimensionaldataspaceswastheprima rytooltorankproposed QoImapsbytheirskewness.Below,weexploitecientSVDcom putationstocomputeboth localandexpectedskewness,andweprovidenumericalresul tsdemonstratinghowexpected valuesofskewnessrelatetoan expected accuracyininversesetapproximations. ThefundamentaldecompositionofTheorem 3.3 alongwithLemma 3.1 providesaconvenientmethodfordeterminingtheskewnessintermsofthe d -dimensionalparallelepipeds describedby Q Corollary3.5. Forany Q 2Q S Q ( ) canbecompletelydeterminedbythethenormsof n -dimensionalvectorsandproductsofsingularvaluesofthe JacobianofQmapsofdimenions d 1 and d S Q ( )=max 1 k d j j k j d 1 ( Pa ( J k; )) d ( Pa ( J )) : (3.9) Proof S Q ( )=max 1 k d S Q ( J ; j k )=max 1 k d j j k j j j ?k j =max 1 k d j j k j d 1 ( Pa ( J k; )) d ( Pa ( J )) ; (3.10) thenapplyingLemma 3.1 wehave max 1 k d j j k j d 1 ( Pa ( J k; )) d ( Pa ( J )) =max 1 k d j j k j Q d 1 r =1 k;r Q dr =1 r : (3.11) where f r g 1 r d arethesingularvaluesof J and f k;r g 1 r d 1 arethesingularvaluesof J k; Corollary 3.5 impliesthatwecanexploitecientsingularvaluedecompos itionstoalgorithmicallyapproximate S Q ( )atanypoint 2 .Since S Q ( )mayvarysubstantiallyover ,wemustquantifythisvariabilityinordertooptimallycho ose Q 2Q Denition3.6. Forany Q 2Q ,wedenethe average (or expected )skewnessas S Q = 1 ( ) Z S Q ( ) d ; (3.12)

PAGE 10

DRAFT 10 T.Butleret.al. Generally,weapproximate S Q usingMonteCarlotechniquestogenerateasetofindependentidenticallydistributed(i.i.d.)samples ( i ) Ni =1 andcompute S Q S Q;N := 1 N N X i =1 S Q ( ( i )) : (3.13) Remark3.2 Noticethedescriptionsofskewnessareindependentofgene ralizedrectangle E 2B D .Inotherwords,skewnessisapropertyinherenttothemap Q itselfanddescribes thewayinwhichthegeometryof E 2B D ischangedbyapplying Q 1 Whilelinearmapscanoftenbeusedtodirectlyconstruct,or atleastdescribeexactly, theinversesofsets,weconsidertheuseofnitesamplingin toapproximatetheinverse sets.Thisismotivatedbyseveralcomputationalconsidera tionsofmanyUQmethodsandthe typesofmaps Q 2Q weconsiderasdescribedbelow. ItiscommonforUQmethodstousenitesamplingtodescribet heapproximatesolutions. Wewouldprefertore-useanycomputationsfromthesolution totheOEDproblem(e.g.,in approximatingminimalvaluesofexpectedskewnessusingEq .( 3.13 ))forsolvingthestochastic inverseproblem.SeeSection 6 formorediscussiononcomputationalconsiderations. 3.2.ScalingandPrecision. Tomotivatewhatfollows,considerthesimpleproblemwhere wemustchoosebetweentwodierentexperimentsleadingtotw odistinctQoImaps Q ( a ) and Q ( b ) sothat Q = Q ( a ) ;Q ( b ) .Let E ( a ) and E ( b ) representthesetofallprobableobservations fromusingeither Q ( a ) or Q ( b ) ,respectively.Then,dependingonwhichexperimentweobse rve, wewouldconcludethateithertheparametersbelongto Q ( a ) ; 1 ( E ( a ) )or Q ( b ) ; 1 ( E ( b ) )almost surely.Suppose ( Q ( a ) ; 1 ( E ( a ) )) ( Q ( b ) ; 1 ( E ( b ) )) ; thenwegenerallyexpectthatensemblesofparametersample sgeneratedfromresultsbasedon theexperimentleadingto Q ( a ) willhavesmallervariance(whichisadescriptionofprecis ion instatisticalterms)thanthosebasedontheexperimentlea dingto Q ( b ) .Thismotivatesa generalmeasure-theoreticgoalfordesigningexperiments whereeventsofhighprobabilityon adataspacearemade\smallinvolume"ontheparameterspace byinvertingtheQoImap. Webeginwithasimplifyingassumptionthat Q 2Q islinear,withgeometricallydistinct componentmaps.Then,thereexistsa d n matrix J ,suchthat Q ( )= J .If d = n and = R n ,itiseasilyshownfromstandardresultsinmeasuretheorya ndlinearalgebrathat ( Q 1 ( E ))= D ( E )det( J 1 )= D ( E ) d Y k =1 k 1 ; (3.14) where f k g 1 k d arethesingularvaluesof J .Notethatif R n isproper,thentheabove equationisnotnecessarilytrueas Q 1 ( E )mayintersecttheboundaryof .Weneglect suchboundaryeectsinthecomputations,andsimplynotetha tincertaincasestheymay playanimportantrolealthoughthisisnotthetypicalcasei nourexperience.If d
PAGE 11

DRAFT ExperimentalDesignwithMeasureTheory 11 recallthat Q 1 ( E )isdenedbyacylinderin withcrosssectionsgivenby d -dimensional parallelepipeds.WecanuseCorollary 3.2 tocomputethemeasureofthese d -dimensional parallelepipeds.Thismotivatesthefollowing Denition3.7. Forany Q 2Q and 2 ,wedenethe local( -measure)scaling eect of Q as M Q ( )= d Y k =1 k 1 ; (3.15) where f k g 1 k d arethesingularvaluesoftheJacobian J ;Q .The average (or expected )) ( -measure)scalingeect isgivenby M Q = 1 ( ) Z M Q ( ) d : (3.16) Aswithaverageskewnessof Q ,wegenerallyapproximate M Q usingasetofi.i.d.random samples ( i ) Ni =1 andcomputing M Q M Q;N := 1 N N X i =1 M Q ( ( i )) : (3.17) Wesummarizetheaboveresultsintothefollowing Corollary3.8. Forany Q 2Q ,thelocalskewness,localscalingeect,averageskewness ,and averagescalingeectof Q canbeestimatedusingnormsofrow-vectorsandsingularval uesof theJacobian J ( i ) ;Q atanitesetofpoints ( i ) 1 i N Remark3.3 Analternativeto M Q thataccountsforpossiblydierent D ( E )istousesimple multiplicationof D ( E )assuggestedbyEq.( 3.14 ). 4.OptimalExperimentalDesign. Webeginwithasimplemotivatingexamplethatshows howtheskewnessinaQoImapaectsexpectederrorsin -measuresofinversesetsapproximatedbyanitesetofuniformi.i.d.randomsamples.While wecouldusesimpleMonte Carloestimatesofthe -measureofinversesets,weinsteadoptfortheVoronoicell sdened byasetofsamplesinordertobetterillustratetheerrorsin estimating -measureofinverse sets(e.g.,seeFigure 4.1 ). Example1. Suppose =[0 ; 1] 2 ,andwehaveachoicebetweentwoexperimentsleadingto twodistincttwo-dimensionallinearQoImaps Q ( a ) and Q ( b ) sothat Q = Q ( a ) ;Q ( b ) .Map Q ( a ) isdescribedbya 2 2 matrixwithorthogonalrow-vectorssothat S Q =1 andmap Q ( b ) isdescribedbya 2 2 matrixwithnon-orthogonalrowvectorssothat S Q 4 .Wechoose E ( a ) 2B ( a ) D and E ( b ) 2B ( b ) D astworectangleschosensothat Q ( a ) ; 1 ( E ( a ) ) and Q ( b ) ; 1 ( E ( b ) ) haveexactlythesame -measure.InFigure 4.1 ,weshowsomerepresentativeplotsofthe approximationsoftheseinverseeventsusingdierentnumb ersofuniformi.i.d.randomsamplesin .Theerrorsintheseapproximations,givenbythesymmetricd ierences,arevisually illustratedbythedarkestshadedregionanddecreasewitha dditionalsamples.InFigure 4.2

PAGE 12

DRAFT 12 T.Butleret.al. andTable 4.1 ,weshowtheconvergenceofthemeasureofthesymmetricdie rencetobenear theknownconvergencerateofMCmethods, N 1 = 2 .Inotherwords,weexpectconvergenceat thesameasymptoticratenomatterhowskewedthemap.Howeve r,theresultsfortheskewed map Q ( b ) generallyrequireapproximatelyfourtimesthenumberofra ndomsamplesinorderto obtainthesamelevelofexpectedaccuracyin -measureasseenusing Q ( a ) (seeTable 4.1 ). Figure4.1:Theexactinversesets Q ( a ) ; 1 ( E ( a ) )(leftcolumn)and Q ( b ) ; 1 ( E ( b ) )(rightcolumn) aretheintersectionofthedierentregionsineachplotandh avetheexactsame -measure. Theerrorin -measureoftheapproximationtotheseeventsisgivenbythe symmetric dierenceoftheexactinversesetswiththeimplicitlydene dVoronoicellsassociatedwith niterandomsamplingoftheseeventsandisillustratedbyt hedarkestshadedregion.We showthisusing50samples(top),and3200samples(bottom). Intheaboveexample,weonlyconsideredtheimpactofskewne ss.Forsucientlylarge numbersofsamples,theerrorin -measureofapproximatinginversesetsfromusingeither QoImapcanbemadearbitrarilysmall.Whentherearenolimit sonthecomputational budget,skewnessfromaQoImapmaynolongerbeaconcern,and wewouldgenerallychoose theQoImapthatleadstooptimalprecision.However,facedw iththepracticalrealities ofnitecomputationalresources,weassumethatthegenera lgoalistocomputeaccurate estimatesofinversesetsthatalsoimprovetheprecisionof predictions.Insomecases,the choiceistrivial.Forexample,if S Q ( a ) < S Q ( b ) and M Q ( a ) < M Q ( b ) ,thenweclearlychoosethe experimentleadingto Q ( a ) .However,ifwechangeoneofthoseinequalities,thenthech oice wemakebecomesambiguous.Below,wedescribeoneapproachf orquantitativelyassessing whichQoImapisoptimalinsuchcases.

PAGE 13

DRAFT ExperimentalDesignwithMeasureTheory 13 r r n Figure4.2: Convergenceplot(log-log scale)forthemeansymmetricdier-enceshowninTable 4.1 Num.Samples Q ( a ) Q ( b ) 50 4 : 66 E 2 8 : 01 E 2 200 2 : 37 E 2 4 : 45 E 2 800 1 : 22 E 2 2 : 33 E 2 3200 6 : 13 E 3 1 : 22 E 2 Table4.1: Meanerror(computedover100 trialsforeachnumberofsamples)givenbysymmetricdierencein -measureapproximationsofinversesetsusingnon-skewedandskewedmaps, Q ( a ) and Q ( b ) ,respectively. 4.1.AMulticriteriaOptimizationProblem. Welet S R denotethesetofallpossible valuesof S Q and M R denotethesetofallpossiblevaluesof M Q forall Q belongingtoa specied Q .Clearly, S isboundedbelowby1,whichrepresentsthecaseof\optimalg lobal skewness"inaQoImap Q .Similarly, M isboundedbelowby0,whichrepresentsthecase whereasetofoutputdataexactlyidentiesaparticulargen eralizedcontourofparameters responsibleforthedata. For Q ,wecanclearlyorderall Q 2Q accordingtoeitherthevaluesof S Q or M Q .In otherwords,thevaluesof S Q and M Q separatelydescribeanorderingindexonthespace ofallpossibleQoIgivenby Q .However,themapping Q 7! ( S Q ; M Q )describesadouble indexingwithnonaturalordering.Wedenemetricsinorder toquantifythedistancetothe optimalpointof(1 ; 0)intheCartesianproductspacedescribedbythepairs( S Q ; M Q ).While therearemanyoptionsfordeningmetrics,athoroughinves tigationontheeectofdierent metricsisbeyondthescopeofthisworkwhichisfocusedonth eexpositionofthisgeneral approach.Below,wechooseaparticularformforthemetrics on S and M thatminimizesthe eectofthepossiblydisparaterangesofvalueswemayobserv ein S and M onthesolution tothemulticriteriaoptimizationproblem. Wedene( S ;d S )and( M ;d M )usingthemetrics d S ( x;y )= j x y j 1+ j x y j forall x;y 2S ; (4.1) and d M ( x;y )= j x y j 1+ j x y j forall x;y 2M : (4.2) Choose 2 (0 ; 1)andlet Y denotetheCartesianproductspace SM ,withmetricdened by d Y ( x;y )= !d S ( x 1 ;y 1 )+(1 ) d M ( x 2 ;y 2 )forall x;y 2 Y : (4.3) Notethat determinestherelativeimportanceweplaceoneitherpreci sionoraccuracy.

PAGE 14

DRAFT 14 T.Butleret.al. Choosing =0implieswedisregardskewnessinthecriterionwhereasch oosing =1implies wedisregardthescalingeect.Remark4.1 Choosinganoptimal givenacomputationalbudgetisthetopicofcurrent research.Below,weprovidesomesimpleexampleswith =0 ; 0 : 5 ; and1toillustratehow dierentchoicesof inruencestheselectionoftheoptimalQoI. Denition4.1. Given Q and 2 (0 ; 1) ,the optimalexperimentaldesignproblem (OEDproblem) isdenedbythemulticriteriaoptimizationproblem min Q 2Q d Y ( p;y ) ; (4.4) where p =(1 ; 0) isthe idealpoint and y =( S Q ; M Q ) for Q 2Q 4.2.DiscreteOptimization. When Q denesasetwithnitecardinality,thesolutionto theminimizationproblemdenedbyEq.( 4.4 )canbefoundbyanexhaustivesearch.This problemcanclearlybecomecomputationallyexpensiveasth enumberofpossiblemapsgets large,anditmaybenecessarytoinitiallylter Q toasmallersetofpossibilities,e.g.,using domainspecicinformation.However,itiscompletelystra ightforwardtoimplementandis embarrassinglyparallel. Example2. Let R 2 andsupposewecanchoosefromacongurationofexperiments thatleadtoachoiceofanytwomeasurementsfrom f Q 1 ( ) ;Q 2 ( ) ;Q 3 ( ) g ; where Q 1 ( )=0 : 5 1 +0 : 5 2 ;Q 2 ( )=2 : 5 1 +0 : 5 2 ;Q 3 ( )= 0 : 2 1 +0 : 3 2 : Inotherwords, Q = Q ( a ) ;Q ( b ) ;Q ( c ) wherewehave Q ( a ) =( Q 1 ;Q 2 ) Q ( b ) =( Q 1 ;Q 3 ) ,and Q ( c ) =( Q 2 ;Q 3 ) .Thelinearityofthemapsmakescomputationsof S Q and M Q trivial.Wesee inFigure 4.3 andTable 4.2 theoptimalchoiceofpairofQoItouseintheinverseproblem for threedierentoptimizationcriteria;minimize M Q (choosing =0 ),minimize S Q (choosing =1 ),andminimizeEq. ( 4.3 ) with =0 : 5 .Itisvisuallyevidentfromtheleftplotof Figure 4.3 thattheQoIthatminimizes M Q alsoproducesaninversesetwiththesharpest corners,i.e.,thisisthemapwiththemostskewness.Inthemi ddleplotofFigure 4.3 we observetheoppositeeect,theQoIthatminimizes S Q maximizes M Q .Intherightplot ofFigure 4.3 ,weobserveaQoImapthatservesasthebest\compromise"and produces reasonablysmallvaluesof S Q and M Q simultaneously. 4.3.ContinuousOptimization. Inmanyphysicalapplications,whilewemaybelimited inplacingatotalof d measurementdevicescorrespondingto d geometricallydistinctobservablemodeloutputs,theactualcongurationofthedevicesm ayonlybelimitedtobelonging toaparticularspace-timecylinder.Forexample,wemaybea bletoplacetwocontactthermometersanywherealongathinrodinordertorecordthetemp eratureataparticularpoint intime,buttheremaybenoadditionalrestrictions.Insuch acase,thespace Q isdened implicitlyandrepresentsan uncountable set.Itmaybepossibletouse r Q S Q and r Q M Q (andpossiblyHessiansof S Q and M Q on Q ),toemploycontinuousoptimizationmethodsto determinelocallyoptimalsolutionstotheminimizationpr oblemdenedbyEq.( 4.4 ).While

PAGE 15

DRAFT ExperimentalDesignwithMeasureTheory 15 Figure4.3:TheinversesetsfromthethreedierentQoImapsd ening Q aregivenbythe intersectionoftheindividualcomponentmapinversesets. Fromlefttorightweshowthe eectofusing Q ( a ) =( Q 1 ;Q 2 ), Q ( b ) =( Q 1 ;Q 3 ),and Q ( c ) =( Q 2 ;Q 3 ). Opt.Criteria Optimal Q 2Q M Q S Q d Y ( p;y ), =0 : 5 Minimize M Q Q ( a ) 4 : 0 E 2 1 : 80 E +0 4 : 84 E 1 Minimize S Q Q ( b ) 1 : 6 E 1 1 : 02 E +0 1 : 57 E 1 Eq.( 4.3 ), =0 : 5 Q ( c ) 4 : 7 E 2 1 : 08 E +0 1 : 20 E 1 Table4.2:Weenumerate Q (rstcolumn)andshow M Q S Q and d Y (0 ;y )for =0 : 5inthe second,thirdandfourthcolumns,respectively.thisisbeyondthescopeofthiswork,weprovideaconcreteco nceptualexampletoillustrate thegeneralcomplexity.Furtherinvestigationofthispart iculartypeofoptimizationproblem islefttofuturework. Example3. Considertheproblemwhereweusetheheatequationtopredict temperatures onathinmetalrodofunitlength.Let n=[0 ; 1] denotethespatialdomain.Supposetherod ismanufacturedbyweldingtogethertwothinrodsofequalle ngth 0 : 5 andofsimilaralloytype together.However,duetothemanufacturingprocess,suppo setheactualalloycompositions maydierslightlyleadingtouncertainthermaldiusivity propertiesonthelefthalfandright halfoftherod,denotedby 1 and 2 ,respectively.Supposetheexperimentswecanruninvolve placingtwocontactthermometersalongthefullrodthatcan recordtwoseparatetemperature measurementsatthesametimewhentherodissubjecttoapart icularlowtemperaturesource. ThegoaloftheOEDistodeterminewheretoplacethetwocontac tthermometersinorderto quantifyandreduceuncertaintiesinthethermaldiusivit ypropertiesonbothsidesoftherod. Usingknowledgeofthetypesofalloysandimpuritiesintrod ucedbythemanufacturing process,supposethatforanyrod =( 1 ; 2 ) 2 =[0 : 01 ; 0 : 2] 2 .Thecongurationofthe contactthermometersonaroddenesacoordinate,whichwed enoteby ( x 1 ;x 2 2 n n ,where x and y denotethepositionsoftherstandsecondcontactthermome tersalong n ,respectively. Apoint ( x 1 ;x 2 ) thencorrespondstoaparticular Q 2Q mapping toa D .Wecantherefore plot S Q M Q and d Y ( p;y ) (foranychoiceof )asfunctionsover n n .Weshowsuchplots inFigure 4.4 .Intheseplots,lightercolorscorrespondtomoreoptimalva luesanddarker colorsarelessoptimal.Observethesymmetryalongtheline x 1 = x 2 ,whichcorrespondstoa renamingofcontactthermometeroneascontactthermometer twoandviceversa.Placingthe

PAGE 16

DRAFT 16 T.Butleret.al. 0.00 0.25 0.50 0.75 1.00 x 1 0.00 0.25 0.50 0.75 1.00x 2 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.00 0.25 0.50 0.75 1.00 x 1 0.00 0.25 0.50 0.75 1.00x 2 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.00 0.25 0.50 0.75 1.00 x 1 0.00 0.25 0.50 0.75 1.00x 2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 Figure4.4:Fromlefttoright:Plotsof M Q S Q andvalueof d Y ( p;y )with =0 : 5over Q 2Q denedbythespatiallocationoftherstmeasurementonthe horizontalaxisandthe spatiallocationofthesecondmeasurementontheverticala xis.Darkercolorsindicateless optimalchoicesof Q andlightercolorsindicatemoreoptimalchoicesof Q contactthermometerssymmetricallyawayfromthemidpoint |neartheendsoftherod|can leadtooptimalQoImapsforallofthedierentoptimization criteria,whichmatchesphysical intuition(seethelowerrightandupperleftcornersofthep lotsinFigure 4.4 ).However,there aregenerallyseverallocalmaximaandminimaintheseplots .Thus,givenanyinitialguess foranoptimalQoImap,andanyparticularoptimizationcrit eria,continuousoptimization algorithmswillingeneralproduceonlyalocally,notgloba lly,optimalQoImap. 5.FormulationandSolutionofaStochasticInverseProblem. Webrierysummarizethe measure-theoreticframeworkfortheformulationandsolut ionofastochasticinverseproblem. Formoredetailsonthetheoryandcomputationalalgorithms ,wedirecttheinterestedreader to[ 6 7 8 9 ].InSection 7 ,weshowtheeectofoptimallychoosingtheQoIonsolutionst o thestochasticinverseproblem. GivenaQoImap Q : !D andaprobabilitymeasure P D on( D ; B D ),thestochastic inverseproblemistodetermineaprobabilitymeasure P on( ; B ).When Q hasgeometricallydistinctcomponentmapsand d = n ,then P isuniquebyasimplechangeofvariables formula.However,when d
PAGE 17

DRAFT ExperimentalDesignwithMeasureTheory 17 oftheinverseproblem, Q denesabijectionbetween L and D ,sothemeasure P D uniquely determines P L butcannotprovideanyinformationabouttheprobabilities alongthegeneralizedcontours.Employinganansatz[ 8 ]forthefamilyofconditionalprobabilitymeasures denesaprobabilitymeasure P on( ; B )thatisuniquefortheparticularansatz.Following[ 8 ],weusethestandardansatzbasedontheuniquedisintegrat ionofvolumemeasures thateectivelyproportionsprobabilityalongcontoureven tsaccordingtotheirrelativesize. Theresultingprobabilitymeasuresexploittheavailableg eometricinformationresultingin non-parameteric(andoftenmulti-modal)solutionstothes tochasticinverseproblem.These solutionsaretypicallyapproximatedusingrandomsamplin g,see[ 8 9 ]formoreinformation anddetailsabouttheapproximationsandcomputationalalg orithms.Theinterestedreader mayalsosee[ 10 28 ]formoreengineeringorienteddiscussionsofthecomputat ionalalgorithm inthecontextofparticularapplicationsinvolvingshallo wwaterequationsandsubsurface contaminanttransport.Below,weprovideasimpleexamples howinghowtheskewnessina QoImapeectstheconvergencetothesolutionofthestochast icinverseproblem. Example4. Let =[0 ; 1] 2 R 2 andconsidertwolineartwo-dimensionalQoImaps Q ( a ) and Q ( b ) with S Q ( a ) =1 and S Q ( b ) =2 ,butchosensothat M Q ( a ) = M Q ( b ) =1 .Toillustrate howskewnessaectstheaccuracyinapproximatingtheinver sionofaprobabilitymeasureusing astraightforwardi.i.d.uniformrandomsamplingalgorith m[ 8 9 ],weconsiderthesimple problemofinvertingauniformdistributiondenedonrecta ngles E ( a ) D ( a ) and E ( b ) D ( b ) where E ( a ) and E ( b ) arechosensothat ( Q ( a ) ; 1 ( E ( a ) ))= ( Q ( b ) ; 1 ( E ( b ) ) ,andboth Q ( a ) ; 1 ( E ( a ) ) and Q ( b ) ; 1 ( E ( b ) ) areinteriorto .Withthesechoices,asimplechangeof variablesformulashowsthatthesolutiontothestochastic inverseproblemisgivenbyauniform distributiondenedeitheron Q ( a ) ; 1 ( E ( a ) ) or Q ( b ) ; 1 ( E ( b ) ) ,whichwecomputeonaregular uniformgridof 200 200 pointsin asareferencesolutionforeachcase. Notethattheprobabilityofanyofthei.i.d.randomsamples belongingtoeither Q ( a ) ; 1 ( E ( a ) ) and Q ( b ) ; 1 ( E ( b ) ) isexactlythesamesince ( Q ( a ) ; 1 ( E ( a ) ))= ( Q ( b ) ; 1 ( E ( b ) ) .However, theskewnessof Q ( b ) aectstheset Q ( b ) ; 1 ( E ( b ) ) sothatthereismoreapproximationerrorof theuniformprobabilitymeasureonthiseventinawaysimila rtoasshowninExample 1 .We usetheHellingermetric[ 17 ]tocomputethedistancefromarandomsampleapproximation tothereferenceprobabilitymeasureassociatedtoeitherm ap.InFigure 5.1 andTable 5.1 ,we showthemeanHellingerdistancevaluescomputedbyrepeati ngeachrandomsettrial50times. Observethatapproximatelytwiceasmanyrandomsamplesare requiredtoachieveasimilar levelofexpectedaccuracyintheapproximateprobabilitym easurefromusing Q ( b ) comparedto using Q ( a ) 6.ComputationalConsiderations. AsmentionedinSection 4 Q maydeneeithera niteorinnitesetofpossibleQoImaps.When Q isnite,wemayuseengineeringor domain-specicknowledgetoreducethesetfurther.Thisma yalsobedonewhen Q isinnite. Example5. Here,wesummarizeanexamplestudiedin[ 19 ]wheresomeofthecomputationalideasfromthisworkwereappliedtoarealapplicatio n.Theproblemistoidentifythe spatialcongurationoffourbuoysrecordingsimulatedmax imumstormsurgeforHurricane GustavinBaySt.Louisinordertoquantifyuncertaintiesin thespatiallyheterogeneousManning'scoecientoffriction.Allowingbuoystobelocateda tanypointinthedomainleads toanuncountablyinnitesetofcongurationsanddenesac ontinuousOEDproblem.In-

PAGE 18

DRAFT 18 T.Butleret.al. n r Figure5.1: Convergenceplot(log-log scale)usingmeanHellingerdistancesofapproximationstoareferenceprob-abilitymeasureasseeninTable 5.1 Num.Samples Q ( a ) Q ( b ) 200 1 : 35 E 01 2 : 01 E 01 400 9 : 93 E 02 1 : 45 E 01 800 6 : 95 E 02 1 : 03 E 01 1600 5 : 07 E 02 7 : 52 E 02 3200 3 : 68 E 02 5 : 39 E 02 6400 2 : 60 E 02 3 : 99 E 02 Table5.1: MeanHellingerdistance(computed over50trialsforeachnumberofsamples)ofrandomsampleapproximationtoreferenceprobabilitymeasuresolvedusing Q ( a ) and Q ( b ) where S Q ( a ) =1 and S Q ( b ) =2 stead,weappliedknowledgeofthemodelanddomaintoavoidc ongurationsthatleadtobuoys exhibitingeitherlittleorsimilarsensitivitiestotheMa nning'scoecientoffrictionvalues. Usingthisknowledge,weidentied194possiblelocations( seetheleftmostplotinthediagramofFigure 6.1 ).Thiscreateda Q denedby 194 4 =57 ; 211 ; 376 possiblecongurations ofthebuoys.Simulationswerecomputedusingthestate-ofthe-artADvancedCIRCulation (ADCIRC)model[ 26 4 18 ]requiringsignicantcomputationalresources,whichlim itedthe numberofsampleswecouldusein toafewhundredincludinganymodelsimulationsused tocomputeJacobiansoftheQoImaps.Therefore,wechosetoop timizeonlyforskewness. WeshowtheeectusingtheoptimalQoImapcomparedtoaQoIma pwithnon-optimalskewnessinFigure 6.1 wheretheinitiallteringoflocationsforsensitivitiesa lsoleadstogreater precisioninthemarginals.Forafulldiscussionoftheresu lts,see[ 19 ]. Theaboveexampleillustratessomeofthecomputationalcon siderationsinoptimizing Q undertheconstraintsofacomputationalbudget.Specical ly,wegenerallymustalways answerthefollowingquestions. Canevery Q 2Q besavedtoleforeachsimulationofthemodel? HowaretheJacobiansofeach Q computedtoapproximate S Q and M Q ? Canwere-useanymodelsimulationsinsolvingthestochasti cinverseproblem? WediscusssomeofthesequestionsinmoredetailinSection 8 .Withrespecttotheabove example,wewereabletostorethemaximumstormsurgeatall1 94proposedbuoylocationsfor eachsampleofManning'scoecientoffrictionvalues.Weus ednitedierencingcenteredat randomsamplestoapproximatetheJacobiansandcouldre-us ethesesamplesinthemeasuretheoreticinversion. Insomecases,itmaybepossibletosavepartsofasimulation inaspatial-temporalmodel inordertoapproximatethe Q 2Q .Forexample,wecouldselectasetoftimenodesatwhich thefullspatialsolutionissavedanda(piecewise)polynom ialtintimeisusedtosimulate thepossibleQoImapsatanypointinspace-time.Thiscreate sacomputationaloverheadthat aectstheoverallcomputationalbudget,butisoutsidethes copeofthiswork.

PAGE 19

DRAFT ExperimentalDesignwithMeasureTheory 19 0 09 0 100 110 120 130 140 1510 14 0 16 0 18 0 20 0 2220 40 80 120 160 200 240 280 320 360 1 ,2 (Lebesgue) Optimal Sub-Optimal UQ UQ 0 09 0 100 110 120 130 140 1510 14 0 16 0 18 0 20 0 2220 80 160 240 320 400 480 560 640 1 ,2 (Lebesgue) Figure6.1:Left:Thedotsidentifythe194proposedbuoysta tionsforrecordingmaximum surgeelevationinBaySt.Louis.Thecolorindicatesthedie renceinmaximumwaterelevationsovertheentirespaceofManning'scoecientoffricti onfordominantlandclassications. Middle:Optimalandsuboptimalbuoylocationsdeterminedb yrankingskewness.Right:Representative2Dmarginal( 1 ; 3 )probabilityplotsofdierentManning'scoecientoffrict ion valuescomputedfrommeasure-theoreticinversion.Thewhi tedotisthereferenceparameter usedtosimulatedataofHurricaneGustav.Figuresareadopt edfrom[ 19 ]. 7.NumericalExample:OEDandEnd-to-EndUQ.7.1.TheModeland Q Considerthesimpletimedependentdiusionmodelforheat transferinathinmetalplatewithuncertaindiusioncoeci entsthatbothsummarizesresults andmatchesphysicalintuition.Themodel,andparticularl oadusedduringexperiments,are describedbythepartialdierentialequation c @T @t = r ( r T )+ f;x 2 n ;t 2 ( t 0 ;t f ) ; where 8>><>>: f ( x )= AH ( t t s )exp ( x 1 ) 2 +( x 2 ) 2 w T ( x ;0)=0 ; @T @n =0 ;x 2 @ n ; and x =( x 1 ;x 2 ) 2 n=[ 1 2 ; 1 2 ] [ 1 2 ; 1 2 ]representsthedimensionsofthethinmetalplate.We let =1 : 5denotethedensityoftheplate, c =1 : 5theheatcapacity,and istheuncertain thermalconductivity.Theforcingfunction f representsanexternalheatsourcedescribedby aGaussianshapewithmaximumamplitudeatthecenterofthep late,andwelet A =50 and w =0 : 05.Thefunction H ( t t s )istheHeavisidefunctionindicatingthattheexternal sourceisturnedoattime t s = t f = 2,wherewetake t f =5 : 0.ThehomogeneousNeumann boundaryconditionsmodelperfectinsulationaroundthebo undary.

PAGE 20

DRAFT 20 T.Butleret.al. SimilartoExample 3 ,weassumethatthemetalplateismanufacturedbyweldingto getheralongtheverticalline x 1 =0tworectangularplatesofthesamesizeandalloytype. However,duetoimperfectionsintheoriginalmanufacturin gprocess,weassumethatthealloy compositioncanvarysignicantlyacrossthecenterlineof theplatedenedby x 1 =0.We let =( 1 ; 2 ) 2 =[0 : 01 ; 0 : 2] 2 denotethethermalconductivityoftheplatewhere 1 and 2 denotethethermalconductivitywhen x 1 < 0andwhen x 1 > 0(theleftandrighthalves), respectively. Forthenumericalsolutionofthemodel,weusethestateofth eartopen-sourcenite elementsoftwareFEniCS[ 2 25 ].Thespatialdiscretizationisgivenbya40 40triangular meshusingpiecewiselinearniteelements.Weusedaunifor mmeshtodiscretize[0 ;t f ]with 500totaltimestepsandaCrankNicolsontimesteppingschem e. Weassumeduringanexperimentthattwocontactthermometer scanbeusedtorecord thetemperatureattwoseparatepointsinspace-time.Thech oicesoftwospace-timepointsto recordthetemperaturedeterminesthespace Q .Thegoalistodeterminewhich Q 2Q helps usobtainthe\best"inferencesaboutthethermalconductiv itiesoneachsideoftheplate. Intuitively,weexpecttotakeonetemperaturemeasurement oneachsideoftheplate,butit isnotclearatwhattimestepseachtemperaturemeasurement shouldberecordedorexactly wheretoplacethesensorsoneachsideoftheplate. Forsimplicity,welimit Q toadiscretesetofpointsinspace-time.First,weselected 20 pointsontheplateforplacingthethermometers,seethepoi ntslabeled p i for1 i 20in theleftplotofFigure 7.1 .Sincevaluesoftemperatureataxedpointinspacearesimi lar inconsecutivetimesteps,weassumedthetemperaturewasre cordedatevery25 th timestep resultinginatotalof20dierentpointsintimeforrecordin gtemperature.Thisresultedin 400possibledierentspace-timelocationstoplaceacontac tthermometer,andchoosingany twodenesaQoImap.Inotherwords, Q isdenedby 400 2 =79 ; 800possibleQoImaps. Thesensorsofthecontactthermometersrecordingdataarem athematicallymodeledby takingtheaveragetemperatureonasmalldiscaboutapoint p i 2 nataxedtime.Specically,let B r ( p i )denotethesmalldisccenteredatpoint p i ,theneachQoIismodeledas Q ( p i ; t j ) := 1 n ( B r ( p i )) Z n T ( x ; t j ) B r ( p i ) dx: Here,thesubscriptgivenby( p i ; t j )isanindexingdescribingthepointinspace p i andtime t j ofthemeasurement.Inthecomputations,wetake r =0 : 05. 7.2.OEDandanInverseProblem. Thestochasticinverseproblemsformulatedbelow aresolvedusingtheopensourceBET[ 20 ]Pythonpackage,andthisexampleisavailable withinthedocumentedexamplesoftheBETpackage.Werefert heinterestedreaderto theBETpackageanddocumentationthereinformoredetailso nthecomputationsdiscussed below. Foreach Q 2Q ,weapproximate S Q and M Q usingEqs.( 3.13 )and( 3.17 ),respectively,to compute S Q;N and M Q;N with N =5000i.i.d.uniformrandomsamplesin .Wedetermine threeoptimalQoImapscorrespondingtothreedierentoptim izationcriteria;minimize M Q;N minimize S Q;N ,andminimizethedistancedenedinEq.( 4.3 )with =0 : 5.Let Q ( M ) Q ( S )

PAGE 21

DRAFT ExperimentalDesignwithMeasureTheory 21 Figure7.1:Left:TheQoIlocationsinthedomainnlabeled p i for1 i 20.Right:Semi-log scatterplotofpointsin Y givenby( S Q;N ; M Q;N )foreach Q 2Q .AlongtheParetofrontier fromtoplefttobottomright,thelargercirclesindicateth eQoImapswhichminimize S Q;N solveEq.( 4.3 )with =0 : 5,andminimize M Q;N ,respectively. Opt.Criteria OptimalQoImaps M Q;N S Q;N d Y ( p;y ) Minimize M Q;N Q ( M ) =( Q ( p 18 ;t 14 ) ;Q ( p 17 ;t 15 ) ) 1 : 72 E 05 1 : 46 E +00 3 : 31 E 01 Minimize S Q;N Q ( S ) =( Q ( p 1 ;t 1 ) ;Q ( p 12 ;t 1 ) ) 3 : 76 E +03 1 : 00 E +00 9 : 99 E 01 Eq.( 4.3 ), =0 : 5 Q ( Y ) =( Q ( p 17 ;t 4 ) ;Q ( p 18 ;t 4 ) ) 6 : 26 E 04 1 : 0004 E +00 9 : 84 E 04 Table7.1:Thedierentoptimizationcriteriaforchoosingo ptimalQoIandvaluesof ( M Q;N ; S Q;N )showninFigure 7.2 andthevalueof d Y ( p;y )usedtodetermineoptimalQoI mapbasedondierentoptimizationcriteria.and Q ( Y ) denotethesolutionstotheseoptimizationproblems,respe ctively(seeTable 7.1 andtherightplotofFigure 7.1 ).Noticethatfollowingthescatterofpointsintherightpl ot ofFigure 7.1 thatasmoothcurvecanapparentlybettedtotheboundaryof thepointsin Y nearestthecoordinateaxes.Inmulticriteriaoptimizatio nthisisreferredtoasthePareto frontier,andweseeinallthreecasesthattheoptimalQoIma pproducesapointin Y thatlies veryneartheParetofrontier.Specically,minimizing M Q;N producesapointatthebottom ofthisfrontier,minimizing S Q;N producesapointattheleftofthisfrontier,andminimizing Y producesapointnearthebottom\corner"ofthisfrontier(t hecornerissomewhatshifted duetothesemi-lognatureoftheplot).Thisdemonstratesat ypeoftrade-othatiscommon inotherareasofmodeldesign,e.g.,thebias/variancetrad e-othatoccursinsemiparametric regression.Asintuitivelyexpected,eachQoImapisdened bycontactthermometerslocated onoppositesidesoftheplate.However,therearenotabledi erencesinspace-timelocations, describedbytheindicesofthecomponentmaps,ofeachoptim alQoImap.Forexample,the QoImapminimizing M Q;N isproducedbymeasurementsmadeneartheendofthesimulati on. Bycomparison,theQoImapminimizing S Q;N isproducedbymeasurementsmadeattherst

PAGE 22

DRAFT 22 T.Butleret.al. timestep. Wenowformulateandsolveastochasticinverseproblemfore achoftheQoImapssolving thedierentoptimizationproblems.First,wesimulateanob served\true"datumforallthree casesbyselectingarandomlychosenpoint,denotedby ref 2 (seethelargerblackcircular dotinthetopplotsofFigure 7.2 ),andcomputing Q ( M ) ( ref ), Q ( S ) ( ref ),and Q ( Y ) ( ref ). Weassumetheuncertaintyinthisobserveddatumisafunctio nofthemeasurementdeviceand nottheactualmap,i.e.,themeasurementisuncertaindueto imperfectionsinthecontact thermometersbutarenotinruencedbythespace-timechoice deningtheQoImap.We choosethisuncertaintytobe0 : 5degrees,andassumethatbyrepeatedexperimentswiththe sameQoImapthatalldatainthe2-dimensionalsquarewithsi delengthsof0 : 5centeredat thetruedatumareequallylikelytooccur.Let E ( M ) E ( S ) ,and E ( Y ) denotethissquarein thedataspacesdenedby Q ( M ) Q ( S ) ,and Q ( Y ) ,respectively. Theassumptionsonthedataspacesimplythattheobservedpr obabilitymeasuretoinvert fromeachofthedataspacesisuniformoneachoftheevents E ( M ) E ( S ) ,and E ( Y ) .Thesupportoftheassociatedinversedensitiessolvingthestocha sticinverseproblemsarethengiven by Q ( M ) ; 1 ( E ( M ) ), Q ( S ) ; 1 ( E ( S ) ),and Q ( Y ) ; 1 ( E ( Y ) )approximatedbytheshadedareasin thetoprowofplotsofFigure 7.2 usingthesame N =5000i.i.d.uniformrandomsamples usedtocompute S Q;N and M Q;N (i.e.,were-usethesamplestocomputethesolutiontothe stochasticinverseproblem).Theapproximatedensitieswi thintheseapproximatedeventsare computedusingtheBETpackageandshowninthebottomrowofp lotsinFigure 7.2 InthetopmiddleplotofFigure 7.2 ,weseethat Q ( S ) ; 1 ( E ( S ) )= .Thus,whilethis inverseimagecanbewellapproximatedbyrelativelyfewsam ples(orinfactnotusingany samplesatall),itisuselessduetothelackofprecisiontha tgivesnonewinformationabout thelocationoftheparametersthatproducedtheobservedda tumotherthantheybelongto theoriginalparameterspace. 7.3.TheImpactonPredictionsandComputationalComplexity. Forsimplicity,we considerapredictionproblemforasimilarQoIusingasimil armodel.Wedenotetheprediction QoIas Q ( p ) whichrepresentstheaveragetemperaturealongtherightbo undaryoftheplateat thenaltimestepwiththeGaussianshapedexternalsourcen owcenteredat( 1 = 2 ; 1 = 2) 2 n,i.e.,centeredatthebottomleftcorneroftheplate. Weapproximatethepredicted(pushforward)densitiesof Q ( p ) associatedwiththedierentinversedensitiesshowninFigure 7.2 usingstandardMonteCarlosamplingandkernel densityestimation.TheresultsaresummarizedinFigure 7.3 .Notethesimilaritiesfrom propagatingtheinversedensitiesassociatedwith Q ( Y ) and Q ( M ) ,andthedierencesofthese propagateddensitiesfromthepropagationoftheinversede nsityassociatedwith Q ( S ) .Specifically,thepropagateddensitiesassociatedwith Q ( Y ) or Q ( M ) producepredictionintervals (denedbythesupportofthepropagateddensities)between 4to5timessmallercompared tothepredictionintervalobtainedbypropagatingtheinve rsedensityassociatedwith Q ( S ) Removingtheeectsofthekerneldensityestimation,thepre dictionintervalsassociatedwith Q ( M ) and Q ( Y ) havelengthsofabout5 : 7and6 : 2,respectively.Thisisconsistentwiththe generalexpectationthatpredictionsassociatedwith Q ( M ) aretypicallythemostprecisedue totheoverallreductioninvarianceduetothesmallersuppo rtoftheinversedensity(ascan bevisuallyobservedinFigure 7.2 ).However,thisslightimprovementinprecisioncompared

PAGE 23

DRAFT ExperimentalDesignwithMeasureTheory 23 n r n r n n r Figure7.2: (top):Randomsampleapproximationof Q ( M ) ; 1 ( E ( M ) )) Q ( S ) ; 1 ( E ( S ) ) ,and Q ( Y ) ( E ( Y ) ) .(bottom):Theapproximateinversedensitieswhosesupport scorrespondtothe topplotsusingthedierentoptimalQoImaps. Figure7.3:Approximatedensitiesof Q ( p ) obtainedbypropagatingtheinversedensitiesshown inFigure 7.2 .Thedierentdensityapproximationsof Q ( p ) arelabeledaccordingtotheoptimal QoImapusedtoconstructtheassociatedinversedensitypro pagatedto Q ( p ) topredictionresultsassociatedwith Q ( Y ) comesatanincreasedcomputationalcomplexity toachievethesameexpectedlevelsofaccuracysince S Q ( M ) ;N > S Q ( Y ) ;N accordingtoTable 7.1 .Infact,itiseasilyseenfromthetoprightplotofFigure 7.2 thatthesupportofthe inversedensityassociatedwithusing Q ( Y ) canbewellapproximatedbyasinglerectanglein ,whichisnottrueforthesupportoftheinversedensityasso ciatedwith Q ( M )

PAGE 24

DRAFT 24 T.Butleret.al. 8.ConclusionsandFutureWork. Wedevelopedageneralcomputationalframework usingecientsingularvaluedecompositionsofsampledJac obianstoquantifythegeometric propertiesofaverageskewnessandscalingeectsofmapsina space Q .Ageneralmulticriteria optimizationproblemwasusedtodenetheOEDproblem,ands everalexamplesillustrated thevariousconceptspresented.Ameasure-theoreticUQmet hodwasusedtoformulateand solveanend-to-endquanticationandreductionofuncerta intiesexamplebasedondierent solutionstotheOEDproblem. Therearemanypossibledirectionsoffutureresearch,andw ecommentonaselectfew thatarethetopicsofcurrentresearch.Onesuchtopicisthe analysisofhowthisOEDapproachimpactsnonmeasure-theoreticUQmethodsusedtofor mulateandsolvestochastic inverseproblems,e.g.,usingBayesianapproaches,andtoc omparethisOEDapproachother approachesbasedondierentconceptualideasofoptimality formulatedforspecicUQmethods.Wearealsoactivelyresearchingasinglecriteriaopti mizationproblemthatexploitsthe skewnessimplicitlyandfocusesentirelyonprecisionbase donquanticationof -scalingof coversofinverseeventsbysimplegeometricobjects. Thereisalsoafundamentalquestionregardingtheuseof d>n QoIincaseswheredata collectionischeap,e.g.,whentherearemanyexistingsens orsintheeldproducingpossibly veryredundantdata.Thisisatypeof\datarich,informatio npoor"problem,andweare studyingapproachesforutilizingalltheavailableQoIdat ainsuchcases.Onemethodthat appearspromisingistopartition anddenelocallyoptimalQoImapsthatreducesolutionof theglobalinverseproblemintoasolutionofonlyasubsetof locallyoptimalinverseproblems. Anothertopicofcurrentinteresttotheauthorsinvolvesad escriptionoftheimprovement orchangeinoptimalityfromusingadditionalQoItoincreas ethedimensionoftheQoImap. Thisrequiresquantifyingboththeinformationgainandinc rease/decreaseincomputational complexityfromusingmore/lessQoI. 9.Acknowledgments. ThismaterialisbaseduponworksupportedbytheU.S.Depart mentofEnergyOceofScience,OceofAdvancedScienticCo mputingResearch,Applied MathematicsprogramunderAwardNumberDE-SC00009279aspa rtoftheDiaMonDMultifacetedMathematicsIntegratedCapabilityCenter.T.Bu tlerandS.Walsh'sworkisalso supportedinpartbytheNationalScienceFoundation(DMS-1 228206). TheauthorswouldalsoliketoacknowledgeDr.LindleyGraha mandDr.StevenMattis fortheirproductiveconversationsandfeedbackonthiswor k,aswellastheirhelpwiththe BET[ 20 ]Pythonpackagewhichleadtothedevelopmentandimplement ationofthesensitivity subpackagebyS.Walshwhichwasusedforquantifyingskewne ssandscalingeectsofQoI mapsinallexamplesshowninthiswork. REFERENCES [1] R.D.CookA.C.Atkinson D-optimumdesignsforheteroscedasticlinearmodels ,Journalofthe AmericanStatisticalAssociation,90(1995),pp.204{212. [2] MartinAlns,JanBlechta,JohanHake,AugustJohansson,Ben jaminKehlet,AndersLogg, ChrisRichardson,JohannesRing,MarieRognes,andGarthWe lls TheFEniCSProject Version1.5 ,ArchiveofNumericalSoftware,3(2015).

PAGE 25

DRAFT ExperimentalDesignwithMeasureTheory 25 [3] A.Atkinson,A.Donev,andR.Tobias OptimumExperimentalDesigns,withSAS ,OxfordUniversity Press,2007. [4] J.Baugh,A.Altuntas,T.Dyer,andJ.Simon SubdomainModelinginADCIRCVersion50 ,2013. [5] F.Bisetti,D.Kim,O.Knio,Q.Long,andR.Tempone Optimalbayesianexperimentaldesignfor priorsofcompactsupportwithapplicationtoshock-tubeex perimentsforcombustionkinetics ,Int.J. NumericalMethodsinEngineering,108(2016),pp.136{155. [6] J.Breidt,T.Butler,andD.Estep AMeasure-TheoreticComputationalMethodforInverseSens itivityProblemsI:MethodandAnalysis ,SIAMJournalonNumericalAnalysis,49(2011),pp.1836{18 59. [7] T.Butler,D.Estep,andJ.Sandelin AComputationalMeasureTheoreticApproachtoInverse SensitivityProblemsII:APosterioriErrorAnalysis ,SIAMJournalonNumericalAnalysis,50(2012), pp.22{45. [8] T.Butler,D.Estep,S.Tavener,C.Dawson,andJ.J.Westerin k AMeasure-TheoreticComputationalMethodForInverseSensitivityProblemsIII:Mult ipleQuantitiesofInterest ,SIAMJournal onUncertaintyQuantication,(2014),pp.1{27. [9] T.Butler,D.Estep,S.Tavener,T.Wildey,C.Dawson,andL.G raham SolvingStochastic InverseProblemsusingSigma-AlgebrasonContourMaps .arXiv:1407.3851,2014. [10] T.Butler,L.Graham,D.Estep,C.Dawson,andJ.J.Westerink Denitionandsolutionofa stochasticinverseproblemforthemanning'snparametere ldinhydrodynamicmodels ,Advancesin WaterResources,78(2015),pp.60{79. [11] S.L.Cotter,M.Dashti,andA.M.Stuart Approximationofbayesianinverseproblems ,SIAM JournalofNumericalAnalysis,48(2010),pp.322{345. [12] M.CrampinandF.A.E.Pirani ApplicableDierentialGeometry ,CambridgeUniversityPress,1987. [13] C.DellacherieandP.A.Meyer ProbabilitiesandPotential ,North-HollandPublishingCo.,Amsterdam,1978. [14] GeirEvensen Sequentialdataassimilationwithanonlinearquasi-geost rophicmodelusingmontecarlo methodstoforecasterrorstatistics ,JournalofGeophysicalResearch:Oceans(1978{2012),99( 1994), pp.10143{10162. [15] GeirEvensen TheensembleKalmanlter:theoreticalformulationandpra cticalimplementation ,Ocean Dynamics,53(2003),pp.343{367. [16] GeraldB.Folland RealAnalysis:ModernTechniquesandTheirApplications ,Wiley,1999. [17] AlisonL.GibbsandFrancisEdwardSu Onchoosingandboundingprobabilitymetrics ,INTERNAT. STATIST.REV.,(2002),pp.419{435. [18] LindleyGraham PolyADCIRCVersion0.3.0 .online,Jan2016. [ https://github.com/UT-CHG/PolyADCIRC ]. [19] L.Graham,T.Butler,S.Walsh,C.Dawson,D.Estep,andJ.J.W esterink AMeasureTheoreticAlgorithmforEstimatingBottomFrictioninaCoa stalInlet:CaseStudyofBaySt.Louis duringHurricaneGustav(2008) ,(2016).Inreview. [20] LindleyGraham,StevenMattis,ScottWalsh,TroyButler,Mi chaelPilosov, andDamonMcDougall BET:Butler,Estep,TavenerMethodv2.0.0 ,Aug.2016. https://doi.org/10.5281/zenodo.59964. [21] X.HuanandT.M.Marzouk Simulation-basedoptimalBayesianexperimentaldesignfo rnonlinear systems ,JournalofComputationalPhysics,232(2013),pp.288{317 [22] RudolphEmilKalmanetal. Anewapproachtolinearlteringandpredictionproblems ,Journalof basicEngineering,82(1960),pp.35{45. [23] J.Kiefer Optimumexperimentaldesigns ,JournaloftheRoyalStatisticalSociety.SeriesB(Method ological),21(1959),pp.272{319. [24] P.C.Leube,A.Geiges,andW.Nowak Bayesianassessmentoftheexpecteddataimpactonpredicti on condenceinoptimalsamplingdesign ,WaterResourcesResearch,48(2012),pp.n/a{n/a. [25] AndersLogg,Kent-AndreMardal,andGarthWells ,eds., AutomatedSolutionofDierential EquationsbytheFiniteElementMethod ,SpringerBerlinHeidelberg,2012. [26] RALuettichandJJWesterink ADCIRCUserManual:A(Parallel)AdvancedCirculationMode lfor Oceanic,CoastalANDEstuarineWaters ,UniversityofNorthCarolinaatChapelHillandUniversity ofNotreDame,version49ed.,April12010. [27] DonaldWMarquardt Analgorithmforleast-squaresestimationofnonlinearpar ameters ,Journalof

PAGE 26

DRAFT 26 T.Butleret.al. theSocietyforIndustrial&AppliedMathematics,11(1963) ,pp.431{441. [28] S.A.Mattis,T.D.Butler,C.N.Dawson,D.Estep,andV.V.Ves selinov Parameterestimation andpredictionforgroundwatercontaminationbasedonmeas uretheory ,WaterResourcesResearch, 51(2015),pp.7608{7629. [29] IMNavon Practicalandtheoreticalaspectsofadjointparameterest imationandidentiabilityinmeteorologyandoceanography ,DynamicsofAtmospheresandOceans,27(1998),pp.55{79. [30] W.Nowak,F.P.J.deBarros,andY.Rubin Bayesiangeostatisticaldesign:Task-drivenoptimal siteinvestigationwhenthegeostatisticalmodelisuncert ain ,WaterResourcesResearch,46(2010), pp.n/a{n/a. [31] VishwasRaoandAdrianSandu Aposteriorierrorestimatesforthesolutionofvariationa linverse problems ,SIAM/ASAJournalonUncertaintyQuantication,3(2015), pp.737{761. [32] A.M.Stuart Inverseproblems:ABayesianperspective ,ActaNumerica,19(2010),pp.451{559. [33] AlbertTarantola InverseProblemTheoryandMethodsforModelParameterEsti mation ,siam,2005.