Citation
First Semester in Numerical Analysis with Python

Material Information

Title:
First Semester in Numerical Analysis with Python
Creator:
Liu, Yaning
Physical Description:
Book

Subjects

Subjects / Keywords:
Open educational resources
OER

Notes

Acquisition:
Collected for Auraria Institutional Repository by the Self-Submittal tool. Submitted by Yaning Liu.
Publication Status:
Unpublished
General Note:
This is an OER book on Numerical Analysis

Record Information

Source Institution:
Auraria Institutional Repository
Holding Location:
Auraria Library
Rights Management:
This item is licensed with the Creative Commons Attribution Non-Commercial Share Alike License. This license lets others remix, tweak, and build upon this work non-commercially, as long as they credit the author and license their new creations under the identical terms.

Downloads

This item is only available as the following downloads:


Full Text

PAGE 1

FirstSemesterinNumericalAnalysiswithPython YaningLiu DepartmentofMathematicalandStatisticalSciences UniversityofColoradoDenver DenverCO80204 Giraykten DepartmentofMathematics FloridaStateUniversity TallahasseeFL32306

PAGE 2

TomywifeYanjunPan-YL

PAGE 3

Contents 1Introduction5 1.1ReviewofCalculus................................5 1.2Pythonbasics...................................8 1.3Computerarithmetic...............................21 2Solutionsofequations:Root-nding42 2.1Erroranalysisforiterativemethods.......................45 2.2Bisectionmethod.................................46 2.3Newton'smethod.................................50 2.4Secantmethod..................................59 2.5Muller'smethod..................................61 2.6Fixed-pointiteration...............................64 2.7High-orderxed-pointiteration.........................72 3Interpolation75 3.1Polynomialinterpolation.............................76 3.2Highdegreepolynomialinterpolation......................92 3.3Hermiteinterpolation...............................96 3.4Piecewisepolynomials:splineinterpolation...................104 4NumericalQuadratureandDierentiation121 4.1Newton-Cotesformulas..............................121 4.2CompositeNewton-Cotesformulas.......................127 4.3Gaussianquadrature...............................131 4.4Multipleintegrals.................................137 4.5Improperintegrals................................144 4.6Numericaldierentiation.............................145 2

PAGE 4

CONTENTS 3 5ApproximationTheory153 5.1Discreteleastsquares..............................153 5.2Continuousleastsquares.............................170 5.3Orthogonalpolynomialsandleastsquares...................173 References187 Index 188

PAGE 5

Preface ThebookisbasedonFirstsemesterinNumericalAnalysiswithJulia,writtenbyGiray kten 1 .Thecontentsoftheoriginalbookareretained,whileallthealgorithmsareimplementedinPythonVersion3.8.0.PythonisanopensourceunderOSI,interpreted, general-purposeprogramminglanguagethathasalargenumberofusersaroundtheworld. PythonisrankedthethirdinAugust2020bytheTIOBEprogrammingcommunityindex 2 ,a measureofpopularityofprogramminglanguages,andisthetop-rankedinterpretedlanguage. WehopethisbookwillbetterservereaderswhoareinterestedinarstcourseinNumerical Analysis,butaremorefamiliarwithPythonfortheimplementationofthealgorithms. Therstchapterofthebookhasaself-containedtutorialforPython,includinghowto setupthecomputerenvironment.Anaconda,theopen-sourceindividualedition,isrecommendedforaneasyinstallationofPythonandeortlessmanagementofPythonpackages, andtheJupyterenvironment,aweb-basedinteractivedevelopmentenvironmentforPython aswellasmanyotherprogramminglanguages,wasusedthroughoutthebookandisrecommendedtothereadersforeasycodedevelopement,graphvisualizationandreproducibility. ThebookwasalsoinspiredbyaseriesofOpenEducationalResourcesworkshopsatUniversityofColoradoDenverandsupportedpartiallybytheprofessionaldevelopmentfunding thereof.YaningLiualsothankshisstudentsinhisNumericalAnalysisclasses,whoenjoyed usingPythontoimplementthealgorithmsandmotivatedhimtowriteaNumericalAnalysis textbookwithcodesinPython. YaningLiuGiraykten August2020August2020 Denver,ColoradoTallahassee,Florida 1 https://open.umn.edu/opentextbooks/textbooks/rst-semester-in-numerical-analysis-with-julia 2 https://www.tiobe.com/tiobe-index/ 4

PAGE 6

Chapter1 Introduction 1.1ReviewofCalculus ThereareseveralconceptsandfactsfromCalculusthatweneedinNumericalAnalysis.In thissectionwewilllistsomedenitionsandtheoremsthatwillbeneededlater.Forthe mostpartfunctionsinthisbookrefertorealvaluedfunctionsdenedonrealnumbers R , oraninterval a;b R : Denition1. 1.Afunction f hasthelimit L at x 0 ,writtenas lim x ! x 0 f x = L; iffor any > 0 ; thereexists > 0 suchthat j f x )]TJ/F48 11.9552 Tf 11.956 0 Td [(L j < whenever 0 < j x )]TJ/F48 11.9552 Tf 11.955 0 Td [(x 0 j <: 2.Afunction f iscontinuousat x 0 if lim x ! x 0 f x = f x 0 ; and f iscontinuousonaset A ifitiscontinuousateach x 0 2 A: 3.Let f x n g 1 n =1 beaninnitesequenceofrealnumbers.Thesequencehasthelimit x; i.e., lim n !1 x n = x or,writtenas x n ! x as n !1 ifforany > 0 ; thereexistsan integer N> 0 suchthat j x n )]TJ/F48 11.9552 Tf 11.955 0 Td [(x j < whenever n>N: Theorem2. Thefollowingareequivalentforarealvaluedfunction f : 1. f iscontinuousat x 0 2.If f x n g 1 n =1 isanysequenceconvergingto x 0 ; then lim n !1 f x n = f x 0 : Denition3. Wesay f x isdierentiableat x 0 if f 0 x 0 =lim x ! x 0 f x )]TJ/F48 11.9552 Tf 11.955 0 Td [(f x 0 x )]TJ/F48 11.9552 Tf 11.955 0 Td [(x 0 =lim h ! 0 f x 0 + h )]TJ/F48 11.9552 Tf 11.955 0 Td [(f x 0 h exists. 5

PAGE 7

CHAPTER1.INTRODUCTION 6 Notation: C n A denotesthesetofallfunctions f suchthat f anditsrst n derivatives arecontinuouson A: If f isonlycontinuouson A ,thenwewrite f 2 C 0 A :C 1 A consistsoffunctionsthathavederivativesofallorders,forexample, f x =sin x or f x = e x : Thefollowingwell-knowntheoremsofCalculuswilloftenbeusedintheremainderofthe book. Theorem4 Meanvaluetheorem . If f 2 C 0 [ a;b ] and f isdierentiableon a;b ,then thereexists c 2 a;b suchthat f 0 c = f b )]TJ/F49 7.9701 Tf 6.586 0 Td [(f a b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a : Theorem5 Extremevaluetheorem . If f 2 C 0 [ a;b ] thenthefunctionattainsaminimum andmaximumvalueover [ a;b ] : If f isdierentiableon a;b ; thentheextremevaluesoccur eitherattheendpoints a;b orwhere f 0 iszero. Theorem6 Intermediatevaluetheorem . If f 2 C 0 [ a;b ] and K isanynumberbetween f a and f b ,thenthereexists c 2 a;b with f c = K: Theorem7 Taylor'stheorem . Suppose f 2 C n [ a;b ] and f n +1 existson a;b ; and x 0 2 a;b : Then,for x 2 a;b f x = P n x + R n x where P n isthe n thorderTaylorpolynomial P n x = f x 0 + f 0 x 0 x )]TJ/F48 11.9552 Tf 11.955 0 Td [(x 0 + f 00 x 0 x )]TJ/F48 11.9552 Tf 11.955 0 Td [(x 0 2 2! + ::: + f n x 0 x )]TJ/F48 11.9552 Tf 11.955 0 Td [(x 0 n n ! and R n istheremainderterm R n x = f n +1 x )]TJ/F48 11.9552 Tf 11.955 0 Td [(x 0 n +1 n +1! forsome between x and x 0 : Example8. Let f x = x cos x )]TJ/F48 11.9552 Tf 11.955 0 Td [(x: 1.Find P 3 x about x 0 = = 2 anduseittoapproximate f : 8 : 2.Computetheexactvaluefor f : 8 ,andtheerror j f : 8 )]TJ/F48 11.9552 Tf 11.955 0 Td [(P 3 : 8 j : 3.Usetheremainderterm R 3 x tondanupperboundfortheerror j f : 8 )]TJ/F48 11.9552 Tf 11.648 0 Td [(P 3 : 8 j : Comparetheupperboundwiththeactualerrorfoundinpart2.

PAGE 8

CHAPTER1.INTRODUCTION 7 Solution. 1.Firstnotethat f = 2= )]TJ/F48 11.9552 Tf 9.298 0 Td [(= 2 : Dierentiating f weget: f 0 x =cos x )]TJ/F48 11.9552 Tf 11.955 0 Td [(x sin x )]TJ/F15 11.9552 Tf 11.955 0 Td [(1 f 0 = 2= )]TJ/F48 11.9552 Tf 9.298 0 Td [(= 2 )]TJ/F15 11.9552 Tf 11.955 0 Td [(1 f 00 x = )]TJ/F15 11.9552 Tf 9.299 0 Td [(2sin x )]TJ/F48 11.9552 Tf 11.955 0 Td [(x cos x f 00 = 2= )]TJ/F15 11.9552 Tf 9.298 0 Td [(2 f 000 x = )]TJ/F15 11.9552 Tf 9.299 0 Td [(3cos x + x sin x f 000 = 2= = 2 : Therefore P 3 x = )]TJ/F48 11.9552 Tf 9.299 0 Td [(= 2 )]TJ/F15 11.9552 Tf 11.955 0 Td [( = 2+1 x )]TJ/F48 11.9552 Tf 11.955 0 Td [(= 2 )]TJ/F15 11.9552 Tf 11.955 0 Td [( x )]TJ/F48 11.9552 Tf 11.955 0 Td [(= 2 2 + 12 x )]TJ/F48 11.9552 Tf 11.955 0 Td [(= 2 3 : Thenweapproximate f : 8 by P 3 : 8= )]TJ/F15 11.9552 Tf 9.299 0 Td [(0 : 3033 using4-digitswithrounding. 2.Theexactvalueis f : 8= )]TJ/F15 11.9552 Tf 9.299 0 Td [(0 : 2426 andtheabsoluteerroris j f : 8 )]TJ/F48 11.9552 Tf 12.723 0 Td [(P 3 : 8 j = 0 : 06062 : 3.Tondanupperboundfortheerror,write j f : 8 )]TJ/F48 11.9552 Tf 11.955 0 Td [(P 3 : 8 j = j R 3 : 8 j where R 3 : 8= f : 8 )]TJ/F48 11.9552 Tf 11.955 0 Td [(= 2 4 4! and isbetween 0 : 8 and = 2 .Weneedtodierentiate f onemoretime: f x = 4sin x + x cos x: Since 0 : 8 <<= 2 ,wecanndanupperboundfor f x ; andthus anupperboundfor R 3 : 8 ; usingtriangleinequality: j R 3 : 8 j = f : 8 )]TJ/F48 11.9552 Tf 11.955 0 Td [(= 2 4 4! = j 4sin + cos j : 01471 0 : 05883 j sin j +0 : 01471 j jj cos j : Notethaton 0 : 8 <<= 2 , sin isapositiveincreasingfunction,and j sin j < sin = 2=1 .Forthesecondterm,wecanndanupperboundbyobserving j j attainsamaximumvalueof = 2 on 0 : 8 <<= 2 ,and cos ,whichisadecreasing positivefunctionon 0 : 8 <<= 2 ,hasamaximumvalueof cos : 8=0 : 6967 . Puttingthesetogether,weget j R 3 : 8 j < 0 : 05883+ : 01471 = 2 : 6967 0 : 07493 : Therefore,ourestimatefortheactualerrorwhichis0.06062frompart2is0.07493.

PAGE 9

CHAPTER1.INTRODUCTION 8 Exercise1.1-1: FindthesecondorderTaylorpolynomialfor f x = e x sin x about x 0 =0 . aCompute P 2 : 4 toapproximate f : 4 .Usetheremainderterm R 2 : 4 tondan upperboundfortheerror j P 2 : 4 )]TJ/F48 11.9552 Tf 10.306 0 Td [(f : 4 j .Comparetheupperboundwiththeactual error. bCompute R 1 0 P 2 x dx toapproximate R 1 0 f x dx .Findanupperboundfortheerror using R 1 0 R 2 x dx ,andcompareittotheactualerror. 1.2Pythonbasics Werecommendusingthefreeandopen-sourcePythonpackagemanagerAnacondaIndividualEdition https://www.anaconda.com/products/individual .Gotothewebpageand choosetheinstalleraccordingtoyouroperatingsystem.Atthetimeofwritingthisbook, PythonVersion3.8isinstalledbydefault,whichisalsotheversionweuseinthisbook. TherearedierentenvironmentsandeditorstorunPython.HerewewillusetheJupyter environment,whichwillbereadyafterAnacondaisinstalled.Thereareseveraltutorials andotherresourcesonPythonat https://www.python.org/ whereonecanndup-to-date informationonPython. TheJupyterenvironmentusestheso-calledJupyternotebookwhereonecanwriteand editaPythoncode,runthecode,andexporttheworkintovariousleformatsincluding Latexandpdf.MostofourinteractionwithPythonwillbethroughtheJupyternotebooks. NotethatAnaconda,onceinstalled,hasalreadypreinstalledalargenumberofcommonly usedPythonpackages,soitisnotnecessarytoinstallnewpackagesforthepurposeof runningthecodesinthisbook. AfterinstallingAnaconda,openaJupyternotebookbyfollowingAnaconda ! Jupyter Notebook ! new ! Python3.Hereisascreenshotofmynotebook:

PAGE 10

CHAPTER1.INTRODUCTION 9 Let'sstartwithsomebasiccomputations. In[1]:2+3 Out[1]:5 Nowweimportthe sin and log function,aswellasthe constantfromthe math package, In[2]:frommathimportsin,pi,log andcompute sin = 4 : In[3]:sinpi/4 Out[3]:0.7071067811865475 OnewaytolearnaboutafunctionistosearchforitonlineinthePythondocumentation https://docs.python.org/3/ .Forexample,thesyntaxforthelogarithmfunctioninthe math packageis log x [ ;b ] where b istheoptionalbase.If b isnotprovided,thenatural logarithmof x tobase e iscomputed. In[4]:log4,2 Out[4]:2.0

PAGE 11

CHAPTER1.INTRODUCTION 10 NumPyarrays NumPyisausefulPythonpackageforarraydatastructure,randomnumbergeneration, linearalgebraalgorithms,andsoon.ANumPyarrayisadatastructurethatcanbeused torepresentvectorsandmatrices,forwhichthecomputationsarealsomadeeasier.Import theNumPypackageanddeneanaliasnpforit. In[5]:importnumpyasnp Hereisthebasicsyntaxtocreatea 1 DNumPyarrayrepresentingavector: In[6]:x=np.array[10,20,30] x Out[6]:array[10,20,30] Thefollowinglineofcodeshowstheentriesofthecreatedarrayareintegersof64bits. In[7]:x.dtype Out[7]:dtype ' int64 ' Ifweinputareal,Pythonwillchangethetypeaccordingly: In[8]:x=np.array[10,20,30,0.1] x Out[8]:array[10.,20.,30.,0.1] In[9]:x.dtype Out[9]:dtype ' float64 ' A1DNumPyarraydoesnotassumeaparticularroworcolumnarrangementofthe data,andhencetakingtransposefora1DNumPyarrayisnotvalid.Hereisanotherway toconstructa1Darray,andsomearrayoperations: In[10]:x=np.array[10*iforiinrange1,6] x Out[10]:array[10,20,30,40,50] In[11]:x[-1]

PAGE 12

CHAPTER1.INTRODUCTION 11 Out[11]:50 In[12]:minx Out[12]:10 In[13]:np.sumx Out[13]:150 In[14]:x=np.appendx,99 x Out[14]:array[10,20,30,40,50,99] In[15]:x[3] Out[15]:40 In[16]:x.size Out[16]:6 TheNumPypackagehasawiderangeofmathematicalfunctionssuchas sin , log ,etc., whichcanbeappliedelementwisetoanarray: In[17]:x=np.array[1,2,3] In[18]:np.sinx Out[18]:array[0.84147098,0.90929743,0.14112001] Plotting ThereareseveralpackagesforplottingfunctionsandwewillusethePyPlotpackage.The packageispreinstalledinAnaconda.Tostartthepackage,use In[19]:importmatplotlib.pyplotasplt %matplotlibinline ThefollowingpieceofcodeisasimpleexampleofplottingwithPyPlot.

PAGE 13

CHAPTER1.INTRODUCTION 12 In[20]:x=np.linspace0,2*np.pi,1000 y=np.sin3*x plt.plotx,y,color= ' red ' ,linewidth=2.0,linestyle= ' -' plt.title ' Thesinefunction ' ; Let'splottwofunctions, sin3 x and cos x ,andlabelthemappropriately. In[21]:x=np.linspace0,2*np.pi,1000 y=np.sin3*x z=np.cosx plt.plotx,y,color= ' red ' ,linewidth=2.0,linestyle= ' -' ,label= ' sinx ' plt.plotx,z,color= ' blue ' ,linewidth=1.0,linestyle= ' ' ,label= ' cosx ' plt.legendloc= ' uppercenter ' ;

PAGE 14

CHAPTER1.INTRODUCTION 13 Matrixoperations NumPyuses 2 Darraystorepresentmatrices.Let'screatea 3 3 matrix 2 Darray: In[22]:A=np.array[[-1,0.26,0.74],[0.09,-1,0.26],[1,1,1]] A Out[22]:array[[-1.,0.26,0.74], [0.09,-1.,0.26], [1.,1.,1.]] Transposeof A iscomputedas: In[23]:A.T Out[23]:array[[-1.,0.09,1.], [0.26,-1.,1.], [0.74,0.26,1.]] Hereisitsinverse. In[24]:np.linalg.invA

PAGE 15

CHAPTER1.INTRODUCTION 14 Out[24]:array[[-0.59693007,0.22740193,0.38260375], [0.08053818,-0.82433201,0.15472807], [0.51639189,0.59693007,0.46266818]] Tocomputetheproductof A andtheinverseof A ,use: In[25]:np.dotA,np.linalg.invA Out[25]:array[[1.00000000e+00,1.24617643e-17,-1.74319952e-17], [-3.98224397e-17,1.00000000e+00,1.74319952e-17], [0.00000000e+00,0.00000000e+00,1.00000000e+00]] Let'strymatrixvectormultiplication.Denesomevector v as: In[26]:v=np.array[0,0,1] v Out[26]:array[0,0,1] Nowtrynp.dot A;v tomultiplythem. In[27]:np.dotA,v Out[27]:array[0.74,0.26,1.] Tosolvethematrixequation Ax = v ,type: In[28]:np.linalg.solveA,v Out[28]:array[0.38260375,0.15472807,0.46266818] Thesolutionto Ax = v canbealsocomputedas x = A )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 v asfollows: In[29]:np.dotnp.linalg.invA,v Out[29]:array[0.38260375,0.15472807,0.46266818] Powersof A canbecomputedas: In[30]:np.linalg.matrix_powerA,5 Out[30]:array[[-2.80229724,0.39583437,2.76651959], [0.12572728,-1.39301724,0.84621677], [3.74250756,3.24338756,4.51654028]]

PAGE 16

CHAPTER1.INTRODUCTION 15 Logicoperations Herearesomebasiclogicoperations: In[31]:2==3 Out[31]:False In[32]:2<=3 Out[32]:True In[33]:2==2or1<0 Out[33]:True In[34]:2==2and1<0 Out[34]:False In[35]:4%2==0 #Checkif4isanevennumber Out[35]:True In[36]:5%2==0 Out[36]:False In[37]:5%2==1 #Checkif5isanoddnumber Out[37]:True Deningfunctions Therearetwowaystodeneafunction.Hereisthebasicsyntax: In[38]:defsquareitx: returnx**2 In[39]:squareit3 Out[39]:9 Thereisalsoacompactformtodeneafunction,ifthebodyofthefunctionisashort,simple expression:

PAGE 17

CHAPTER1.INTRODUCTION 16 In[40]:cubeit=lambdax:x**3 In[41]:cubeit5 Out[41]:125 Supposewewanttopicktheelementsofanarraythataregreaterthan0.Thiscanbedone using: In[42]:x=np.array[-2,3,4,5,-3,0] x[x>0] Out[42]:array[3,4,5] Tocountthenumberofelementsthataregreaterthan 0 inthearrayabove,use In[43]:x[x>0].size Out[43]:3 Types InPython,thereareseveraltypesforintegersandoating-pointnumberssuchasint8,int64,oat32, oat64,andmoreadvancedtypesforBooleanvariablesandstrings.Whenwewriteafunction,we donothavetodeclarethetypeofitsvariables:Pythongureswhatthecorrecttypeiswhen thecodeiscompiled.Thisiscalledadynamictypesystem.Forexample,considerthe squareit functionwedenedbefore: In[44]:defsquareitx: returnx**2 Thetypeof x isnotdeclaredinthefunctiondenition.Wecancallitwithrealorinteger inputs,andPythonwillknowwhattodo: In[45]:squareit5 Out[45]:25 In[46]:squareit5.5 Out[46]:30.25 Nowsupposethetypeoftheinputisaoating-pointnumber.Wecanwriteanotherversionof squareit thatspeciesthetype.

PAGE 18

CHAPTER1.INTRODUCTION 17 In[47]:deftypesquareitx:float: returnx**2 Theinput x isnowstaticallytyped.However,thepurposehereistoaddanannotationto remindtheusersoftheinputtypethatshouldbeused.Infact,Pythoninterpreterwillnotperform anytypecheckingautomatically,unlesssomeadditionalpackagesareused.Inotherwords,there willbenodierencebetweenthe squareit and typesquareit functions. In[48]:typesquareit5.5 Out[48]:30.25 In[49]:typesquareit5 Out[49]:25 Itcanbeseenthatthefunction typesquareit hasnoproblemtakingtheinteger 5 asaninput. Controlow Let'screateaNumPyarrayof10entriesofoating-type.Asimplewaytodoitisbyusingthe function np.zerosn ,whichcreatesanarrayofsize n ,andsetseachentrytozero.Asimilar functionis np.onesn whichcreatesanarrayofsize n witheachentrysetto1. In[50]:values=np.zeros10 values Out[50]:array[0.,0.,0.,0.,0.,0.,0.,0.,0.,0.] Nowwewillsettheelementsofthearraytovaluesofsinfunction. In[51]:forninrange10: values[n]=np.sinn+1**2 In[52]:values Out[52]:array[0.84147098,-0.7568025,0.41211849,-0.28790332,-0.13235175, -0.99177885,-0.95375265,0.92002604,-0.62988799,-0.50636564] Hereisanotherwaytodothis.Startwithcreatinganemptyarray: In[53]:newvalues=np.array[] Thenusea while statementtogeneratethevalues,andappendthemtothearray.

PAGE 19

CHAPTER1.INTRODUCTION 18 In[54]:n=1 whilen<=10: newvalues=np.appendvalues,np.sinn**2 n+=1 newvalues Out[54]:array[0.84147098,-0.7568025,0.41211849,-0.28790332,-0.13235175, -0.99177885,-0.95375265,0.92002604,-0.62988799,-0.50636564, -0.50636564] Hereishowthe if statementworks: In[55]:deffx,y: ifxy: printx, ' isgreaterthan ' ,y else: printx, ' isequalto ' ,y In[56]:f2,3 2islessthan3 In[57]:f3,2 3isgreaterthan2 In[58]:f1,1 1isequalto1 Inthenextexampleweuse if and while tondalltheoddnumbersin f 1 ;:::; 10 g .Theempty arraycreatedintherstlineisofint64type. In[59]:odds=np.array[].astype ' int64 ' n=1 whilen<=10: ifn%2==1: odds=np.appendodds,n n+=1 odds

PAGE 20

CHAPTER1.INTRODUCTION 19 Out[59]:array[1,3,5,7,9] Hereisaninterestingpropertyofthefunction break : In[60]:n=1 whilen<=20: ifn%2==0: printn break n+=1 2 Whydidtheaboveexecutionstopat2?Let'stryremoving break : In[61]:n=1 whilen<=20: ifn%2==0: printn n+=1 2 4 6 8 10 12 14 16 18 20 Thefunction break causesthecodetoexitthewhilelooponceitisevaluated. Randomnumbers Theseare5uniformrandomnumbersfrom,1. In[62]:importnumpyasnp

PAGE 21

CHAPTER1.INTRODUCTION 20 In[63]:np.random.rand5 Out[63]:array[0.43296376,0.66060029,0.91030457,0.27065652,0.58976353] Andthesearerandomnumbersfromthestandardnormaldistribution: In[64]:np.random.randn5 Out[64]:array[-0.60385045,-0.1138149,0.80330871,-0.85704952,1.26651986] Hereisafrequencyhistogramof 10 5 randomnumbersfromthestandardnormaldistribution using50bins: In[65]:y=np.random.randn10**5 plt.histy,50; Sometimesweareinterestedinrelativefrequencyhistogramswheretheheightofeachbinisthe relativefrequencyofthenumbersinthebin.Addingtheoption"density=true"outputsarelative frequencyhistogram: In[66]:y=np.random.randn10**5 plt.histy,50,density=True;

PAGE 22

CHAPTER1.INTRODUCTION 21 Exercise1.2-1: InPythonyoucancomputethefactorialofapositiveinteger n bythebuiltinfunction factorialn fromthesubpackage special inSciPy scipy.special.factorialn .Write yourownversionofthisfunction,called factorial 2 ,usingaforloop.Usethe time.time function tocomparetheexecutiontimeofyourversionandthebuilt-inversionofthefactorialfunction. Exercise1.2-2: WriteaPythoncodetoestimatethevalueof usingthefollowingprocedure: Placeacircleofdiameteroneintheunitsquare.Generate10,000pairsofrandomnumbers u;v fromtheunitsquare.Countthenumberofpairs u;v thatfallintothecircle,andcallthisnumber n .Then n= 10000 isapproximatelytheareaofthecircle.ThisapproachisknownastheMonte Carlomethod. Exercise1.2-3: Considerthefollowingfunction f x;n = n X i =1 i Y j =1 x n )]TJ/F49 7.9701 Tf 6.587 0 Td [(j +1 : aCompute f ; 3 byhand. bWriteaPythoncodethatcomputes f .Verify f ; 3 matchesyouranswerabove. 1.3Computerarithmetic Thewaycomputersstorenumbersandperformcomputationscouldsurprisethebeginner.InPython ifyoutype p 3 2 theresultwillbe2.9....96,where9isrepeated15times.Herearetwoobvious butfundamentaldierencesinthewaycomputersdoarithmetic: onlynitelymanynumberscanberepresentedinacomputer; anumberrepresentedinacomputercanonlyhavenitelymanydigits. Thereforethenumbersthatcanberepresentedinacomputerexactlyisonlyasubsetofrational numbers.Anytimethecomputerperformsanoperationwhoseoutcomeisnotanumberthatcanbe representedexactlyinthecomputer,anapproximationwillreplacetheexactnumber.Thisiscalled the roundoerror :errorproducedwhenacomputerisusedtoperformrealnumbercalculations. Floating-pointrepresentationofrealnumbers Hereisageneralmodelforrepresentingrealnumbersinacomputer: x = s :a 1 a 2 :::a t e .1

PAGE 23

CHAPTER1.INTRODUCTION 22 where s ! signof x = 1 e ! exponent,withbounds L e U :a 1 :::a t = a 1 + a 2 2 + ::: + a t t ; themantissa ! base t ! numberofdigits;theprecision. Intheoating-pointrepresentation1.1,ifwespecify e insuchawaythat a 1 6 =0 ; thenthe representationwillbeunique.Thisiscalledthe normalized oating-pointrepresentation.For exampleif =10 ; inthenormalizedoating-pointwewouldwrite 0 : 012 as 0 : 12 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ,insteadof choiceslike 0 : 012 10 0 or 0 : 0012 10 . Inmostcomputerstoday,thebaseis =2 : Bases8and16wereusedinoldIBMmainframesin thepast.Somehandheldcalculatorsusebase10.Aninterestinghistoricalexampleisashort-lived computernamedSetundevelopedatMoscowStateUniversitywhichusedbase3. Thereareseveralchoicestomakeinthegeneraloating-pointmodel1.1forthevaluesof s;;t;e .TheIEEE64-bitoating-pointrepresentationisthespecicmodelusedinmostcomputers today: x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 s :a 2 a 3 :::a 53 2 2 e )]TJ/F46 7.9701 Tf 6.586 0 Td [(1023 : .2 Somecomments: Noticehow s appearsindierentformsinequations1.1and1.2.In1.2, s iseither0or 1.If s =0 ,then x ispositive.If s =1 ;x isnegative. Since =2 ; inthenormalizedoating-pointrepresentationof x therstnonzerodigitafter thedecimalpointhastobe1.Thenwedonothavetostorethisnumber.That'swhywe write x asadecimalnumberstartingat1in1.2.Eventhoughprecisionis t =52 ; weare abletoaccessuptothe53rddigit a 53 : Theboundsfortheexponentare: 0 e 2047 : Wewilldiscusswhere2047comesfrom shortly.Butrst,let'sdiscusswhywehave e )]TJ/F81 10.9091 Tf 8.548 0 Td [(1023 astheexponentintherepresentation1.2, asopposedtosimply e whichwehadintherepresentation1.1.Ifthesmallestexponent possiblewas e =0 ,thenthesmallestpositivenumberthecomputercangeneratewouldbe : 00 ::: 0 2 =1 :certainlyweneedthecomputertorepresentnumberslessthan1!That'swhy weusetheshiftedexpression e )]TJ/F81 10.9091 Tf 11.306 0 Td [(1023 ,calledthe biasedexponent ,intherepresentation 1.2.Notethattheboundsforthebiasedexponentare )]TJ/F81 10.9091 Tf 8.485 0 Td [(1023 e )]TJ/F81 10.9091 Tf 10.91 0 Td [(1023 1024 : Hereisaschemathatillustrateshowthephysicalbitsofacomputercorrespondtotherepresentation

PAGE 24

CHAPTER1.INTRODUCTION 23 above.Eachcellinthetablebelow,numbered1through64,correspondtothephysicalbitsinthe computermemory. 1 2 3 ::: 12 13 ::: 64 Therstbitisthesignbit:itstoresthevaluefor s ,0or1. Thebluebits2through12storetheexponent e not e )]TJ/F81 10.9091 Tf 9.298 0 Td [(1023 : Using11bits,onecangenerate theintegersfrom0to 2 11 )]TJ/F81 10.9091 Tf 10.404 0 Td [(1=2047 .Hereishowyougetthesmallestandlargestvaluesfor e : e = ::: 0 2 =0 e = ::: 1 2 =2 0 +2 1 + ::: +2 10 = 2 11 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 =2047 : Theredbits,andthereare52ofthem,storethedigits a 2 through a 53 : Example9. Findtheoating-pointrepresentationof10.375. Solution. Youcancheckthat 10=1010 2 and 0 : 375= : 011 2 bycomputing 10=0 2 0 +1 2 1 +0 2 2 +1 2 3 0 : 375=0 2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 +1 2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 +1 2 )]TJ/F46 7.9701 Tf 6.586 0 Td [(3 : Then 10 : 375= : 011 2 = : 010011 2 2 3 where : 010011 2 2 3 isthenormalizedoating-pointrepresentationofthenumber.Nowwe rewritethisintermsoftherepresentation1.2: 10 : 375= )]TJ/F81 10.9091 Tf 8.484 0 Td [(1 0 : 010011 2 2 1026 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1023 : Since 1026= 2 ,thebitbybitrepresentationis: 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 0 1 1 0 ::: 0 Noticetherstsignbitis0sincethenumberispositive.Thenext11bitsinbluerepresentthe exponent e =1026 ; andthenextgroupofredbitsarethemantissa,lledwith0'safterthelast digitofthemantissa.InPython,althoughthereisnobuilt-infunctionthatproducesthebitbybit representationofanumber,wecandenethefollowingfunctionnamed oat2bin ,whichprovides thebitrepresentationofaoating-pointnumber,basedonthe struct package:

PAGE 25

CHAPTER1.INTRODUCTION 24 In[1]:importstruct deffloat2binf: d,=struct.unpack">Q",struct.pack">d",f returnf ' {d:064b} ' In[2]:float2bin10.375 Out[2]: ' 0100000000100100110000000000000000000000000000000000000000000000 ' Specialcases:zero,innity,NAN Intheoating-pointarithmetictherearetwozeros: +0 : 0 and )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 0 ,andtheyhavespecialrepresentations.Intherepresentationofzero,allexponentandmantissabitsaresetto0.Thesignbit is0for +0 : 0 ,and1,for )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 0 : 0 : 0 ! 0 0 allzeros 0 0 allzeros 0 )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 0 ! 1 0 allzeros 0 0 allzeros 0 Whentheexponentbitsaresettozero,wehave e =0 andthus e )]TJ/F81 10.9091 Tf 11.828 0 Td [(1023= )]TJ/F81 10.9091 Tf 8.485 0 Td [(1023 : This arrangement,allexponentbitssettozero,isreservedfor 0 : 0 and subnormalnumbers .Subnormalnumbersareanexceptiontoournormalizedoating-pointrepresentation,anexceptionthatis usefulinvariousways.FordetailsseeGoldberg[9]. Hereishowplusandminusinnityisrepresentedinthecomputer: 1)167(! 0 1 allones 1 0 allzeros 0 )167(! 1 1 allones 1 0 allzeros 0 Whentheexponentbitsaresettoone,wehave e =2047 andthus e )]TJ/F81 10.9091 Tf 11.682 0 Td [(1023=1024 : This arrangementisreservedfor aswellasotherspecialvaluessuchasNaNnot-a-number. Inconclusion,eventhough )]TJ/F81 10.9091 Tf 8.485 0 Td [(1023 e )]TJ/F81 10.9091 Tf 9.17 0 Td [(1023 1024 in1.2,whenitcomestorepresentingnonzerorealnumbers,weonlyhaveaccesstoexponentsinthefollowingrange: )]TJ/F81 10.9091 Tf 8.485 0 Td [(1022 e )]TJ/F81 10.9091 Tf 8.539 0 Td [(1023 1023 : Therefore,thesmallestpositiverealnumberthatcanberepresentedinacomputeris x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 0 : 00 ::: 0 2 2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1022 =2 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1022 0 : 2 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(307 andthelargestis x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 0 : 11 ::: 1 2 2 1023 = 1+ 1 2 + 1 2 2 + ::: + 1 2 52 2 1023 = )]TJ/F81 10.9091 Tf 10.91 0 Td [(2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(52 1023 0 : 18 10 309 :

PAGE 26

CHAPTER1.INTRODUCTION 25 Duringacalculation,ifanumberlessthanthesmallestoating-pointnumberisobtained,thenwe obtainthe underowerror .Anumbergreaterthanthelargestgives overowerror . Exercise1.3-1: Considerthefollowingtoymodelforanormalizedoating-pointrepresentationinbase2: x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 s :a 2 a 3 2 2 e where )]TJ/F81 10.9091 Tf 8.484 0 Td [(1 e 1 .Findallpositivemachinenumbers thereare12ofthemthatcanberepresentedinthismodel.Convertthenumberstobase10, andthencarefullyplotthemonthenumberline,byhand,andcommentonhowthenumbersare spaced. Representationofintegers Intheprevioussection,wediscussedrepresentingrealnumbersinacomputer.Herewewillgive abriefdiscussionofrepresentingintegers.Howdoesacomputerrepresentaninteger n ?Asin realnumbers,westartwithwriting n inbase2.Wehave64bitstorepresentitsdigitsandsign. Asintheoating-pointrepresentation,wecanallocateonebitforthesign,andusetherest,63 bits,forthedigits.Thisapproachhassomedisadvantageswhenwestartaddingintegers.Another approach,knownasthe two'scomplement ,ismorecommonlyused,includinginPython. Foranexample,assumewehave8bitsinourcomputer.Torepresent12intwo'scomplement oranypositiveinteger,wesimplywriteitinitsbase2expansion: 2 .Torepresent )]TJ/F81 10.9091 Tf 8.485 0 Td [(12 , wedothefollowing:ipalldigits,replacing1by0,and0by1,andthenadd1totheresult.When weipdigitsfor12,weget 2 ,andadding1inbinary,gives 2 .Therefore )]TJ/F81 10.9091 Tf 8.485 0 Td [(12 isrepresentedas 2 intwo'scomplementapproach.Itmayseemmysterioustogo throughallthistroubletorepresent )]TJ/F81 10.9091 Tf 8.485 0 Td [(12 ,untilyouaddtherepresentationsof12and )]TJ/F81 10.9091 Tf 8.484 0 Td [(12 , 2 + 2 =100000000 2 andrealizethattherst8digitsofthesumfromrighttoleft,whichiswhatthecomputercan onlyrepresentignoringthereddigit1,is 2 .Sojustlike 12+ )]TJ/F81 10.9091 Tf 8.485 0 Td [(12=0 inbase10,the sumoftherepresentationsofthesenumbersisalso0. Wecanrepeatthesecalculationswith64-bits,usingPython.Thefunction int2bin denedbelow outputsthedigitsofaninteger,usingtwo'scomplementfornegativenumbers: In[1]:importstruct defint2bini: d,=struct.unpack">Q",struct.pack">q",i returnf ' {d:064b} ' In[2]:int2bin12

PAGE 27

CHAPTER1.INTRODUCTION 26 Out[2]: ' 0000000000000000000000000000000000000000000000000000000000001100 ' In[3]:int2bin-12 Out[3]: ' 1111111111111111111111111111111111111111111111111111111111110100 ' Youcanverifythatthesumoftheserepresentationsis0,whentruncatedto64-digits. Hereisanotherexampleillustratingtheadvantagesoftwo'scomplement.Consider )]TJ/F81 10.9091 Tf 8.485 0 Td [(3 and5, withrepresentations, )]TJ/F81 10.9091 Tf 8.485 0 Td [(3= 2 and 5= 2 : Thesumof )]TJ/F81 10.9091 Tf 8.485 0 Td [(3 and5is2;whataboutthebinarysumoftheirrepresentations?Wehave 2 + 2 =100000010 2 andifweignoretheninthbitinred,theresultis 2 ,whichisindeed2.Noticethatifwefollowed thesameapproachusedintheoating-pointrepresentationandallocatedtheleftmostbittothe signoftheinteger,wewouldnothavehadthisproperty. Wewillnotdiscussintegerrepresentationsandintegerarithmeticanyfurther.Howeverone usefulfacttokeepinmindisthefollowing:intwo'scomplement,using64bits,onecanrepresent integersbetween )]TJ/F81 10.9091 Tf 8.485 0 Td [(2 63 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(9223372036854775808 and 2 63 )]TJ/F81 10.9091 Tf 10.27 0 Td [(1=9223372036854775807 .Anyinteger beloworaboveyields underow or overow error.However,Pythonisspecialcomparedto theotherprogramminglanguagesinthatitsupportsarbitrary-precisionintegerimplementation. IntegersofanysizecanbesupportedinPython,whileitisnottrueforoats. Example10. FromCalculus,weknowthat lim n )]TJ/F49 7.9701 Tf 6.587 0 Td [(> 1 n n n ! = 1 .Therefore,computing n n n ! ,whichis aoat,forlarge n willcauseoverowatsomepoint.HereisaPythoncodeforthiscalculation: In[1]:fromscipy.specialimportfactorial f=lambdan:n**n/factorialn Let'shaveacloserlookatthisfunction.ThePythonfunction factorialn computesthe factorial,returnedasaoat,asthegammafunctionat n +1 .Ifwecall f withanintegerinput, thentheabovecodewillcompute n n asanintegerandfactorial n asaoat.Thenitwilldivide thesenumbers,byrstconverting n n toaoat,andobtainaoating-pointfactorial.Let'scompute f n = n n n ! as n =1 ;:::; 1000 : In[2]:forninrange1,1001: printn,fn . .

PAGE 28

CHAPTER1.INTRODUCTION 27 . 1351.4629522660965042e+57 1363.962087287769907e+57 1371.0730739055834461e+58 1382.90634235201109e+58 1397.871822311096416e+58 1402.132136506640898e+59 1415.775183434192781e+59 1421.5643266954396485e+60 1434.2374040203696554e+60 --------------------------------------------------------------------------OverflowErrorTracebackmostrecentcalllast in 1forninrange,1001: ---->2printn,fn inn 1fromscipy.specialimportfactorial ---->2f=lambdan:n**n/factorialn OverflowError:inttoolargetoconverttofloat Noticethattheprocesscannotproceedbeyond n =143 .Whereexactlydoestheerroroccur? Pythoncancompute 144 144 exactlyarbitrary-precision;however,whenitisconvertedtoaoatingpointnumber,overowoccurs.For 143 143 ,overowdoesnothappen. In[3]:float143**143 Out[3]:1.6332525972973913e+308 In[4]:float144**144

PAGE 29

CHAPTER1.INTRODUCTION 28 --------------------------------------------------------------------------OverflowErrorTracebackmostrecentcalllast in ---->1float**144 OverflowError:inttoolargetoconverttofloat Thefunction f n canbecodedinamuchbetterwayifwerewrite n n n ! as n n n n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 ::: n 1 .Each fractioncanbecomputedseparately,andthenmultiplied,whichwillslowdownthegrowthofthe numbers.Hereisanewcodeusinga for statement. In[5]:deffn: pr=1. foriinrange1,n: pr*=n/n-i returnpr Let'scompute f n = n n n ! as n =1 ;:::; 1000 again: In[6]:foriinrange1,1001: printi,fi . . . 7052.261381911989747e+304 7066.142719392952918e+304 7071.6685832310803885e+305 7084.532475935558631e+305 7091.2311857272492938e+306 7103.34435267514059e+306 7119.084499321839277e+306 7122.4676885942863887e+307 7136.70316853566547e+307 714inf

PAGE 30

CHAPTER1.INTRODUCTION 29 715inf 716inf 717inf 718inf . . . Thepreviousversionofthecodegaveoverowerrorwhen n =144 .Thisversionhasnodiculty incomputing n n =n ! for n =144 .Infact,wecangoashighas n =713 .Overowinoating-point arithmeticyieldstheoutputinf,whichstandsforinnity. Anotherwaytoaccommodatealargervalueof n istodenethefunction f inthisway: In[7]:fromscipy.specialimportfactorial f=lambdan:n**n/factorialn,True WiththeadditioninputTrue,thefunction factorialn returnsanintegerwitharbitrary precision,insteadofaoat,sothatboththenumeratoranddenominatorareintegers.Thelimit processstopsonlywhenthedivisionproducesaoating-pointnumberthatissolargethatoverow occurs.Asaresult,thesameresultasintheimprovedalgorithmaboveisexpected.Pleaseverify thattheprocesscancontinueuntil n =713 withthemodied f function. Wewilldiscussseveralfeaturesofcomputerarithmeticintherestofthissection.Thediscussion iseasiertofollowifweusethefamiliarbase10representationofnumbersinsteadofbase2.Tothis end,weintroducethe normalizeddecimaloating-pointrepresentation : 0 :d 1 d 2 :::d k 10 n where 1 d 1 9 ; 0 d i 9 forall i =2 ; 3 ;:::;k: Informally,wecallthesenumbers k )]TJ/F79 10.9091 Tf 8.485 0 Td [(digitdecimal machinenumbers. Chopping&Rounding Let x bearealnumberwithmoredigitsthecomputercanhandle: x =0 :d 1 d 2 :::d k d k +1 ::: 10 n : Howwillthecomputerrepresent x ? Let'susethenotation fl x fortheoating-pointrepresentation of x: Therearetwochoices,choppingandrounding: Inchopping,wesimplytaketherst k digitsandignoretherest: fl x =0 :d 1 d 2 :::d k : Inrounding,if d k +1 5 weadd1to d k toobtain fl x : If d k +1 < 5 ; thenwesimplydoasin chopping.

PAGE 31

CHAPTER1.INTRODUCTION 30 Example11. Find5-digit k =5 choppingandroundingvaluesofthenumbersbelow: =0 : 314159265 ::: 10 1 Choppinggives fl =0 : 31415 androundinggives fl =0 : 31416 : 0 : 0001234567 Weneedtowritethenumberinthenormalizedrepresentationrstas 0 : 1234567 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(3 : Now choppinggives 0 : 12345 androundinggives 0 : 12346 : Absoluteandrelativeerror Sincecomputersonlygiveapproximationstorealnumbers,weneedtobeclearonhowwemeasure theerrorofanapproximation. Denition12. Suppose x isanapproximationto x . j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j iscalledthe absoluteerror j x )]TJ/F49 7.9701 Tf 6.586 0 Td [(x j j x j iscalledthe relativeerror x 6 =0 Relativeerrorusuallyisabetterchoiceofmeasure,andweneedtounderstandwhy. Example13. Findabsoluteandrelativeerrorsof 1. x =0 : 20 10 1 ;x =0 : 21 10 1 2. x =0 : 20 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 ;x =0 : 21 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 3. x =0 : 20 10 5 ;x =0 : 21 10 5 Noticehowtheonlydierenceinthethreecasesistheexponentofthenumbers.Theabsolute errorsare: 0 : 01 10 , 0 : 01 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 , 0 : 01 10 5 .Theabsoluteerrorsaredierentsincetheexponents aredierent.However,therelativeerrorineachcaseisthesame: 0 : 05 : Denition14. Thenumber x issaidtoapproximate x to s signicantdigitsorguresif s is thelargestnonnegativeintegersuchthat j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j j x j 5 10 )]TJ/F49 7.9701 Tf 6.587 0 Td [(s : InExample13wehad j x )]TJ/F49 7.9701 Tf 6.586 0 Td [(x j j x j =0 : 05 5 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 butnotlessthanorequalto 5 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(3 : Therefore wesay x =0 : 21 approximates x =0 : 20 to2signicantdigitsbutnotto3digits. Whenthecomputerapproximatesarealnumber x by fl x ; whatcanwesayabouttheerror? Thefollowingresultgivesanupperboundfortherelativeerror.

PAGE 32

CHAPTER1.INTRODUCTION 31 Lemma15. Therelativeerrorofapproximating x by fl x inthe k -digitnormalizeddecimal oating-pointrepresentationsatises j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl x j j x j 8 < : 10 )]TJ/F49 7.9701 Tf 6.587 0 Td [(k +1 ifchopping 1 2 )]TJ/F49 7.9701 Tf 6.586 0 Td [(k +1 ifrounding. Proof. Wewillgivetheproofforchopping;theproofforroundingissimilarbuttedious.Let x =0 :d 1 d 2 :::d k d k +1 ::: 10 n : Then fl x =0 :d 1 d 2 :::d k 10 n ifchoppingisused.Observethat j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl x j j x j = 0 :d k +1 d k +2 ::: 10 n )]TJ/F49 7.9701 Tf 6.586 0 Td [(k 0 :d 1 d 2 ::: 10 n = 0 :d k +1 d k +2 ::: 0 :d 1 d 2 ::: 10 )]TJ/F49 7.9701 Tf 6.587 0 Td [(k : Wehavetwosimplebounds: 0 :d k +1 d k +2 :::< 1 and 0 :d 1 d 2 ::: 0 : 1 ,thelattertruesincethesmallest d 1 canbe,is1.Usingtheseboundsintheequationaboveweget j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl x j j x j 1 0 : 1 10 )]TJ/F49 7.9701 Tf 6.586 0 Td [(k =10 )]TJ/F49 7.9701 Tf 6.587 0 Td [(k +1 : Remark16. Lemma15easilytransferstothebase2oating-pointrepresentation: x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 s :a 2 :::a 53 2 2 e )]TJ/F46 7.9701 Tf 6.587 0 Td [(1023 ,by j x )]TJ/F82 10.9091 Tf 10.91 0 Td [(fl x j j x j 8 < : 2 )]TJ/F49 7.9701 Tf 6.586 0 Td [(t +1 =2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(53+1 =2 )]TJ/F46 7.9701 Tf 6.586 0 Td [(52 ifchopping 1 2 )]TJ/F49 7.9701 Tf 6.586 0 Td [(t +1 =2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(53 ifrounding. Machineepsilon Machineepsilon isthesmallestpositiveoatingpointnumberforwhich fl + > 1 : Thismeans, ifweaddto 1 : 0 anynumberlessthan ,themachinecomputesthesumas 1 : 0 . Thenumber1.0initsbinaryoating-pointrepresentationissimply : 0 ::: 0 2 where a 2 = a 3 = ::: = a 53 =0 : Wewanttondthesmallestnumberthatgivesasumlargerthan1.0,whenitis addedto1.0.Theanswerdependsonwhetherwechoporround. Ifwearechopping,examinethebinaryaddition

PAGE 33

CHAPTER1.INTRODUCTION 32 a 2 a 52 a 53 1.0...00 +0.0...01 1.0...01 andnotice : 0 ::: 01 2 = )]TJ/F46 7.9701 Tf 6.196 -4.541 Td [(1 2 52 =2 )]TJ/F46 7.9701 Tf 6.586 0 Td [(52 isthesmallestnumberwecanaddto1.0suchthatthesum willbedierentthan1.0. Ifwearerounding,examinethebinaryaddition a 2 a 52 a 53 1.0...00 +0.0...001 1.0...001 wherethesumhastoberoundedto53digitstoobtain a 2 a 52 a 53 1.0...01 . Observethatthenumberaddedto 1 : 0 aboveis : 0 ::: 01 2 = )]TJ/F46 7.9701 Tf 6.195 -4.541 Td [(1 2 53 =2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(53 ,whichisthesmallest numberthatwillmakethesumlargerthan1.0withrounding. Insummary,wehaveshown = 8 < : 2 )]TJ/F46 7.9701 Tf 6.586 0 Td [(52 ifchopping 2 )]TJ/F46 7.9701 Tf 6.586 0 Td [(53 ifrounding : Asaconsequence,noticethatwecanrestatetheinequalityinRemark16inacompactwayusing themachineepsilonas: j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl x j j x j : Remark17. Thereisanotherdenitionofmachineepsilon:itisthedistancebetween1.0andthe nextoating-pointnumber. a 2 a 52 a 53 number1.01.0...00 nextnumber1.0...01 distance0.0...01 Notethatthedistanceabsolutevalueofthedierenceis )]TJ/F46 7.9701 Tf 6.196 -4.541 Td [(1 2 52 =2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(52 : Inthisalternative denition,machineepsilonisnotbasedonwhetherroundingorchoppingisused.Also,notethat thedistancebetweentwoadjacentoating-pointnumbersisnotconstant,butitissmallerfor smallervalues,andlargerforlargervaluesseeExercise1.3-1.

PAGE 34

CHAPTER1.INTRODUCTION 33 Propagationoferror Wediscussedtheresultingerrorwhenchoppingorroundingisusedtoapproximatearealnumberby itsmachineversion.Nowimaginecarryingoutalongcalculationwithmanyarithmeticaloperations, andateachstepthereissomeerrorduetosay,rounding.Wouldalltheroundingerrorsaccumulate andcausehavoc?Thisisaratherdicultquestiontoansweringeneral.Foramuchsimpler example,consideraddingtworealnumbers x;y: Inthecomputer,thenumbersarerepresentedas fl x ;fl y : Thesumofthesenumberis fl x + fl y ;however,thecomputercanonlyrepresent itsoating-pointversion, fl fl x + fl y : Thereforetherelativeerrorinaddingtwonumbersis: x + y )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl fl x + fl y x + y : Inthissection,wewilllookatsomespecicexampleswhereroundoerrorcancauseproblems,and howwecanavoidthem. Subtractionofnearlyequalquantities:Cancellationofleadingdigits Thebestwaytoexplainthisphenomenonisbyanexample.Let x =1 : 123456 ;y =1 : 123447 : We willcompute x )]TJ/F82 10.9091 Tf 11.256 0 Td [(y andtheresultingroundoerrorusingroundingand6-digitarithmetic.First, wend fl x ;fl y : fl x =1 : 12346 ;fl y =1 : 12345 : Theabsoluteandrelativeerrorduetoroundingis: j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl x j =4 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 ; j y )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl y j =3 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl x j j x j =3 : 56 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 ; j y )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl y j j y j =2 : 67 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 : Fromtherelativeerrors,weseethat fl x and fl y approximate x and y tosixsignicantdigits. Let'sseehowtheerrorpropagateswhenwesubtract x and y: Theactualdierenceis: x )]TJ/F82 10.9091 Tf 10.909 0 Td [(y =1 : 123456 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 123447=0 : 000009=9 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 : Thecomputerndsthisdierencerstbycomputing fl x ;fl y ; thentakingtheirdierenceand approximatingthedierencebyitsoating-pointrepresentation: fl fl x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl y : fl fl x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl y = fl : 12346 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 12345=10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(5 :

PAGE 35

CHAPTER1.INTRODUCTION 34 Theresultingabsoluteandrelativeerrorsare: j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(y )]TJ/F81 10.9091 Tf 10.909 0 Td [( fl fl x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl y j =10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(y )]TJ/F81 10.9091 Tf 10.909 0 Td [( fl fl x )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl y j j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(y j =0 : 1 : Noticehowlargetherelativeerroriscomparedtotheabsoluteerror!Themachineversionof x )]TJ/F82 10.9091 Tf 10.486 0 Td [(y approximates x )]TJ/F82 10.9091 Tf 11.467 0 Td [(y toonlyonesignicantdigit.Whydidthishappen?Whenwesubtracttwo numbersthatarenearlyequal,theleadingdigitsofthenumberscancel,leavingaresultcloseto theroundingerror.Inotherwords,theroundingerrordominatesthedierence. Divisionbyasmallnumber Let x =0 : 444446 andcompute x 10 )]TJ/F47 5.9776 Tf 5.756 0 Td [(5 inacomputerwith5-digitarithmeticandrounding.Wehave fl x =0 : 44445 ,withanabsoluteerrorof 4 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 andrelativeerrorof 9 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 : Theexactdivision is x 10 )]TJ/F47 5.9776 Tf 5.756 0 Td [(5 =0 : 444446 10 5 .Thecomputercomputes: fl )]TJ/F49 7.9701 Tf 13 -4.541 Td [(x 10 )]TJ/F47 5.9776 Tf 5.756 0 Td [(5 =0 : 44445 10 5 ; whichhasanabsolute errorof 0 : 4 andrelativeerrorof 9 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 : Theabsoluteerrorwentfrom 4 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 to 0 : 4 .Perhaps notsurprisingly,divisionbyasmallnumbermagniestheabsoluteerrorbutnottherelativeerror. Considerthecomputationof 1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(cos x sin x when x isnearzero.Thisisaproblemwherewehaveboth subtractionofnearlyequalquantities whichhappensinthenumerator,and divisionbyasmallnumber ,when x isclosetozero.Let x =0 : 1 .Continuingwithve-digitrounding,wehave fl sin0 : 1=0 : 099833 ;fl cos0 : 1=0 : 99500 fl 1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(cos0 : 1 sin0 : 1 =0 : 050084 : Theexactresultto8digitsis0.050041708,andtherelativeerrorofthiscomputationis 8 : 5 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(4 . Nextwewillseehowtoreducethiserrorusingasimplealgebraicidentity. Waystoavoidlossofaccuracy Herewewilldiscusssomeexampleswhereacarefulrewritingoftheexpressiontocomputecanmake theroundoerrormuchsmaller. Example18. Let'srevisitthecalculationof 1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(cos x sin x :

PAGE 36

CHAPTER1.INTRODUCTION 35 Observethatusingthealgebraicidentity 1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(cos x sin x = sin x 1+cos x removesbothdicultiesencounteredbefore:thereisnocancellationofsignicantdigitsordivision byasmallnumber.Usingve-digitrounding,wehave fl sin0 : 1 1+cos0 : 1 =0 : 050042 : Therelativeerroris 5 : 8 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 ,aboutafactorof100smallerthantheerrorintheoriginalcomputation. Example19. Considerthequadraticformula:thesolutionof ax 2 + bx + c =0 is r 1 = )]TJ/F82 10.9091 Tf 8.485 0 Td [(b + p b 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(4 ac 2 a ;r 2 = )]TJ/F82 10.9091 Tf 8.485 0 Td [(b )]TJ 10.909 9.199 Td [(p b 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(4 ac 2 a : If b p b 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(4 ac ,thenwehaveapotentiallossofprecisionincomputingoneoftherootsdueto cancellation.Let'sconsideraspecicequation: x 2 )]TJ/F81 10.9091 Tf 11.299 0 Td [(11 x +1=0 .Therootsfromthequadratic formulaare: r 1 = 11+ p 117 2 10 : 90832691 ,and r 2 = 11 )]TJ 6.587 6.598 Td [(p 117 2 0 : 09167308680 : Nextwillusefour-digitarithmeticwithroundingtocomputetheroots: fl p 117=10 : 82 fl r 1 = fl fl fl : 0+ fl p 117 fl : 0 ! = fl fl : 0+10 : 82 2 : 0 = fl 21 : 82 2 : 0 =10 : 91 fl r 2 = fl fl fl : 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(fl p 117 fl : 0 ! = fl fl : 0 )]TJ/F81 10.9091 Tf 10.909 0 Td [(10 : 82 2 : 0 = fl 0 : 18 2 : 0 =0 : 09 : Therelativeerrorsare: relerrorin r 1 = 10 : 90832691 )]TJ/F81 10.9091 Tf 10.909 0 Td [(10 : 91 10 : 90832691 =1 : 5 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(4 relerrorin r 2 = 0 : 09167308680 )]TJ/F81 10.9091 Tf 10.909 0 Td [(0 : 09 0 : 09167308680 =1 : 8 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 : Noticethelargerrelativeerrorin r 2 comparedtothatof r 1 ; aboutafactorof 100 ,whichisdueto cancellationofleadingdigitswhenwecompute 11 : 0 )]TJ/F81 10.9091 Tf 10.909 0 Td [(10 : 82 . Onewaytoxthisproblemistorewritetheoendingexpressionbyrationalizingthenumerator: r 2 = 11 : 0 )]TJ 10.909 9.024 Td [(p 117 2 = 1 2 11 : 0 )]TJ 10.909 9.024 Td [(p 117 11 : 0+ p 117 11 : 0+ p 117 = 1 2 4 11 : 0+ p 117 = 2 11 : 0+ p 117 :

PAGE 37

CHAPTER1.INTRODUCTION 36 Ifweusethisformulatocompute r 2 weget: fl r 2 = fl 2 : 0 fl : 0+ fl p 117 = fl 2 : 0 21 : 82 =0 : 09166 : Thenewrelativeerrorin r 2 is: relerrorin r 2 = 0 : 09167308680 )]TJ/F81 10.9091 Tf 10.909 0 Td [(0 : 09166 0 : 09167308680 =1 : 4 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(4 ; animprovementaboutafactorof 100 ; eventhoughinthenewwayofcomputing r 2 therearetwo operationswhereroundingerrorhappensinsteadofone. Example20. Thesimpleprocedureofaddingnumbers,eveniftheydonothavemixedsigns,can accumulatelargeerrorsduetoroundingorchopping.Severalsophisticatedalgorithmstoaddlarge listsofnumberswithaccumulatederrorsmallerthanstraightforwardadditionexistintheliterature see,forexample,Higham[11]. Forasimpleexample,considerusingfour-digitarithmeticwithrounding,andcomputingthe averageoftwonumbers, a + b 2 .For a =2 : 954 and b =100 : 9 ,thetrueaverageis 51 : 927 .However, four-digitarithmeticwithroundingyields: fl 100 : 9+2 : 954 2 = fl fl : 854 2 = fl 103 : 9 2 =51 : 95 ; whichhasarelativeerrorof 4 : 43 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(4 : Ifwerewritetheaveragingformulaas a + b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a 2 ,ontheother hand,weobtain51.93,whichhasamuchsmallerrelativeerrorof 5 : 78 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(5 .Thefollowingtable displaystheexactand4-digitcomputations,withthecorrespondingrelativeerrorateachstep. aba + b a + b 2 b )]TJ/F48 11.9552 Tf 11.955 0 Td [(a b )]TJ/F49 7.9701 Tf 6.587 0 Td [(a 2 a + b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a 2 4-digitrounding2.954100.9103.951.9597.9548.9851.93 Exact103.85451.92797.94648.97351.927 Relativeerror4.43e-44.43e-44.08e-51.43e-45.78e-5 Example21. Therearetwostandardformulasgivenintextbookstocomputethesamplevariance s 2 ofthenumbers x 1 ;:::;x n : 1. s 2 = 1 n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 h P n i =1 x 2 i )]TJ/F46 7.9701 Tf 12.556 4.295 Td [(1 n P n i =1 x i 2 i , 2.Firstcompute x = 1 n P n i =1 x i ,andthen s 2 = 1 n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 P n i =1 x i )]TJ/F81 10.9091 Tf 11.602 0 Td [( x 2 . Bothformulascansuerfromroundoerrorsduetoaddinglargelistsofnumbersif n islarge, asmentionedinthepreviousexample.However,therstformulaisalsopronetoerrordueto cancellationofleadingdigitsseeChanetal[6]fordetails. Foranexample,considerfour-digitroundingarithmetic,andletthedatabe 1 : 253 ; 2 : 411 ; 3 : 174 . Thesamplevariancefromformula1andformula2are,0.93and0.9355,respectively.Theexact

PAGE 38

CHAPTER1.INTRODUCTION 37 value,upto6digits,is0.935562.Formula2isanumericallymorestablechoiceforcomputingthe variancethantherstone. Example22. Wehaveasumtocompute: e )]TJ/F46 7.9701 Tf 6.586 0 Td [(7 =1+ )]TJ/F81 10.9091 Tf 8.485 0 Td [(7 1 + )]TJ/F81 10.9091 Tf 8.484 0 Td [(7 2 2! + )]TJ/F81 10.9091 Tf 8.485 0 Td [(7 3 3! + ::: + )]TJ/F81 10.9091 Tf 8.484 0 Td [(7 n n ! : Thealternatingsignsmakethisapotentiallyerrorpronecalculation. Pythonreportsthe"exact"valuefor e )]TJ/F46 7.9701 Tf 6.586 0 Td [(7 as0.0009118819655545162.IfweusePythonto computethissumwith n =20 ,theresultis0.009183673977218275.HereisthePythoncodefor thecalculation: In[1]:importnumpyasnp fromscipy.specialimportfactorial In[2]:sum=1.0 forninrange1,21: sum+=-7**n/factorialn sum Out[2]:0.009183673977218275 Thisresulthasarelativeerrorof9.1.Wecanavoidthishugeerrorifwesimplyrewritethe abovesumas e )]TJ/F46 7.9701 Tf 6.587 0 Td [(7 = 1 e 7 = 1 1+7+ 7 2 2! + 7 3 3! + ::: : ThePythoncodeforthiscomputationusing n =20 isbelow: In[3]:sum=1.0 forninrange1,21: sum+=7**n/factorialn sum=1/sum sum Out[3]:0.0009118951837867185 Theresultis0.0009118951837867185,whichhasarelativeerrorof 1 : 4 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(5 .

PAGE 39

CHAPTER1.INTRODUCTION 38 Exercise1.3-2: The x -interceptofthelinepassingthroughthepoints x 1 ;y 1 and x 2 ;y 2 canbecomputedusingeitheroneofthefollowingformulas: x = x 1 y 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 y 1 y 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(y 1 or, x = x 1 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [( x 2 )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 1 y 1 y 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(y 1 withtheassumption y 1 6 = y 2 . aShowthattheformulasareequivalenttoeachother. bComputethe x -interceptusingeachformulawhen x 1 ;y 1 = : 02 ; 3 : 32 and x 2 ;y 2 = : 31 ; 4 : 31 .Usethree-digitroundingarithmetic. cUsePythonoracalculatortocomputethe x -interceptusingthefull-precisionofthedevice youcanuseeitheroneoftheformulas.Usingthisresult,computetherelativeandabsolute errorsoftheanswersyougaveinpartb.Discusswhichformulaisbetterandwhy. Exercise1.3-3: WritetwofunctionsinPythontocomputethebinomialcoecient )]TJ/F49 7.9701 Tf 5 -3.995 Td [(m k using thefollowingformulas: a )]TJ/F49 7.9701 Tf 5 -3.996 Td [(m k = m ! k ! m )]TJ/F49 7.9701 Tf 6.587 0 Td [(k ! m ! is scipy.special.factorialm inPython. b )]TJ/F49 7.9701 Tf 5 -3.996 Td [(m k = m k m )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 k )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ::: m )]TJ/F49 7.9701 Tf 6.587 0 Td [(k +1 1 Then,experimentwithvariousvaluesfor m;k toseewhichformulacausesoverowrst. Exercise1.3-4: PolynomialscanbeevaluatedinanestedformalsocalledHorner'smethod thathastwoadvantages:thenestedformhassignicantlylesscomputation,anditcanreduce roundoerror.For p x = a 0 + a 1 x + a 2 x 2 + ::: + a n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 + a n x n itsnestedformis p x = a 0 + x a 1 + x a 2 + ::: + x a n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 + x a n ::: : Considerthepolynomial p x = x 2 +1 : 1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 : 8 . aCompute p : 5 usingthree-digitrounding,andthree-digitchoppingarithmetic.Whatare theabsoluteerrors?Notethattheexactvalueof p : 5 is13.3. bWrite x 2 +1 : 1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 : 8 innestedformbythesesimplesteps: x 2 +1 : 1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 : 8= x 2 +1 : 1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 : 8= x +1 : 1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 : 8 :

PAGE 40

CHAPTER1.INTRODUCTION 39 Thencompute p : 5 usingthree-digitroundingandchoppingusingthenestedform.What aretheabsoluteerrors?Comparetheerrorswiththeonesyoufoundina. Exercise1.3-5: Considerthepolynomialwritteninstandardform: 5 x 4 +3 x 3 +4 x 2 +7 x )]TJ/F81 10.9091 Tf 9.713 0 Td [(5 . aWritethepolynomialinitsnestedform.Seethepreviousproblem. bHowmanymultiplicationsdoesthenestedformrequirewhenweevaluatethepolynomialata realnumber?Howmanymultiplicationsdoesthestandardformrequire?Canyougeneralize youranswertoany n thdegreepolynomial?

PAGE 41

CHAPTER1.INTRODUCTION 40 Aryaandtheunexpectedchallengesofdataanalysis MeetArya!Aryaisacollegestudentinterestedinmath,biology,literature,andacting.Likea typicalcollegestudent,shetextswhilewalkingoncampus,complainsaboutdemandingprofessors, andarguesinherblogsthathomeworkshouldbeoutlawed. Aryaistakingachemistryclass,andsheperformssomeexperimentsinthelabtondthe weightoftwosubstances.Duetodicultyin makingprecisemeasurements,shecanonlyassesstheweightstofour-signicantdigitsofaccuracy:2.312gramsand0.003982grams.Arya's professorwantstoknowtheproductofthese weights,whichwillbeusedinaformula. Aryacomputestheproductusinghercalculator: 2 : 312 0 : 003982=0 : 009206384 ,andstaresat theresultinbewilderment.Thenumbersshemultipliedhadfour-signicantdigits,buttheproduct hassevendigits!Couldthisbetheresultofsomemagic,likearabbithoppingoutofamagician's hatthatwasonlyahandkerchiefamomentago?Aftersomeinternaldeliberations,Aryadecides toreporttheanswertoherprofessoras 0 : 009206 .DoyouthinkAryawascorrectinnotreporting allofthedigitsoftheproduct?

PAGE 42

CHAPTER1.INTRODUCTION 41 Sourcesoferrorinappliedmathematics Hereisalistofpotentialsourcesoferrorwhenwesolveaproblem. 1.Errorduetothesimplifyingassumptionsmadeinthedevelopmentofamathematicalmodel forthephysicalproblem. 2.Programmingerrors. 3.Uncertaintyinphysicaldata:errorincollectingandmeasuringdata. 4.Machineerrors:rounding/chopping,underow,overow,etc. 5.Mathematicaltruncationerror:errorthatresultsfromtheuseofnumericalmethodsinsolving aproblem,suchasevaluatingaseriesbyanitesum,adeniteintegralbyanumerical integrationmethod,solvingadierentialequationbyanumericalmethod. Example23. ThevolumeoftheEarthcouldbecomputedusingtheformulaforthevolumeofa sphere, V =4 = 3 r 3 ,where r istheradius.Thiscomputationinvolvesthefollowingapproximations: 1.TheEarthismodeledasaspheremodelingerror 2.Radius r 6370 kmisbasedonempiricalmeasurementsuncertaintyinphysicaldata 3.Allthenumericalcomputationsaredoneinacomputermachineerror 4.Thevalueof hastobetruncatedmathematicaltruncationerror Exercise1.3-6: Thefollowingisfrom"Numericalmathematicsandcomputing"byCheney &Kincaid[7]: In1996,theAriane5rocketlaunchedbytheEuropeanSpaceAgencyexploded40seconds afterlift-ofromKourou,FrenchGuiana.Aninvestigationdeterminedthatthehorizontalvelocity requiredtheconversionofa64-bitoating-pointnumbertoa16-bitsignedinteger.Itfailedbecause thenumberwaslargerthan32,767,whichwasthelargestintegerofthistypethatcouldbestoredin memory.Therocketanditscargowerevaluedat$500million. Searchonline,orinthelibrary,tondanotherexampleofcomputerarithmeticgoingvery wrong!Writeashortparagraphexplainingtheproblem,andgiveareference.

PAGE 43

Chapter2 Solutionsofequations:Root-nding AryaandthemysteryoftheRhindpapyrus Collegelifeisfullofadventures,somehopefullyofintellectualnature,andAryaisdoingherpartby takingahistoryofscienceclass.ShelearnsabouttheRhindpapyrus;anancientEgyptianpapyrus purchasedbyanantiquariannamedHenryRhindinLuxor,Egypt,in1858. Figure2.1:RhindMathematicalPapyrus.BritishMuseumImageunderaCreativeCommonslicense. Thepapyrushasacollectionofmathematicalproblemsandtheirsolutions;atranslationisgiven byChaceandManning[2].ThefollowingisProblem26,takenfrom[2]: 42

PAGE 44

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 43 Aquantityandits 1 = 4 addedtogetherbecome15.Whatisthequantity? Assume4. n 14 n 1 = 41 Total 5 Asmanytimesas5mustbemultipliedtogive15,somanytimes4mustbemultiplied togivetherequirednumber.Multiply5soastoget15. n 15 n 210 Total 3 Multiply3by4. 13 26 n 412 Thequantityis 12 1 = 43 Total 15 Arya'sinstructorknowsshehastakenmathclassesandasksherifshecoulddecipherthis solution.AlthoughArya'sinitialreactiontothisassignmentcanbebestdescribedusingtheword "despair",shequicklyrealizesitisnotasbadasshethought.Hereisherthoughtprocess:the question,inourmodernnotationis,nd x if x + x= 4=15 .Thesolutionstartswithaninitialguess p =4 .Itthenevaluates x + x= 4 when x = p ,andndstheresulttobe5:however,whatweneedis 15,not5,andifwemultiplybothsidesof p + p= 4=5 by3,weget p + p = 4=15 .Therefore, thesolutionis 3 p =12 . Hereisamoregeneralanalysisofthissolutiontechnique.Supposewewanttosolvetheequation g x = a ,andthat g isalinearmap,thatis, g x = g x foranyconstant .Then,thesolution

PAGE 45

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 44 is x = ap=b where p isaninitialguesswith g p = b .Toseethis,simplyobserve g ap b = a b g p = a: Thegeneralproblem HowcanwesolveequationsthatarefarcomplicatedthantheancientEgyptianssolved?For example,howcanwesolve x 2 +5cos x =0 ?Stateddierently,howcanwendtheroot p such that f p =0 ,where f x = x 2 +5cos x ?Inthischapterwewilllearnsomeiterativemethods tosolveequations.An iterativemethod producesasequenceofnumbers p 1 ;p 2 ;::: suchthat lim n !1 p n = p; and p istherootweseek.Ofcourse,wecannotcomputetheexactlimit,sowe stoptheiterationatsomelarge N ,anduse p N asanapproximationto p . Thestoppingcriteria Withanyiterativemethod,akeyquestionishowtodecidewhentostoptheiteration.Howwell does p N approximate p ? Let > 0 beasmalltolerancepickedaheadoftime.Herearesomestoppingcriteria:stopwhen 1. j p N )]TJ/F82 10.9091 Tf 10.91 0 Td [(p N )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 j < , 2. p N )]TJ/F49 7.9701 Tf 6.586 0 Td [(p N )]TJ/F47 5.9776 Tf 5.756 0 Td [(1 p N <;p N 6 =0 ,or 3. j f p N j <: However,dicultiescanarisewithanyofthesecriteria: 1.Itispossibletohaveasequence f p n g suchthat p n )]TJ/F82 10.9091 Tf 10.91 0 Td [(p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ! 0 but f p n g diverges. 2.Itispossibletohave j f p N j smallcalledresidualbut p N notcloseto p . Inournumericalresults,wewillexperimentwithvariousstoppingcriteria.However,thesecond criterionisusuallypreferredovertheothers.

PAGE 46

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 45 Exercise2.-1: Solvethefollowingproblemsanddiscusstheirrelevancetothestopping criteria. aConsiderthesequence p n where p n = P n i =1 1 i .Arguethat p n diverges,but lim n !1 p n )]TJ/F82 10.9091 Tf -430.25 -16.815 Td [(p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 =0 : bLet f x = x 10 .Clearly, p =0 isarootof f ,andthesequence p n = 1 n convergesto p .Show that f p n < 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(3 if n> 1 ,buttoobtain j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n j < 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(3 , n mustbegreaterthan 10 3 . 2.1Erroranalysisforiterativemethods Assumewehaveaniterativemethod f p n g thatconvergestotheroot p ofsomefunction.Howcan weassesstherateofconvergence? Denition24. Suppose f p n g convergesto p .Ifthereareconstants C> 0 and > 1 suchthat j p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j C j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j ; .1 for n 1 ,thenwesay f p n g convergesto p withorder . Specialcases: If =1 and C< 1 ,wesaytheconvergenceislinear,andtherateofconvergenceis C .In thiscase,usinginduction,wecanshow j p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j C n j p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j : .2 TherearesomemethodsforwhichEquation2.2holds,butEquation2.1doesnothold forany C< 1 .Westillcallthesemethodstobeoflinearconvergence.Anexampleisthe bisectionmethod. If > 1 ,wesaytheconvergenceissuperlinear.Inparticular,thecase =2 iscalled quadraticconvergence. Example25. Considerthesequencesdenedby p n +1 =0 : 7 p n and p 1 =1 p n +1 =0 : 7 p 2 n and p 1 =1 :

PAGE 47

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 46 Therstsequenceconvergesto0linearly,andthesecondquadratically.Hereareafewiterations ofthesequences: n LinearQuadratic 1 0 : 70 : 7 4 0 : 244 : 75 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(3 8 5 : 76 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 3 : 16 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(40 Observehowfastquadraticconvergenceiscomparedtolinearconvergence. 2.2Bisectionmethod Let'srecalltheIntermediateValueTheoremIVT,Theorem6:Ifacontinuousfunction f dened on [ a;b ] satises f a f b < 0 ; thenthereexists p 2 [ a;b ] suchthat f p =0 : Hereistheideabehindthemethod.Ateachiteration,dividetheinterval [ a;b ] intotwosubintervalsandevaluate f atthemidpoint.Discardthesubintervalthatdoesnotcontaintherootand continuewiththeotherinterval. Example26. ComputetherstthreeiterationsbyhandforthefunctionplottedinFigure2.2. Figure2.2 Step1:Tostart,weneedtopickaninterval [ a;b ] thatcontainstheroot,thatis, f a f b < 0 .From theplot,itisclearthat [0 ; 4] isapossiblechoice.Inthenextfewsteps,wewillbeworkingwith asequenceofintervals.Forconvenience,let'slabelthemas [ a;b ]=[ a 1 ;b 1 ] ; [ a 2 ;b 2 ] ; [ a 3 ;b 3 ] ; etc.Ourrstintervalisthen [ a 1 ;b 1 ]=[0 ; 4] .Nextwendthemidpointoftheinterval, p 1 =4 = 2=2 ; anduseittoobtaintwosubintervals [0 ; 2] and [2 ; 4] .Onlyoneofthem containstheroot,andthatis [2 ; 4] .

PAGE 48

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 47 Step2:Fromthepreviousstep,ourcurrentintervalis [ a 2 ;b 2 ]=[2 ; 4] .Wendthemidpoint 1 p 2 = 2+4 2 =3 ,andformthesubintervals [2 ; 3] ; [3 ; 4] .Theonethatcontainstherootis [3 ; 4] . Step3:Wehave [ a 3 ;b 3 ]=[3 ; 4] .Themidpointis p 3 =3 : 5 .Wearenowprettyclosetotheroot visually,andwestopthecalculations! Inthissimpleexample,wedidnotconsider Stoppingcriteria It'spossiblethatthestoppingcriterionisnotsatisedinareasonableamountoftime.We needamaximumnumberofiterationswearewillingtorunthecodewith. Remark27. 1.Anumericallymorestableformulatocomputethemidpointis a + b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a 2 seeExample20. 2.Thereisaconvenientstoppingcriterionforthebisectionmethodthatwasnotmentioned before.Onecanstopwhentheinterval [ a;b ] atstep n issuchthat j a )]TJ/F82 10.9091 Tf 10.383 0 Td [(b j < .Thisissimilar totherststoppingcriteriondiscussedearlier,butnotthesame.Onecanalsousemorethan onestoppingcriterion;anexampleisinthePythoncodethatfollows. Pythoncodeforthebisectionmethod InExample26,wekepttrackoftheintervalsandmidpointsobtainedfromthebisectionmethod, bylabelingthemas [ a 1 ;b 1 ] ; [ a 2 ;b 2 ] ;:::; and p 1 ;p 2 ;::: .Soatstep n ofthemethod,weknowweare workingontheinterval [ a n ;b n ] anditsmidpointis p n .Thisapproachwillbeusefulwhenwestudy theconvergenceofthemethodinthenexttheorem.However,keepingtrackoftheintervalsand midpointsisnotneededinthecomputercode.Instead,inthePythoncodebelow,wewilllet [ a;b ] bethecurrentintervalweareworkingon,andwhenweobtainanewintervalinthefollowing step,wewillsimplycallthenewinterval [ a;b ] ,overwritingtheoldone.Similarly,wewillcallthe midpoint p ,andupdateitateachstep. In[1]:importnumpyasnp In[2]:defbisectionf,a,b,eps,N: n=1 p=0. #toensurethevalueofpcarriesoutofthewhileloop whilen<=N: p=a+b-a/2 ifnp.isclosefp,0ornp.absa-b
PAGE 49

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 48 print ' pis ' ,p, ' andtheiterationnumberis ' ,n return iffa*fp<0: b=p else: a=p n+=1 y=fp print ' Methoddidnotconverge.Thelastiterationgives ' , p, ' withfunctionvalue ' ,y Let'susethebisectionmethodtondtherootof f x = x 5 +2 x 3 )]TJ/F81 10.9091 Tf 10 0 Td [(5 x )]TJ/F81 10.9091 Tf 10 0 Td [(2 ,with =10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(4 .Note that [0 ; 2] containsaroot,since f < 0 and f > 0 .Weset N =20 below. In[3]:bisectionlambdax:x**5+2*x**3-5*x-2,0,2,1e-4,20 pis1.319671630859375andtheiterationnumberis16 Thevalueofthefunctionattheestimatedrootis: In[4]:x=1.319671630859375 x**5+2*x**3-5*x-2 Out[4]:0.000627945623044468 Let'sseewhathappensif N issettoosmallandthemethoddoesnotconverge. In[5]:bisectionlambdax:x**5+2*x**3-5*x-2,0,2,1e-4,5 Methoddidnotconverge.Thelastiterationgives1.3125with functionvalue-0.14562511444091797 Theorem28. Supposethat f 2 C 0 [ a;b ] and f a f b < 0 .Thebisectionmethodgeneratesa sequence f p n g approximatingazero p of f x with j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 2 n ;n 1 : Proof. Letthesequences f a n g and f b n g denotetheleft-endandright-endpointsofthesubintervals generatedbythebisectionmethod.Sinceateachsteptheintervalishalved,wehave b n )]TJ/F82 10.9091 Tf 10.909 0 Td [(a n = 1 2 b n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(a n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 :

PAGE 50

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 49 Bymathematicalinduction,weget b n )]TJ/F82 10.9091 Tf 10.909 0 Td [(a n = 1 2 b n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(a n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 = 1 2 2 b n )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(a n )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 = ::: = 1 2 n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 b 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 1 : Therefore b n )]TJ/F82 10.9091 Tf 10.909 0 Td [(a n = 1 2 n )]TJ/F47 5.9776 Tf 5.756 0 Td [(1 b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a : Observethat j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j 1 2 b n )]TJ/F82 10.9091 Tf 10.909 0 Td [(a n = 1 2 n b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a .3 andthus j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j! 0 as n !1 : Corollary29. Thebisectionmethodhaslinearconvergence. Proof. Thebisectionmethoddoesnotsatisfy2.1forany C< 1 ,butitsatisesavariantof2.2 with C =1 = 2 fromtheprevioustheorem. Findingthenumberofiterationstoobtainaspeciedaccuracy: Canwend n thatwill ensure j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j 10 )]TJ/F49 7.9701 Tf 6.586 0 Td [(L forsomegiven L ? Wehave,fromtheproofoftheprevioustheoremsee2.3: j p n )]TJ/F82 10.9091 Tf 11.072 0 Td [(p j 1 2 n b )]TJ/F82 10.9091 Tf 11.072 0 Td [(a .Therefore,we canmake j p n )]TJ/F82 10.9091 Tf 11.249 0 Td [(p j 10 )]TJ/F49 7.9701 Tf 6.587 0 Td [(L ,bychoosing n largeenoughsothattheupperbound 1 2 n b )]TJ/F82 10.9091 Tf 11.249 0 Td [(a isless than 10 )]TJ/F49 7.9701 Tf 6.587 0 Td [(L : 1 2 n b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 10 )]TJ/F49 7.9701 Tf 6.586 0 Td [(L n log 2 b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 10 )]TJ/F49 7.9701 Tf 6.586 0 Td [(L : Example30. Determinethenumberofiterationsnecessarytosolve f x = x 5 +2 x 3 )]TJ/F81 10.9091 Tf 10.526 0 Td [(5 x )]TJ/F81 10.9091 Tf 10.526 0 Td [(2=0 withaccuracy 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(4 ;a =0 ;b =2 : Solution. Since n log 2 )]TJ/F46 7.9701 Tf 13.267 -4.541 Td [(2 10 )]TJ/F47 5.9776 Tf 5.756 0 Td [(4 =4log 2 10+1=14 : 3 ,thenumberofrequirediterationsis15. Exercise2.2-1: Findtherootof f x = x 3 +4 x 2 )]TJ/F81 10.9091 Tf 10.417 0 Td [(10 usingthebisectionmethod,withthe followingspecications: aModifythePythoncodeforthebisectionmethodsothattheonlystoppingcriterioniswhether f p =0 removetheothercriterionfromthecode.Also,addaprintstatementtothecode, sothateverytimeanew p iscomputed,Pythonprintsthevalueof p andtheiterationnumber. bFindthenumberofiterations N necessarytoobtainanaccuracyof 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(4 fortheroot,using thetheoreticalresultsofSection2.2.Thefunction f x hasonerealrootin ; 2 ,soset a =1 ;b =2 . cRunthecodeusingthevaluefor N obtainedinpartbtocompute p 1 ;p 2 ;:::;p N set a = 1 ;b =2 inthemodiedPythoncode.

PAGE 51

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 50 dTheactualroot,correcttosixdigits,is p =1 : 36523 .Findtheabsoluteerrorwhen p N isused toapproximatetheactualroot,thatis,nd j p )]TJ/F82 10.9091 Tf 11.356 0 Td [(p N j .Comparethiserror,withtheupper boundfortheerrorusedinpartb. Exercise2.2-2: Findanapproximationto 25 1 = 3 correcttowithin 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(5 usingthebisection algorithm,followingthestepsbelow: aFirstexpresstheproblemas f x =0 with p =25 1 = 3 theroot. bFindaninterval a;b thatcontainstheroot,usingIntermediateValueTheorem. cDetermine,analytically,thenumberofiteratesnecessarytoobtaintheaccuracyof 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(5 . dUsethePythoncodeforthebisectionmethodtocomputetheiteratefromc,andcompare theactualabsoluteerrorwith 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(5 . 2.3Newton'smethod Suppose f 2 C 2 [ a;b ] ; i.e., f;f 0 ;f 00 arecontinuouson [ a;b ] : Let p 0 bea"good"approximationto p suchthat f 0 p 0 6 =0 and j p )]TJ/F82 10.9091 Tf 10.169 0 Td [(p 0 j is"small".FirstTaylorpolynomialfor f at p 0 withtheremainder termis f x = f p 0 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 f 0 p 0 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 2 2! f 00 x where x isanumberbetween x and p 0 : Substitute x = p andnote f p =0 toget: 0= f p 0 + p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 f 0 p 0 + p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 2 2! f 00 p where p isanumberbetween p and p 0 : Rearrangetheequationtoget p = p 0 )]TJ/F82 10.9091 Tf 13.502 7.38 Td [(f p 0 f 0 p 0 )]TJ/F81 10.9091 Tf 12.104 7.38 Td [( p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 2 2 f 00 p f 0 p 0 : .4 If j p )]TJ/F82 10.9091 Tf 11.209 0 Td [(p 0 j is"small"then p )]TJ/F82 10.9091 Tf 11.209 0 Td [(p 0 2 isevensmaller,andtheerrortermcanbedroppedtoobtain thefollowingapproximation: p p 0 )]TJ/F82 10.9091 Tf 13.502 7.38 Td [(f p 0 f 0 p 0 : TheideainNewton'smethodistosetthenextiterate, p 1 ,tothisapproximation: p 1 = p 0 )]TJ/F82 10.9091 Tf 13.503 7.38 Td [(f p 0 f 0 p 0 :

PAGE 52

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 51 Equation2.4canbewrittenas p = p 1 )]TJ/F81 10.9091 Tf 12.104 7.38 Td [( p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 2 2 f 00 p f 0 p 0 : .5 Summary: Startwithaninitialapproximation p 0 to p andgeneratethesequence f p n g 1 n =1 by p n = p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 )]TJ/F82 10.9091 Tf 13.502 7.38 Td [(f p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 f 0 p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 ;n 1 : .6 ThisiscalledNewton'smethod. Graphicalinterpretation: Startwith p 0 : Drawthetangentlineat p 0 ;f p 0 andapproximate p bytheintercept p 1 ofthe line: f 0 p 0 = 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p 0 p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 = )]TJ/F82 10.9091 Tf 11.078 7.38 Td [(f p 0 f 0 p 0 p 1 = p 0 )]TJ/F82 10.9091 Tf 13.502 7.38 Td [(f p 0 f 0 p 0 : Nowdrawthetangentat p 1 ;f p 1 andcontinue. Remark31. 1.ClearlyNewton'smethodwillfailif f 0 p n =0 forsome n .Graphicallythismeansthe tangentlineisparalleltothe x -axissowecannotgetthe x -intercept. 2.Newton'smethodmayfailtoconvergeiftheinitialguess p 0 isnotcloseto p: InFigure2.3, eitherchoicefor p 0 resultsinasequencethatoscillatesbetweentwopoints. Figure2.3:Non-convergingbehaviorforNewton'smethod 3.Newton'smethodrequires f 0 x isknownexplicitly.

PAGE 53

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 52 Exercise2.3-1: Sketchthegraphfor f x = x 2 )]TJ/F81 10.9091 Tf 11.401 0 Td [(1 .Whataretherootsoftheequation f x =0 ? 1.Let p 0 =1 = 2 andndthersttwoiterations p 1 ;p 2 ofNewton'smethodbyhand.Markthe iteratesonthegraphof f yousketched.Doyouthinktheiterateswillconvergetoazeroof f ? 2.Let p 0 =0 andnd p 1 .Whatareyourconclusionsabouttheconvergenceoftheiterates? PythoncodeforNewton'smethod ThePythoncodebelowisbasedonEquation2.6.Thevariable pin inthecodecorrespondsto p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 ,and p correspondsto p n .Thecodeoverwritesthesevariablesastheiterationcontinues.Also noticethatthecodehastwofunctionsasinputs; f and fprime thederivative f 0 . In[1]:defnewtonf,fprime,pin,eps,N: n=1 p=0. #toensurethevalueofpcarriesoutofthewhileloop whilen<=N: p=pin-fpin/fprimepin ifnp.isclosefp,0ornp.absp-pin
PAGE 54

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 53 ax.spines[ ' right ' ].set_position ' center ' ax.spines[ ' bottom ' ].set_position ' center ' ax.spines[ ' top ' ].set_position ' center ' ax.set_ylim[-40,40] plt.plotx,y; Thederivativeis f 0 =5 x 4 +6 x 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(5 ,wesetpin =1 ,eps = =10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(4 ,and N =20 ,inthecode. In[4]:newtonlambdax:x**5+2*x**3-5*x-2,lambdax:5*x**4+6*x**2-5,1,1e-4,20 pis1.3196411672093726andtheiterationnumberis6 Recallthatthebisectionmethodrequired16iterationstoapproximatetherootin [0 ; 2] as p =1 : 31967 .However,thestoppingcriterionusedinbisectionandNewton'smethodsareslightly dierent.1.3196istherightmostrootintheplot.Butthereareotherrootsofthefunction.Let's runthecodewithpin=0. In[5]:newtonlambdax:x**5+2*x**3-5*x-2,lambdax:5*x**4+6*x**2-5,0,1e-4,20 pis-0.43641313299799755andtheiterationnumberis4 Nowweusepin= )]TJ/F81 10.9091 Tf 8.484 0 Td [(2 : 0 whichwillgivetheleftmostroot.

PAGE 55

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 54 In[6]:newtonlambdax:x**5+2*x**3-5*x-2,lambdax:5*x**4+6*x**2-5,-2,1e-4,20 pis-1.0000000001014682andtheiterationnumberis7 Theorem32. Let f 2 C 2 [ a;b ] andassume f p =0 ;f 0 p 6 =0 for p 2 a;b : If p 0 ischosen sucientlycloseto p; thenNewton'smethodgeneratesasequencethatconvergesto p with lim n !1 p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n +1 p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n 2 = )]TJ/F82 10.9091 Tf 11.259 7.38 Td [(f 00 p 2 f 0 p : Proof. Since f 0 iscontinuousand f 0 p 6 =0 ; thereexistsaninterval I =[ p )]TJ/F82 10.9091 Tf 11.552 0 Td [(;p + ] onwhich f 0 6 =0 : Let M = max x 2 I j f 00 x j 2min x 2 I j f 0 x j : Pick p 0 fromtheinterval I whichmeans j p )]TJ/F82 10.9091 Tf 10.416 0 Td [(p 0 j ,sucientlycloseto p sothat M j p )]TJ/F82 10.9091 Tf 10.416 0 Td [(p 0 j < 1 : FromEquation2.5wehave: j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 1 j = j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 jj p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 j 2 f 00 p f 0 p 0 < j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 jj p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 j M< j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 j : .7 Multiplybothsidesof j p )]TJ/F82 10.9091 Tf 9.812 0 Td [(p 1 j < j p )]TJ/F82 10.9091 Tf 9.812 0 Td [(p 0 j by M toget M j p )]TJ/F82 10.9091 Tf 9.812 0 Td [(p 1 j
PAGE 56

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 55 Since M j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 0 j < 1 ; j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n j! 0 as n !1 : Therefore lim n !1 p n = p: Finally, lim n !1 p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n +1 p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n 2 =lim n !1 )]TJ/F81 10.9091 Tf 9.68 7.38 Td [(1 2 f 00 n f 0 p n ; andsince p n ! p; and n isbetween p n and p; n ! p ,andtherefore lim n !1 p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n +1 p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n 2 = )]TJ/F81 10.9091 Tf 9.68 7.38 Td [(1 2 f 00 p f 0 p provingthetheorem. Corollary33. Newton'smethodhasquadraticconvergence. Proof. Recallthatquadraticconvergencemeans j p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j C j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j 2 ; forsomeconstant C> 0 .Takingtheabsolutevaluesofthelimitestablishedintheprevioustheorem, weobtain lim n !1 p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n +1 p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n 2 =lim n !1 j p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j 2 = 1 2 f 00 p f 0 p : Let C 0 = 1 2 f 00 p f 0 p .Fromthedenitionoflimitofasequence,forany > 0 ,thereexistsaninteger N> 0 suchthat j p n +1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(p j j p n )]TJ/F49 7.9701 Tf 6.587 0 Td [(p j 2 N: Set C = C 0 + toobtain j p n +1 )]TJ/F82 10.9091 Tf 9.803 0 Td [(p j C j p n )]TJ/F82 10.9091 Tf 9.803 0 Td [(p j 2 for n>N . Example34. TheBlack-Scholes-MertonBSMformula,forwhichMyronScholesandRobert MertonwereawardedtheNobelprizeineconomicsin1997,computesthefairpriceofacontract knownasthe Europeancalloption .Thiscontractgivesitsownertherighttopurchasetheasset thecontractiswrittenonforexample,astock,foraspecicpricedenotedby K andcalledthe strikepriceorexerciseprice,atafuturetimedenotedby T andcalledtheexpiry.Theformula givesthevalueoftheEuropeancalloption, C ,as C = S d 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(Ke )]TJ/F49 7.9701 Tf 6.586 0 Td [(rT d 2 where S isthepriceoftheassetatthepresenttime, r istherisk-freeinterestrate,and x isthe distributionfunctionofthestandardnormalrandomvariable,givenby x = 1 p 2 Z x e )]TJ/F49 7.9701 Tf 6.586 0 Td [(t 2 = 2 dt: Theconstants d 1 ; d 2 areobtainedfrom d 1 = log S=K + r + 2 = 2 T p T ; d 2 = d 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [( p T:

PAGE 57

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 56 AlltheconstantsintheBSMformulacanbeobserved,exceptfor ,whichiscalledthevolatility oftheunderlyingasset.Ithastobeestimatedfromempiricaldatainsomeway.Wewantto concentrateontherelationshipbetween C and ,andthinkof C asafunctionof only.We rewritetheBSMformulaemphasizing : C = S d 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(Ke )]TJ/F49 7.9701 Tf 6.586 0 Td [(rT d 2 Itmaylookliketheindependentvariable ismissingontherighthandsideoftheaboveformula, butitisnot:theconstants d 1 ; d 2 bothdependon .Wecanalsothinkabout d 1 ; d 2 asfunctions of . Therearetwoquestionsnancialengineersareinterestedin: Computetheoptionprice C basedonanestimateof Observethepriceofanoption ^ C tradedinthemarket,andnd forwhichtheBSM formulagivestheoutput ^ C ,i.e, C = ^ C .Thevolatility obtainedinthiswayiscalled the impliedvolatility . Thesecondquestioncanbeansweredusingaroot-ndingmethod,inparticular,Newton'smethod. Tosummarize,wewanttosolvetheequation: C )]TJ/F81 10.9091 Tf 13.379 2.758 Td [(^ C =0 where ^ C isagivenconstant,and C = S d 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(Ke )]TJ/F49 7.9701 Tf 6.587 0 Td [(rT d 2 : TouseNewton'smethod,weneed C 0 = dC d .Since d 1 ; d 2 arefunctionsof ,wehave dC d = S d d 1 d )]TJ/F82 10.9091 Tf 10.909 0 Td [(Ke )]TJ/F49 7.9701 Tf 6.587 0 Td [(rT d d 2 d : .9 Let'scomputethederivativesontherighthandsideof2.9. d d 1 d = d d 1 p 2 Z d 1 e )]TJ/F49 7.9701 Tf 6.587 0 Td [(t 2 = 2 dt = 1 p 2 0 B B @ d d Z d 1 e )]TJ/F49 7.9701 Tf 6.587 0 Td [(t 2 = 2 dt | {z } u 1 C C A : Wewillusethechainruletocomputethederivative d d Z d 1 e )]TJ/F49 7.9701 Tf 6.586 0 Td [(t 2 = 2 dt | {z } u = du d : du d = du d d 1 d d 1 d :

PAGE 58

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 57 TherstderivativefollowsfromtheFundamentalTheoremofCalculus du d d 1 = e )]TJ/F101 7.9701 Tf 6.586 0 Td [(d 1 2 = 2 ; andthesecondderivativeisanapplicationofthequotientruleofdierentiation d d 1 d = d d log S=K + r + 2 = 2 T p T = p T )]TJ/F81 10.9091 Tf 12.104 7.38 Td [(log S=K + r + 2 = 2 T 2 p T : Puttingthepiecestogether,wehave d d 1 d = e )]TJ/F101 7.9701 Tf 6.586 0 Td [(d 1 2 = 2 p 2 p T )]TJ/F81 10.9091 Tf 12.104 7.38 Td [(log S=K + r + 2 = 2 T 2 p T : Goingbacktothesecondderivativeweneedtocomputeinequation2.9,wehave: d d 2 d = 1 p 2 d d Z d 2 e )]TJ/F49 7.9701 Tf 6.587 0 Td [(t 2 = 2 dt : UsingthechainruleandtheFundamentalTheoremofCalculusweobtain d d 2 d = e )]TJ/F101 7.9701 Tf 6.587 0 Td [(d 2 2 = 2 p 2 d d 2 d : Since d 2 isdenedas d 2 = d 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [( p T ,wecanexpress d d 2 =d intermsof d d 1 =d as: d d 2 d = d d 1 d )]TJ 10.909 9.769 Td [(p T: Finally,wehavethederivativeweneed: dC d = Se )]TJ/F102 5.9776 Tf 7.782 4.323 Td [(d 1 2 2 p 2 p T )]TJ/F81 10.9091 Tf 12.105 8.016 Td [(log S K + r + 2 2 T 2 p T ! + K e )]TJ/F46 7.9701 Tf 6.586 0 Td [( rT + d 2 2 2 p 2 log S K + r + 2 2 T 2 p T ! .10 WearereadytoapplyNewton'smethodtosolvetheequation C )]TJ/F81 10.9091 Tf 13.766 2.757 Td [(^ C =0 .Nowlet'snd somedata. TheGeneralElectricCompanyGEstockis$7.01onDec8,2018,andaEuropeancalloption onthisstock,expiringonDec14,2018,ispricedat$0.10.Theoptionhasstrikeprice K =$7 : 5 . Therisk-freeinterestrateis2.25%.Theexpiry T ismeasuredinyears,andsincethereare252 tradingdaysinayear, T =6 = 252 .WeputthisinformationinPython: In[1]:S=7.01 K=7.5 r=0.0225 T=6/252

PAGE 59

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 58 Wehavenotdiscussedhowtocomputethedistributionfunctionofthestandardnormalrandomvariable x = 1 p 2 R x e )]TJ/F49 7.9701 Tf 6.586 0 Td [(t 2 = 2 dt .InChapter4,wewilldiscusshowtocomputeintegrals numerically,butforthisexample,wewillusethebuilt-infunctionPythonhasfor x .Itisina subpackageofSciPycalled stats : In[2]:fromscipyimportstats Thefollowingdenes stdnormal asthestandardnormalrandomvariable. In[3]:stdnormal=stats.normloc=0,scale=1 Thememberfunction cdf xof stdnormal computesthestandardnormaldistributionfunction at x .Wewriteafunction phix basedonthismemberfunction,matchingournotation x for thedistributionfunction. In[4]:phi=lambdax:stdnormal.cdfx Nextwedene C and C 0 .InthePythoncode,wereplace by x . In[5]:defcx: d1=np.logS/K+r+x**2/2*T/x*np.sqrtT d2=d1-x*np.sqrtT returnS*phid1-K*np.exp-r*T*phid2 Thefunction cprime xisbasedonequation2.10: In[6]:defcprimex: d1=np.logS/K+r+x**2/2*T/x*np.sqrtT d2=d1-x*np.sqrtT A=np.logS/K+r+x**2/2*T/np.sqrtT*x**2 returnS*np.exp-d1**2/2/np.sqrt2*np.pi*np.sqrtT-A +K*np.exp-r*T+d2**2/2*A/np.sqrt2*np.pi Wethenloadthe newton functionandrunittondtheimpliedvolatilitywhichturnsoutto be62%. In[7]:newtonlambdax:cx-0.1,cprime,1,1e-4,60 pis0.6231138483741047andtheiterationnumberis3

PAGE 60

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 59 2.4Secantmethod OnedrawbackofNewton'smethodisthatweneedtoknow f 0 x explicitlytoevaluate f 0 p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 in p n = p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 )]TJ/F82 10.9091 Tf 13.502 7.38 Td [(f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 f 0 p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ;n 1 : Ifwedonotknow f 0 x explicitly,orifitscomputationisexpensive,wemightapproximate f 0 p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 bythenitedierence f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 h .11 forsomesmall h: Wethenneedtocomputetwovaluesof f ateachiterationtoapproximate f 0 . Determining h inthisformulabringssomediculty,butthereisawaytogetaroundthis.Wewill usetheiteratesthemselvestorewritethenitedierence2.11as f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 : Then,therecursionfor p n simpliesas p n = p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 )]TJ/F82 10.9091 Tf 28.425 7.38 Td [(f p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 f p n )]TJ/F47 5.9776 Tf 5.756 0 Td [(1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(f p n )]TJ/F47 5.9776 Tf 5.756 0 Td [(2 p n )]TJ/F47 5.9776 Tf 5.756 0 Td [(1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(p n )]TJ/F47 5.9776 Tf 5.756 0 Td [(2 = p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 ;n 2 : .12 Thisiscalledthesecantmethod.Observethat 1.Noadditionalfunctionevaluationsareneeded, 2.Therecursionrequirestwoinitialguesses p 0 ;p 1 : Geometricinterpretation :Theslopeofthesecantlinethroughthepoints p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ;f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 and p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 ;f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 is f p n )]TJ/F47 5.9776 Tf 5.756 0 Td [(1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(f p n )]TJ/F47 5.9776 Tf 5.757 0 Td [(2 p n )]TJ/F47 5.9776 Tf 5.756 0 Td [(1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(p n )]TJ/F47 5.9776 Tf 5.757 0 Td [(2 .The x -interceptofthesecantline,whichissetto p n ,is 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 = f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 p n = p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 )]TJ/F82 10.9091 Tf 10.91 0 Td [(f p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 f p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 whichistherecursionofthesecantmethod. Thefollowingtheoremshowsthatiftheinitialguessesare"good",thesecantmethodhas superlinearconvergence.AproofcanbefoundinAtkinson[3]. Theorem35. Let f 2 C 2 [ a;b ] andassume f p =0 ;f 0 p 6 =0 ; for p 2 a;b : Iftheinitialguesses p 0 ;p 1 aresucientlycloseto p; thentheiteratesofthesecantmethodconvergeto p with lim n !1 j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n +1 j j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n j r 0 = f 00 p 2 f 0 p r 1 where r 0 = p 5+1 2 1 : 62 ;r 1 = p 5 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 2 0 : 62 .

PAGE 61

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 60 Pythoncodeforthesecantmethod ThefollowingcodeisbasedonEquation2.12;therecursionforthesecantmethod.Theinitial guessesarecalled pzero and pone inthecode.ThesamestoppingcriterionasinNewton'smethod isused.Noticethatonceanewiterate p iscomputed, pone isupdatedas p ,and pzero isupdated as pone . In[1]:defsecantf,pzero,pone,eps,N: n=1 p=0. #toensrethevalueofpcarriesoutofthewhileloop whilen<=N: p=pone-fpone*pone-pzero/fpone-fpzero ifnp.isclosefp,0ornp.absp-pone
PAGE 62

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 61 bOnecancombinethebisectionmethodandNewton'smethodtodevelopahybridmethod thatconvergesforawiderrangeofstartingvalues p 0 ,andhasbetterconvergenceratethan thebisectionmethod. WriteaPythoncodeforabisection-Newtonhybridmethod,asdescribedbelow.Youcanuse thePythoncodesforthebisectionandNewton'smethodsfromthelecturenotes.Yourcode willinput f;f 0 ;a;b;;N where f;f 0 arethefunctionanditsderivative, a;b isanintervalthat containstherooti.e., f a f b < 0 ,and ;N arethetoleranceandthemaximumnumber ofiterations.ThecodewillusethesamestoppingcriterionusedinNewton'smethod. Themethodwillstartwithcomputingthemidpointof a;b ,callit p 0 ,anduseNewton's methodwithinitialguess p 0 toobtain p 1 .Itwillthencheckwhether p 1 2 a;b .If p 1 2 a;b , thenthecodewillcontinueusingNewton'smethodtocomputethenextiteration p 2 .If p 1 = 2 a;b ,thenwewillnotaccept p 1 asthenextiteration:insteadthecodewillswitch tothebisectionmethod,determinewhichsubintervalamong a;p 0 ; p 0 ;b containstheroot, updatestheinterval a;b asthesubintervalthatcontainstheroot,andsets p 1 tothemidpoint ofthisinterval.Once p 1 isobtained,thecodewillcheckifthestoppingcriterionissatised. Ifitissatised,thecodewillreturn p 1 andtheiterationnumber,andterminate.Ifitis notsatised,thecodewilluseNewton'smethod,with p 1 astheinitialguess,tocompute p 2 .Thenitwillcheckwhether p 2 2 a;b ,andcontinueinthisway.Ifthecodedoesnot terminateafter N iterations,outputanerrormessagesimilartoNewton'smethod. Applythehybridmethodto: apolynomialwithaknownroot,andcheckifthemethodndsthecorrectroot; y =log x with a;b = ; 6 ,forwhichNewton'smethodfailedinparta. cDoyouthinkingeneralthehybridmethodconvergestotheroot,providedtheinitialinterval a;b containstheroot,foranystartingvalue p 0 ?Explain. 2.5Muller'smethod Thesecantmethodusesalinearfunctionthatpassesthrough p 0 ;f p 0 and p 1 ;f p 1 tondthe nextiterate p 2 : Muller'smethodtakesthreeinitialapproximations,passesaparabolaquadratic polynomialthrough p 0 ;f p 0 ; p 1 ;f p 1 , p 2 ;f p 2 ,anduses one oftherootsofthepolynomial asthenextiterate. Letthequadraticpolynomialwritteninthefollowingform P x = a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 2 + b x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 + c: .13

PAGE 63

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 62 Solvethefollowingequationsfor a;b;c P p 0 = f p 0 = a p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 2 + b p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 + c P p 1 = f p 1 = a p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 2 + b p 1 )]TJ/F82 10.9091 Tf 10.91 0 Td [(p 2 + c P p 2 = f p 2 = c toget c = f p 2 b = p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 f p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p 2 p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 1 )]TJ/F81 10.9091 Tf 12.105 7.381 Td [( p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 f p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p 2 p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 1 .14 a = f p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p 2 p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 1 )]TJ/F82 10.9091 Tf 22.477 7.38 Td [(f p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f p 2 p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 p 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 1 : Nowthatwehavedetermined P x ,thenextstepistosolve P x =0 ,andsetthenextiterate p 3 toitssolution.Tothisend,put w = x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 in2.13torewritethequadraticequationas aw 2 + bw + c =0 : Fromthequadraticformula,weobtaintheroots ^ w =^ x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(2 c b p b 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(4 ac : .15 Let = b 2 )]TJ/F81 10.9091 Tf 11.718 0 Td [(4 ac .Wehavetworootswhichcouldbecomplexnumbers, )]TJ/F81 10.9091 Tf 8.485 0 Td [(2 c= b + p and )]TJ/F81 10.9091 Tf 8.485 0 Td [(2 c= b )]TJ 10.365 9.236 Td [(p ,andweneedtopickoneofthem.Wewillpicktherootthatiscloserto p 2 ,inother words,therootthatmakes j ^ x )]TJ/F82 10.9091 Tf 11.08 0 Td [(p 2 j thesmallest.Ifthenumbersarecomplex,theabsolutevalue meansthenormofthecomplexnumber.Thereforewehave ^ x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 = 8 < : )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 c b + p if j b + p j > j b )]TJ 10.909 9.236 Td [(p j )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 c b )]TJ 6.587 6.753 Td [(p if j b + p jj b )]TJ 10.909 9.236 Td [(p j : .16 ThenextiterateofMuller'smethod, p 3 ,issettothevalueof ^ x obtainedfromtheabovecalculation, thatis, p 3 =^ x = 8 < : p 2 )]TJ/F46 7.9701 Tf 20.315 4.295 Td [(2 c b + p if j b + p j > j b )]TJ 10.909 9.236 Td [(p j p 2 )]TJ/F46 7.9701 Tf 20.315 4.295 Td [(2 c b )]TJ 6.587 6.753 Td [(p if j b + p jj b )]TJ 10.909 9.236 Td [(p j : Remark36. 1.Muller'smethodcanndrealaswellascomplexroots.

PAGE 64

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 63 2.TheconvergenceofMuller'smethodissuperlinear,thatis, lim n !1 j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n +1 j j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n j = f p 6 f 0 p )]TJ/F47 5.9776 Tf 5.756 0 Td [(1 2 where 1 : 84 ,provided f 2 C 3 [ a;b ] ;p 2 a;b ,and f 0 p 6 =0 . 3.Muller'smethodconvergesforavarietyofstartingvalueseventhoughpathologicalexamples thatdonotyieldconvergencecanbefoundforexample,whenthethreestartingvaluesfall onaline. PythoncodeforMuller'smethod ThefollowingPythoncodetakesinitialguesses p 0 ;p 1 ;p 2 writtenas pzero,pone,ptwo inthecode, computesthecoecients a;b;c fromEquation2.14,andsetstheroot p 3 to p .Itthenupdatesthe threeinitialguessesasthelastthreeiterates,andcontinuesuntilthestoppingcriterionissatised. Weneedtocomputethesquareroot,andtheabsolutevalue,ofpossiblycomplexnumbersin Equations2.15and2.16.ThePythonfunctionforthesquarerootofapossiblycomplexnumber z is complexz 0 : 5 ,anditsabsolutevalueis np.absz . In[1]:defmullerf,pzero,pone,ptwo,eps,N: n=1 p=0 whilen<=N: c=fptwo b1=pzero-ptwo*fpone-fptwo/pone-ptwo*pzero-pone b2=pone-ptwo*fpzero-fptwo/pzero-ptwo*pzero-pone b=b1-b2 a1=fpzero-fptwo/pzero-ptwo*pzero-pone a2=fpone-fptwo/pone-ptwo*pzero-pone a=a1-a2 d=complexb**2-4*a*c**0.5 ifnp.absb-d
PAGE 65

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 64 pzero=pone pone=ptwo ptwo=p n+=1 y=fp print ' Methoddidnotconverge.Thelastiterationgives ' , p, ' withfunctionvalue ' ,y Thepolynomial x 5 +2 x 3 )]TJ/F81 10.9091 Tf 9.239 0 Td [(5 x )]TJ/F81 10.9091 Tf 9.238 0 Td [(2 hasthreerealroots,andtwocomplexrootsthatareconjugates. Let'sndthemall,byexperimentingwithvariousinitialguesses. In[2]:mullerlambdax:x**5+2*x**3-5*x-2,0.5,1.0,1.5,1e-5,10 pis.3196411677283386+0jandtheiterationnumberis4 In[3]:mullerlambdax:x**5+2*x**3-5*x-2,0.5,0,-0.1,1e-5,10 pis-0.43641313299908585+0jandtheiterationnumberis5 In[4]:mullerlambdax:x**5+2*x**3-5*x-2,0,-0.1,-1,1e-5,10 pis-1+0jandtheiterationnumberis1 In[5]:mullerlambdax:x**5+2*x**3-5*x-2,5,10,15,1e-5,20 pis.05838598289491982+1.8626227582154478jandtheiterationnumberis18 2.6Fixed-pointiteration Manyroot-ndingmethodsarebasedontheso-calledxed-pointiteration,amethodwediscussin thissection. Denition37. Anumber p isaxed-pointforafunction g x if g p = p: Wehavetwoproblemsthatarerelatedtoeachother: Fixed-pointproblem :Find p suchthat g p = p: Root-ndingproblem :Find p suchthat f p =0 :

PAGE 66

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 65 Wecanformulatearoot-ndingproblemasaxed-pointproblem,andviceversa.Forexample, assumewewanttosolvetherootndingproblem, f p =0 : Dene g x = x )]TJ/F82 10.9091 Tf 8.909 0 Td [(f x ; andobservethat if p isaxed-pointof g x ,thatis, g p = p )]TJ/F82 10.9091 Tf 10.546 0 Td [(f p = p; then p isarootof f x .Herethefunction g isnotunique:therearemanywaysonecanrepresenttheroot-ndingproblem f p =0 asa xed-pointproblem,andaswewilllearnlater,notallwillbeusefultousindevelopingxed-point iterationalgorithms. Thenexttheoremanswersthefollowingquestions:Whendoesafunction g haveaxed-point? Ifithasaxed-point,isitunique? Theorem38. 1.If g isacontinuousfunctionon [ a;b ] and g x 2 [ a;b ] forall x 2 [ a;b ] ,then g hasatleastonexed-pointin [ a;b ] . 2.If,inaddition, j g x )]TJ/F82 10.9091 Tf 9.437 0 Td [(g y j j x )]TJ/F82 10.9091 Tf 9.437 0 Td [(y j forall x;y 2 [ a;b ] where 0 << 1 ; thenthexed-point isunique. Proof. Consider f x = g x )]TJ/F82 10.9091 Tf 11.604 0 Td [(x: Assume g a 6 = a and g b 6 = b otherwisetheproofisover. Then f a = g a )]TJ/F82 10.9091 Tf 11.744 0 Td [(a> 0 since g a mustbegreaterthan a ifit'snotequalto a .Similarly, f b = g b )]TJ/F82 10.9091 Tf 11.333 0 Td [(b< 0 : ThenfromIVT,thereexists p 2 a;b suchthat f p =0 ; or g p = p: To provepart2,supposetherearetwodierentxed-points p;q .Then j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(q j = j g p )]TJ/F82 10.9091 Tf 10.909 0 Td [(g q j j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(q j < j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(q j whichisacontradiction. Remark39. Let g beadierentiablefunctionon [ a;b ] suchthat j g 0 x j k forall x 2 a;b for somepositiveconstant k< 1 .Thenthehypothesisofpart2ofTheorem38issatisedwith = k: Indeed,fromthemeanvaluetheorem j g x )]TJ/F82 10.9091 Tf 10.909 0 Td [(g y j = j g 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(y j k j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(y j forall x;y 2 [ a;b ] : Thefollowingtheoremdescribeshowwecanndaxedpoint. Theorem40. If g isacontinuousfunctionon [ a;b ] satisfyingtheconditions 1. g x 2 [ a;b ] forall x 2 [ a;b ] , 2. j g x )]TJ/F82 10.9091 Tf 10.909 0 Td [(g y j j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(y j ; for x;y 2 [ a;b ] where 0 << 1 ; thenthe xed-pointiteration p n = g p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ;n 1 convergesto p; theuniquexed-pointof g in [ a;b ] ,foranystartingpoint p 0 2 [ a;b ] :

PAGE 67

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 66 Proof. Since p 0 2 [ a;b ] and g x 2 [ a;b ] forall x 2 [ a;b ],alliterates p n 2 [ a;b ] : Observethat j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n j = j g p )]TJ/F82 10.9091 Tf 10.909 0 Td [(g p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 j j p )]TJ/F82 10.9091 Tf 10.91 0 Td [(p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 j : Thenbyinduction, j p )]TJ/F82 10.9091 Tf 10.316 0 Td [(p n j n j p )]TJ/F82 10.9091 Tf 10.316 0 Td [(p 0 j : Since 0 << 1 ; n ! 0 as n !1 ; andthus p n ! p: Remark41. Theorem40stillholdsifthesecondcondition j g x )]TJ/F82 10.9091 Tf 10.441 0 Td [(g y j j x )]TJ/F82 10.9091 Tf 10.441 0 Td [(y j ,isreplacedby j g 0 x j k forall x 2 [ a;b ] where 0
PAGE 68

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 67 Figure2.5:Fixed-pointiteration: j g 0 p j > 1 . Example43. Considertheroot-ndingproblem x 3 )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 x 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1=0 on [1 ; 3] : 1.Writetheproblemasaxed-pointproblem, g x = x ,forsome g: Verifythatthehypothesis ofTheorem40orRemark41issatisedsothatthexed-pointiterationconverges. 2.Let p 0 =1 : UseCorollary42tond n thatensuresanestimateto p accuratetowithin 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(4 : Solution. 1.Thereareseveralwayswecanwritethisproblemas g x = x : aLet f x = x 3 )]TJ/F81 10.9091 Tf 10.773 0 Td [(2 x 2 )]TJ/F81 10.9091 Tf 10.773 0 Td [(1 ,and p beitsroot,thatis, f p =0 .Ifwelet g x = x )]TJ/F82 10.9091 Tf 10.773 0 Td [(f x , then g p = p )]TJ/F82 10.9091 Tf 11 0 Td [(f p = p; so p isaxed-pointof g .However,thischoicefor g willnot behelpful,since g doesnotsatisfytherstconditionofTheorem40: g x = 2 [1 ; 3] for all x 2 [1 ; 3] g = )]TJ/F81 10.9091 Tf 8.485 0 Td [(5 = 2 [1 ; 3] . bSince p isarootfor f ,wehave p 3 =2 p 2 +1 ,or p = p 2 +1 1 = 3 : Therefore, p isthe solutiontothexed-pointproblem g x = x where g x = x 2 +1 1 = 3 : g isincreasingon [1 ; 3] and g =1 : 44 ;g =2 : 67 ,thus g x 2 [1 ; 3] forall x 2 [1 ; 3] .Therefore, g satisestherstconditionofTheorem40. g 0 x = 4 x 3 x 2 +1 2 = 3 and g 0 =0 : 64 ;g 0 =0 : 56 and g 0 isdecreasingon [1 ; 3] . Therefore g satisestheconditioninRemark41with =0 : 64 . Then,fromTheorem40andRemark41,thexed-pointiterationconvergesif g x = x 2 +1 1 = 3 : 2.Take = k =0 : 64 inCorollary42andusebound: j p )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n j : 64 n max f 1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 ; 3 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 g =2 : 64 n : Wewant 2 : 64 n < 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(4 ; whichimplies n log0 : 64 < )]TJ/F81 10.9091 Tf 8.485 0 Td [(4log10 )]TJ/F81 10.9091 Tf 10.295 0 Td [(log2 ,or n> )]TJ/F46 7.9701 Tf 6.586 0 Td [(4log10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(log2 log0 : 64 22 : 19 : Therefore n =23 isthesmallestnumberofiterationsthatensuresanabsoluteerrorof 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(4 .

PAGE 69

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 68 Pythoncodeforxed-pointiteration Thefollowingcodestartswiththeinitialguess p 0 pzero inthecode,computes p 1 = g p 0 ,and checksifthestoppingcriterion j p 1 )]TJ/F82 10.9091 Tf 11.105 0 Td [(p 0 j < issatised.Ifitissatisedthecodeterminateswith thevalue p 1 .Otherwise p 1 issetto p 0 ,andthenextiterationiscomputed. In[1]:deffixedptg,pzero,eps,N: n=1 whilen
PAGE 70

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 69 Proof. Since g 0 iscontinuousand j g 0 p j < 1 ; thereexistsaninterval I =[ p )]TJ/F82 10.9091 Tf 11.362 0 Td [(;p + ] suchthat j g 0 x j k forall x 2 I; forsome k< 1 : Then,fromRemark39,weknow j g x )]TJ/F82 10.9091 Tf 10.993 0 Td [(g y j k j x )]TJ/F82 10.9091 Tf 10.993 0 Td [(y j forall x;y 2 I: Next,wearguethat g x 2 I if x 2 I .Indeed,if j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j <; then j g x )]TJ/F82 10.9091 Tf 10.91 0 Td [(p j = j g x )]TJ/F82 10.9091 Tf 10.909 0 Td [(g p jj g 0 jj x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j
PAGE 71

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 70 For c = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 = 4 ,thefunctionbecomes g x = x )]TJ/F49 7.9701 Tf 11.147 4.295 Td [(x 2 )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 4 .Pick p 0 =1 : 5 asthestartingpoint.Using thexed-pointiterationcodeofthepreviousexample,weobtain: In[4]:fixedptlambdax:x-x**2-2/4,1.5,1e-5,15 pis1.414214788550556andtheiterationnumberis9 Theabsoluteerroris: In[5]:1.414214788550556-2**.5 Out[5]:1.2261774609001463e-06 Let'sexperimentwithotherstartingvalues.Although p 0 =2 isnotintheintervalofconvergence I ,weexpectconvergencesince g 0 =0 : In[6]:fixedptlambdax:x-x**2-2/4,2,1e-5,15 pis1.414214788550556andtheiterationnumberis10 Let'stry p 0 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(5 .Notethatthisisnotonlyoutsidetheintervalofconvergence I ,but g 0 )]TJ/F81 10.9091 Tf 8.485 0 Td [(5= 3 : 5 > 1 ,sowedonotexpectconvergence. In[7]:fixedptlambdax:x-x**2-2/4,-5,1e-5,15 --------------------------------------------------------------------------OverflowErrorTracebackmostrecentcalllast in ---->1fixedptlambdax:x-x**2-2/4,-5,1e-5,15 infixedptg,pzero,eps,N 2n=1 3whilen4pone=gpzero 5ifnp.abspone-pzero
PAGE 72

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 71 inx ---->1fixedptlambdax:x-x**2-2/4,-5,1e-5,15 OverflowError:, ' Resulttoolarge ' Let'sverifythelinearconvergenceofthexed-pointiterationnumericallyinthisexample.We writeanotherversionofthexed-pointcode, xedpt2 ,andwecompute p n )]TJ 6.587 6.598 Td [(p 2 p n )]TJ/F47 5.9776 Tf 5.756 0 Td [(1 )]TJ 6.586 6.598 Td [(p 2 foreach n . In[1]:importnumpyasnp importmatplotlib.pyplotasplt %matplotlibinline In[2]:deffixedpt2g,pzero,eps,N: n=1 arr=np.array[] error=1. whilen1e-5: pone=gpzero error=np.abspone-pzero arr=np.appendarr,pone-2**0.5/pzero-2**0.5 pzero=pone n+=1 returnarr In[3]:arr=fixedpt2lambdax:x-x**2-2/4,1.5,1e-7,15 arr Out[3]:array[0.27144661,0.28707161,0.291222,0.29240652,0.29275091, 0.29285156,0.29288102,0.29288965,0.29289217] In[4]:plt.plotarr;

PAGE 73

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 72 Figure2.6:Fixed-pointiteration Thegraphsuggeststhelimitof p n )]TJ 6.586 6.598 Td [(p 2 p n )]TJ/F47 5.9776 Tf 5.756 0 Td [(1 )]TJ 6.587 6.598 Td [(p 2 existsanditisaround0.295,supportinglinearconvergence. 2.7High-orderxed-pointiteration IntheproofofTheorem44,weshowed lim n !1 j p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j = j g 0 p j whichimpliedthatthexed-pointiterationhaslinearconvergence,if g 0 p 6 =0 . Ifthislimitwerezero,thenwewouldhave lim n !1 j p n +1 )]TJ/F82 10.9091 Tf 10.91 0 Td [(p j j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j =0 ; whichmeansthedenominatorisgrowingatalargerratethanthenumerator.Wecouldthenaskif lim n !1 j p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j = nonzeroconstant forsome > 1 :

PAGE 74

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 73 Theorem46. Assume p isasolutionof g x = x where g 2 C I forsomeinterval I thatcontains p ,andforsome 2 : Furthermoreassume g 0 p = g 00 p = ::: = g )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 p =0 ; and g p 6 =0 : Theniftheinitialguess p 0 issucientlycloseto p ,thexed-pointiteration p n = g p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 ;n 1 ; willhaveorderofconvergenceof ; and lim n !1 p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p = g p ! : Proof. FromTaylor'stheorem, p n +1 = g p n = g p + p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p g 0 p + ::: + p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1! g )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 p + p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p ! g n where n isanumberbetween p n and p; andallnumbersarein I .Fromthehypothesis,this simpliesas p n +1 = p + p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p ! g n p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p = g n ! : FromTheorem44,if p 0 ischosensucientlycloseto p; then lim n !1 p n = p .Theorderofconvergenceis with lim n !1 j p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j j p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p j =lim n !1 j g n j ! = j g p j ! 6 =0 : ApplicationtoNewton'sMethod RecallNewton'siteration p n = p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 )]TJ/F82 10.9091 Tf 13.502 7.38 Td [(f p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 f 0 p n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 : Put g x = x )]TJ/F49 7.9701 Tf 13.447 5.375 Td [(f x f 0 x : Thenthexed-pointiteration p n = g p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 isNewton'smethod.Wehave g 0 x =1 )]TJ/F81 10.9091 Tf 12.104 7.38 Td [([ f 0 x ] 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x f 00 x [ f 0 x ] 2 = f x f 00 x [ f 0 x ] 2 andthus g 0 p = f p f 00 p [ f 0 p ] 2 =0 : Similarly, g 00 x = f 0 x f 00 x + f x f 000 x f 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x f 00 x f 0 x f 00 x [ f 0 x ] 4

PAGE 75

CHAPTER2.SOLUTIONSOFEQUATIONS:ROOT-FINDING 74 whichimplies g 00 p = f 0 p f 00 p f 0 p 2 [ f 0 p ] 4 = f 00 p f 0 p : If f 00 p 6 =0 ; thenTheorem46impliesNewton'smethodhasquadraticconvergencewith lim n !1 p n +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(p p n )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 2 = f 00 p 2 f 0 p whichwasprovedearlierinTheorem32. Exercise2.7-1: UseTheorem38andRemark39toshowthat g x =3 )]TJ/F49 7.9701 Tf 6.586 0 Td [(x hasaunique xed-pointon [1 = 4 ; 1] .UseCorollary42,part,tondthenumberofiterationsnecessaryto achieve 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(5 accuracy.ThenusethePythoncodetoobtainanapproximation,andcomparethe errorwiththetheoreticalestimateobtainedfromCorollary42. Exercise2.7-2: Let g x =2 x )]TJ/F82 10.9091 Tf 11.93 0 Td [(cx 2 where c isapositiveconstant.Provethatifthe xed-pointiteration p n = g p n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 convergestoanon-zerolimit,thenthelimitis 1 =c .

PAGE 76

Chapter3 Interpolation Inthischapter,wewillstudythefollowingproblem:givendata x i ;y i ;i =0 ; 1 ;:::;n ,nda function f suchthat f x i = y i : Thisproblemiscalledtheinterpolationproblem,and f iscalled theinterpolatingfunction,orinterpolant,forthegivendata. Interpolationisused,forexample,whenweusemathematicalsoftwaretoplotasmoothcurve throughdiscretedatapoints,whenwewanttondthein-betweenvaluesinatable,orwhenwe dierentiateorintegrateblack-boxtypefunctions. Howdowechoose f ?Or,whatkindoffunctiondowewant f tobe?Thereareseveraloptions. Examplesoffunctionsusedininterpolationarepolynomials,piecewisepolynomials,rationalfunctions,trigonometricfunctions,andexponentialfunctions.Aswetrytondagoodchoicefor f for ourdata,somequestionstoconsiderarewhetherwewant f toinheritthepropertiesofthedata forexample,ifthedataisperiodic,shouldweuseatrigonometricfunctionas f ?,andhowwe want f behavebetweendatapoints.Ingeneral f shouldbeeasytoevaluate,andeasytointegrate &dierentiate. Hereisageneralframeworkfortheinterpolationproblem.Wearegivendata,andwepicka familyoffunctionsfromwhichtheinterpolant f willbechosen: Data: x i ;y i ;i =0 ; 1 ;:::;n Family:Polynomials,trigonometricfunctions,etc. Supposethefamilyoffunctionsselectedformsavectorspace.Pickabasisforthevectorspace: 0 x ; 1 x ;:::; n x : Thentheinterpolatingfunctioncanbewrittenasalinearcombinationof thebasisvectorsfunctions: f x = n X k =0 a k k x : Wewant f topassthroughthedatapoints,thatis, f x i = y i .Thendetermine a k sothat: f x i = n X k =0 a k k x i = y i ;i =0 ; 1 ;:::;n; 75

PAGE 77

CHAPTER3.INTERPOLATION 76 whichisasystemof n +1 equationswith n +1 unknowns.Usingmatrices,theproblemistosolve thematrixequation Aa = y for a ,where A = 2 6 6 6 6 6 4 0 x 0 ::: n x 0 0 x 1 ::: n x 1 . . . 0 x n ::: n x n 3 7 7 7 7 7 5 ;a = 2 6 6 6 6 6 4 a 0 a 1 . . . a n 3 7 7 7 7 7 5 ;y = 2 6 6 6 6 6 4 y 0 y 1 . . . y n 3 7 7 7 7 7 5 : 3.1Polynomialinterpolation Inpolynomialinterpolation,wepickpolynomialsasthefamilyoffunctionsintheinterpolation problem. Data: x i ;y i ;i =0 ; 1 ;:::;n Family:Polynomials Thespaceofpolynomialsuptodegree n isavectorspace.Wewillconsiderthreechoicesfor thebasisforthisvectorspace: Basis: Monomialbasis: k x = x k Lagrangebasis: k x = Q n j =0 ;j 6 = k x )]TJ/F49 7.9701 Tf 6.587 0 Td [(x j x k )]TJ/F49 7.9701 Tf 6.587 0 Td [(x j Newtonbasis: k x = Q k )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 j =0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j where k =0 ; 1 ;:::;n . Oncewedecideonthebasis,theinterpolatingpolynomialcanbewrittenasalinearcombination ofthebasisfunctions: p n x = n X k =0 a k k x where p n x i = y i ;i =0 ; 1 ;:::;n: Hereisanimportantquestion.Howdoweknowthat p n ,apolynomialofdegreeatmost n passingthroughthedatapoints,actuallyexists?Or,equivalently,howdoweknowthesystemof equations p n x i = y i ;i =0 ; 1 ;:::;n ,hasasolution? Theanswerisgivenbythefollowingtheorem,whichwewillprovelaterinthissection. Theorem47. Ifpoints x 0 ;x 1 ;:::;x n aredistinct,thenforrealvalues y 0 ;y 1 ;:::;y n ; thereisaunique polynomial p n ofdegreeatmost n suchthat p n x i = y i ;i =0 ; 1 ;:::;n:

PAGE 78

CHAPTER3.INTERPOLATION 77 Wementionedthreefamiliesofbasisfunctionsforpolynomials.Thechoiceofafamilyofbasis functionsaects: Theaccuracyofthenumericalmethodstosolvethesystemoflinearequations Aa = y . Theeaseatwhichtheresultingpolynomialcanbeevaluated,dierentiated,integrated,etc. Monomialformofpolynomialinterpolation Givendata x i ;y i ;i =0 ; 1 ;:::;n ,weknowfromtheprevioustheoremthatthereexistsapolynomial p n x ofdegreeatmost n ,thatpassesthroughthedatapoints.Torepresent p n x ,wewillusethe monomialbasisfunctions, 1 ;x;x 2 ;:::;x n ,orwrittenmoresuccinctly, k x = x k ;k =0 ; 1 ;:::;n: Theinterpolatingpolynomial p n x canbewrittenasalinearcombinationofthesebasisfunctions as p n x = a 0 + a 1 x + a 2 x 2 + ::: + a n x n : Wewilldetermine a i usingthefactthat p n isaninterpolantforthedata: p n x i = a 0 + a 1 x i + a 2 x 2 i + ::: + a n x n i = y i for i =0 ; 1 ;:::;n: Or,inmatrixform,wewanttosolve 2 6 6 6 6 6 4 1 x 0 x 2 0 :::x n 0 1 x 1 x 2 1 x n 1 . . . 1 x n x 2 n x n n 3 7 7 7 7 7 5 | {z } A 2 6 6 6 6 6 4 a 0 a 1 . . . a n 3 7 7 7 7 7 5 | {z } a = 2 6 6 6 6 6 4 y 0 y 1 . . . y n 3 7 7 7 7 7 5 | {z } y for [ a 0 ;:::;a n ] T where [ ] T standsforthetransposeofthevector.Thecoecientmatrix A isknown asthevanderMondematrix.Thisisusuallyanill-conditionedmatrix,whichmeanssolvingthe systemofequationscouldresultinlargeerrorinthecoecients a i : Anintuitivewaytounderstand theill-conditioningistoplotseveralbasismonomials,andnotehowlessdistinguishabletheyareas thedegreeincreases,makingthecolumnsofthematrixnearlylinearlydependent.

PAGE 79

CHAPTER3.INTERPOLATION 78 Figure3.1:Monomialbasisfunctions Solvingthematrixequation Aa = b couldalsobeexpensive.UsingGaussianeliminationto solvethematrixequationforageneralmatrix A requires O n 3 operations.Thismeansthenumber ofoperationsgrowslike Cn 3 ,where C isapositiveconstant. 1 However,therearesomeadvantages tothemonomialform:evaluatingthepolynomialisveryecientusingHorner'smethod,which isthenestedformdiscussedinExercises1.3-4,1.3-5ofChapter1,requiring O n operations. Dierentiationandintegrationarealsorelativelyecient. Lagrangeformofpolynomialinterpolation Theill-conditioningofthevanderMondematrix,aswellasthehighcomplexityofsolvingthe resultingmatrixequationinthemonomialformofpolynomialinterpolation,motivateustoexplore otherbasisfunctionsforpolynomials.Asbefore,westartwithdata x i ;y i ;i =0 ; 1 ;:::;n ,andcall ourinterpolatingpolynomialofdegreeatmost n , p n x .TheLagrangebasisfunctionsuptodegree n alsocalledcardinalpolynomialsaredenedas l k x = n Y j =0 ;j 6 = k x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x k )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j ;k =0 ; 1 ;:::;n: Wewritetheinterpolatingpolynomial p n x asalinearcombinationofthesebasisfunctionsas p n x = a 0 l 0 x + a 1 l 1 x + ::: + a n l n x : 1 TheformaldenitionofthebigOnotationisasfollows:Wewrite f n = O g n as n !1 ifand onlyifthereexistsapositiveconstant M andapositiveinteger n suchthat j f n j Mg n forall n n .

PAGE 80

CHAPTER3.INTERPOLATION 79 Wewilldetermine a i from p n x i = a 0 l 0 x i + a 1 l 1 x i + ::: + a n l n x i = y i for i =0 ; 1 ;:::;n: Or,inmatrixform,wewanttosolve 2 6 6 6 6 6 4 l 0 x 0 l 1 x 0 :::l n x 0 l 0 x 1 l 1 x 1 l n x 1 . . . l 0 x n l 1 x n l n x n 3 7 7 7 7 7 5 | {z } A 2 6 6 6 6 6 4 a 0 a 1 . . . a n 3 7 7 7 7 7 5 | {z } a = 2 6 6 6 6 6 4 y 0 y 1 . . . y n 3 7 7 7 7 7 5 | {z } y for [ a 0 ;:::;a n ] T : Solvingthismatrixequationistrivialforthefollowingreason.Observethat l k x k =1 and l k x i =0 forall i 6 = k: Thenthecoecientmatrix A becomestheidentitymatrix,and a k = y k for k =0 ; 1 ;:::;n: Theinterpolatingpolynomialbecomes p n x = y 0 l 0 x + y 1 l 1 x + ::: + y n l n x : ThemainadvantageoftheLagrangeformofinterpolationisthatndingtheinterpolatingpolynomialistrivial:thereisnoneedtosolveamatrixequation.However,theevaluation,dierentiation, andintegrationoftheLagrangeformofapolynomialismoreexpensivethan,forexample,the monomialform. Example48. FindtheinterpolatingpolynomialusingthemonomialbasisandLagrangebasis functionsforthedata: )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 ; ; 0 ; ; 6 : Monomialbasis: p 2 x = a 0 + a 1 x + a 2 x 2 2 6 4 1 x 0 x 2 0 1 x 1 x 2 1 1 x 2 x 2 2 3 7 5 | {z } A 2 6 4 a 0 a 1 a 2 3 7 5 | {z } a = 2 6 4 y 0 y 1 y 2 3 7 5 | {z } y 2 6 4 1 )]TJ/F81 10.9091 Tf 8.485 0 Td [(11 111 124 3 7 5 2 6 4 a 0 a 1 a 2 3 7 5 = 2 6 4 )]TJ/F81 10.9091 Tf 8.484 0 Td [(6 0 6 3 7 5 WecanuseGaussianeliminationtosolvethismatrixequation,orgethelpfromPython: In[1]:importnumpyasnp In[2]:A=np.array[[1,-1,1],[1,1,1],[1,2,4]] A

PAGE 81

CHAPTER3.INTERPOLATION 80 Out[2]:array[[1,-1,1], [1,1,1], [1,2,4]] In[3]:y=np.array[-6,0,6] y Out[3]:array[-6,0,6] In[4]:np.linalg.solveA,y Out[4]:array[-4.,3.,1.] Sincethesolutionis a =[ )]TJ/F81 10.9091 Tf 8.485 0 Td [(4 ; 3 ; 1] T ; weobtain p 2 x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(4+3 x + x 2 : Lagrangebasis: p 2 x = y 0 l 0 x + y 1 l 1 x + y 2 l 2 x = )]TJ/F81 10.9091 Tf 8.484 0 Td [(6 l 0 x +0 l 1 x +6 l 2 x where l 0 x = x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 = x )]TJ/F81 10.9091 Tf 10.91 0 Td [(1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 = x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 6 l 2 x = x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 = x +1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 +1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 = x +1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 3 therefore p 2 x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 6 +6 x +1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 3 = )]TJ/F81 10.9091 Tf 8.485 0 Td [( x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2+2 x +1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : Ifwemultiplyoutandcollecttheliketerms,weobtain p 2 x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(4+3 x + x 2 ,whichisthe polynomialweobtainedfromthemonomialbasisearlier. Exercise3.1-1: Provethat P n k =0 l k x =1 forall x; where l k aretheLagrangebasisfunctionsfor n +1 datapoints.Hint.Firstverifytheidentityfor n =1 algebraically,foranytwo datapoints.Forthegeneralcase,thinkaboutwhatspecialfunction'sinterpolatingpolynomialin Lagrangeformis P n k =0 l k x .

PAGE 82

CHAPTER3.INTERPOLATION 81 Newton'sformofpolynomialinterpolation TheNewtonbasisfunctionsuptodegree n are k x = k )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 Y j =0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j ;k =0 ; 1 ;:::;n where 0 x = Q )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 j =0 x )]TJ/F82 10.9091 Tf 11.55 0 Td [(x j isinterpretedas1.Theinterpolatingpolynomial p n ,writtenasa linearcombinationofNewtonbasisfunctions,is p n x = a 0 0 x + a 1 1 x + ::: + a n n x = a 0 + a 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 + a 2 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 + ::: + a n x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 : Wewilldetermine a i from p n x i = a 0 + a 1 x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 + ::: + a n x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 = y i ; for i =0 ; 1 ;:::;n; orinmatrixform 2 6 6 6 6 6 6 6 4 100 ::: 0 1 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 00 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 0 . . . . . . . . . . . . 1 x n )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x n )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x n )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 ::: Q n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 i =0 x n )]TJ/F82 10.9091 Tf 10.909 0 Td [(x i 3 7 7 7 7 7 7 7 5 | {z } A 2 6 6 6 6 6 4 a 0 a 1 . . . a n 3 7 7 7 7 7 5 | {z } a = 2 6 6 6 6 6 4 y 0 y 1 . . . y n 3 7 7 7 7 7 5 | {z } y for [ a 0 ;:::;a n ] T : Notethatthecoecientmatrix A islower-triangular,and a canbesolvedby forwardsubstitution,whichisshowninthenextexample,in O n 2 operations. Example49. FindtheinterpolatingpolynomialusingNewton'sbasisforthedata: )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 ; ; 0 ; ; 6 : Solution. Wehave p 2 x = a 0 + a 1 1 x + a 2 2 x = a 0 + a 1 x +1+ a 2 x +1 x )]TJ/F81 10.9091 Tf 11.587 0 Td [(1 : Find a 0 ;a 1 ;a 2 from p 2 )]TJ/F81 10.9091 Tf 8.485 0 Td [(1= )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 a 0 + a 1 )]TJ/F81 10.9091 Tf 8.485 0 Td [(1+1+ a 2 )]TJ/F81 10.9091 Tf 8.485 0 Td [(1+1 )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1= a 0 = )]TJ/F81 10.9091 Tf 8.484 0 Td [(6 p 2 =0 a 0 + a 1 +1+ a 2 +1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1= a 0 +2 a 1 =0 p 2 =6 a 0 + a 1 +1+ a 2 +1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1= a 0 +3 a 1 +3 a 2 =6

PAGE 83

CHAPTER3.INTERPOLATION 82 or,inmatrixform 2 6 4 100 120 133 3 7 5 2 6 4 a 0 a 1 a 2 3 7 5 = 2 6 4 )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 0 6 3 7 5 : Forwardsubstitutionis: a 0 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 a 0 +2 a 1 =0 )]TJ/F81 10.9091 Tf 22.424 0 Td [(6+2 a 1 =0 a 1 =3 a 0 +3 a 1 +3 a 2 =6 )]TJ/F81 10.9091 Tf 22.424 0 Td [(6+9+3 a 2 =6 a 2 =1 : Therefore a =[ )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 ; 3 ; 1] T and p 2 x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(6+3 x +1+ x +1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : Factoringoutandsimplifyinggives p 2 x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(4+3 x + x 2 ,whichisthepolynomialdiscussedin Example48. Summary: Theinterpolatingpolynomial p 2 x forthedata, )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 ; ; 0 ; ; 6 ,representedin threedierentbasisfunctionsis: Monomial: p 2 x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(4+3 x + x 2 Lagrange: p 2 x = )]TJ/F81 10.9091 Tf 10.909 0 Td [( x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2+2 x +1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 Newton: p 2 x = )]TJ/F81 10.9091 Tf 10.909 0 Td [(6+3 x +1+ x +1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 Similartothemonomialform,apolynomialwritteninNewton'sformcanbeevaluatedusing theHorner'smethodwhichhas O n complexity: p n x = a 0 + a 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 + a 2 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 + ::: + a n x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 = a 0 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 a 1 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 a 2 + ::: + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 a n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 a n Example50. Write p 2 x = )]TJ/F81 10.9091 Tf 8.484 0 Td [(6+3 x +1+ x +1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 usingthenestedform. Solution. )]TJ/F81 10.9091 Tf 8.485 0 Td [(6+3 x +1+ x +1 x )]TJ/F81 10.9091 Tf 10.585 0 Td [(1= )]TJ/F81 10.9091 Tf 8.485 0 Td [(6+ x +12+ x ; notethattheleft-handsidehas2 multiplications,andtheright-handsidehas1. Complexityofthethreeformsofpolynomialinterpolation: Thenumberofmultiplicationsrequiredinsolvingthecorrespondingmatrixequationineachpolynomialbasisis: Monomial ! O n 3 Lagrange ! trivial

PAGE 84

CHAPTER3.INTERPOLATION 83 Newton ! O n 2 EvaluatingthepolynomialscanbedoneecientlyusingHorner'smethodformonomialand Newtonforms.AmodiedversionofLagrangeformcanalsobeevaluatedusingHorner's method,butwedonotdiscussithere. Exercise3.1-2: Compute,byhand,theinterpolatingpolynomialtothedata )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 0 , : 5 ; 1 , ; 0 usingthemonomial,Lagrange,andNewtonbasisfunctions.Verifythethreepolynomialsare identical. It'stimetodiscusssometheoreticalresultsforpolynomialinterpolation.Let'sstartwithproving Theorem47whichwestatedearlier: Theorem. Ifpoints x 0 ;x 1 ;:::;x n aredistinct,thenforrealvalues y 0 ;y 1 ;:::;y n ; thereisaunique polynomial p n ofdegreeatmost n suchthat p n x i = y i ;i =0 ; 1 ;:::;n: Proof. Wehavealreadyestablishedtheexistenceof p n withoutmentioningit!TheLagrangeform oftheinterpolatingpolynomialconstructs p n directly: p n x = n X k =0 y k l k x = n X k =0 y k n Y j =0 ;j 6 = k x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x k )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j : Let'sproveuniqueness.Assume p n ;q n aretwodistinctpolynomialssatisfyingtheconclusion.Then p n )]TJ/F82 10.9091 Tf 11.494 0 Td [(q n isapolynomialofdegreeatmost n suchthat p n )]TJ/F82 10.9091 Tf 11.495 0 Td [(q n x i =0 for i =0 ; 1 ;:::;n: This meansthenon-zeropolynomial p n )]TJ/F82 10.9091 Tf 10.57 0 Td [(q n ofdegreeatmost n; has n +1 distinctroots,whichisa contradiction. Thefollowingtheorem,whichwestatewithoutproof,establishestheerrorofpolynomialinterpolation.NoticethesimilaritiesbetweenthisandTaylor'sTheorem7. Theorem51. Let x 0 ;x 1 ;:::;x n bedistinctnumbersintheinterval [ a;b ] and f 2 C n +1 [ a;b ] : Then foreach x 2 [ a;b ] ; thereisanumber between x 0 ;x 1 ;:::;x n suchthat f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n x = f n +1 n +1! x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n : Thefollowinglemmaisusefulinndingupperboundsfor j f x )]TJ/F82 10.9091 Tf 11.521 0 Td [(p n x j usingTheorem51, whenthenodes x 0 ;:::;x n are equallyspaced .

PAGE 85

CHAPTER3.INTERPOLATION 84 Lemma52. Considerthepartitionof [ a;b ] as x 0 = a;x 1 = a + h;:::;x n = a + nh = b .More succinctly, x i = a + ih for i =0 ; 1 ;:::;n and h = b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a n : Thenforany x 2 [ a;b ] n Y i =0 j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x i j 1 4 h n +1 n ! Proof. Since x 2 [ a;b ] ,itfallsintooneofthesubintervals:let x 2 [ x j ;x j +1 ] : Considertheproduct j x )]TJ/F82 10.9091 Tf 10.984 0 Td [(x j jj x )]TJ/F82 10.9091 Tf 10.984 0 Td [(x j +1 j : Put s = j x )]TJ/F82 10.9091 Tf 10.983 0 Td [(x j j and t = j x )]TJ/F82 10.9091 Tf 10.984 0 Td [(x j +1 j : Themaximumof st given s + t = h; using Calculus,canbefoundtobe h 2 = 4 ; whichisattainedwhen x isthemidpoint,andthus s = t = h= 2 . Then n Y i =0 j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x i j = j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 jj x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 jj x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j jj x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j +1 jj x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j +2 jj x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n j j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 jj x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 j h 2 4 j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j +2 jj x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n j j x j +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 jj x j +1 )]TJ/F82 10.9091 Tf 10.91 0 Td [(x j )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 j h 2 4 j x j )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j +2 jj x j )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n j j +1 h 2 h h 2 4 h n )]TJ/F82 10.9091 Tf 10.909 0 Td [(j h = h j j +1! h 2 4 n )]TJ/F82 10.9091 Tf 10.909 0 Td [(j ! h n )]TJ/F49 7.9701 Tf 6.587 0 Td [(j )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 h n +1 n ! 4 : Example53. Findanupperboundfortheabsoluteerrorwhen f x =cos x isapproximatedbyits interpolatingpolynomial p n x on [0 ;= 2] .Fortheinterpolatingpolynomial,use5equallyspaced nodes n =4 in [0 ;= 2] ,includingtheendpoints. Solution. FromTheorem51, j f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 4 x j = j f j 5! j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 4 j : Wehave j f j 1 : Thenodesareequallyspacedwith h = = 2 )]TJ/F81 10.9091 Tf 11.133 0 Td [(0 = 4= = 8 : Thenfromthe previouslemma, j x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 4 j 1 4 8 5 4! andtherefore j f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p 4 x j 1 5! 1 4 8 5 4!=4 : 7 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(4 :

PAGE 86

CHAPTER3.INTERPOLATION 85 Exercise3.1-3: Findanupperboundfortheabsoluteerrorwhen f x =ln x isapproximatedbyaninterpolatingpolynomialofdegreevewithsixnodesequallyspacedintheinterval [1 ; 2] : WenowrevisitNewton'sformofinterpolation,andlearnanalternativemethod,knownas divideddierences ,tocomputethecoecientsoftheinterpolatingpolynomial.Thisapproach isnumericallymorestablethantheforwardsubstitutionapproachweusedearlier.Let'srecallthe interpolationproblem. Data: x i ;y i ;i =0 ; 1 ;:::;n InterpolantinNewton'sform: p n x = a 0 + a 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 + a 2 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 + ::: + a n x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 Determine a i from p n x i = y i ;i =0 ; 1 ;:::;n . Let'sthinkofthe y -coordinatesofthedata, y i ,asvaluesofanunknownfunction f evaluated at x i ,i.e., f x i = y i : Substitute x = x 0 intheinterpolanttoget: a 0 = f x 0 : Substitute x = x 1 toget a 0 + a 1 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 = f x 1 or, a 1 = f x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 x 1 )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 0 : Substitute x = x 2 toget,aftersomealgebra a 2 = f x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(f x 0 x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 1 )]TJ/F49 7.9701 Tf 12.105 5.375 Td [(f x 1 )]TJ/F49 7.9701 Tf 6.586 0 Td [(f x 0 x 1 )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 0 x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 0 x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 whichcanbefurtherrewrittenas a 2 = f x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(f x 1 x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 1 )]TJ/F49 7.9701 Tf 12.105 5.375 Td [(f x 1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(f x 0 x 1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 0 : Inspectingtheformulasfor a 0 ;a 1 ;a 2 suggeststhefollowingsimpliednewnotationcalled divided dierences : a 0 = f x 0 = f [ x 0 ] )167(! 0thdivideddierence a 1 = f x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 = f [ x 0 ;x 1 ] )167(! 1stdivideddierence a 2 = f x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(f x 1 x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 1 )]TJ/F49 7.9701 Tf 12.104 5.374 Td [(f x 1 )]TJ/F49 7.9701 Tf 6.586 0 Td [(f x 0 x 1 )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 = f [ x 1 ;x 2 ] )]TJ/F82 10.9091 Tf 10.909 0 Td [(f [ x 0 ;x 1 ] x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 = f [ x 0 ;x 1 ;x 2 ] )167(! 2nddivideddierence

PAGE 87

CHAPTER3.INTERPOLATION 86 Andingeneral, a k willbegivenbythe k thdivideddierence: a k = f [ x 0 ;x 1 ;:::;x k ] : Withthisnewnotation,Newton'sinterpolatingpolynomialcanbewrittenas p n x = f [ x 0 ]+ n X k =1 f [ x 0 ;x 1 ;:::;x k ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x k )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 = f [ x 0 ]+ f [ x 0 ;x 1 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 + f [ x 0 ;x 1 ;x 2 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 + ::: + f [ x 0 ;x 1 ;:::;x n ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 Hereistheformaldenitionofdivideddierences: Denition54. Givendata x i ;f x i ;i =0 ; 1 ;:::;n; thedivideddierencesaredenedrecursively as f [ x 0 ]= f x 0 f [ x 0 ;x 1 ;:::;x k ]= f [ x 1 ;:::;x k ] )]TJ/F82 10.9091 Tf 10.909 0 Td [(f [ x 0 ;:::;x k )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ] x k )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 0 where k =0 ; 1 ;:::;n: Theorem55. Theorderingofthedatainconstructingdivideddierencesisnotimportant,thatis, thedivideddierence f [ x 0 ;:::;x k ] isinvariantunderallpermutationsofthearguments x 0 ;:::;x k : Proof. Considerthedata x 0 ;y 0 ; x 1 ;y 1 ;:::; x k ;y k andlet p k x beitsinterpolatingpolynomial: p k x = f [ x 0 ]+ f [ x 0 ;x 1 ] x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 0 + f [ x 0 ;x 1 ;x 2 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 + ::: + f [ x 0 ;:::;x k ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x k )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 : Nowlet'sconsiderapermutationofthe x i ; let'slabelthemas ~ x 0 ; ~ x 1 ;:::; ~ x k : Theinterpolating polynomialforthepermuteddatadoesnotchange,sincethedata x 0 ;x 1 ;:::;x k omittingthe y coordinatesisthesameas ~ x 0 ; ~ x 1 ;:::; ~ x k ,justindierentorder.Therefore p k x = f [~ x 0 ]+ f [~ x 0 ; ~ x 1 ] x )]TJ/F81 10.9091 Tf 11.602 0 Td [(~ x 0 + f [~ x 0 ; ~ x 1 ; ~ x 2 ] x )]TJ/F81 10.9091 Tf 11.602 0 Td [(~ x 0 x )]TJ/F81 10.9091 Tf 11.602 0 Td [(~ x 1 + ::: + f [~ x 0 ;:::; ~ x k ] x )]TJ/F81 10.9091 Tf 11.602 0 Td [(~ x 0 x )]TJ/F81 10.9091 Tf 11.603 0 Td [(~ x k )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 : Thecoecientofthepolynomial p k x forthehighestdegree x k is f [ x 0 ;:::;x k ] intherstequation, and f [~ x 0 ;:::; ~ x k ] inthesecond.Thereforetheymustbeequaltoeachother. Example56. Findtheinterpolatingpolynomialforthedata )]TJ/F81 10.9091 Tf 8.484 0 Td [(1 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 ; ; 0 ; ; 6 usingNewton's formanddivideddierences.

PAGE 88

CHAPTER3.INTERPOLATION 87 Solution. Wewanttocompute p 2 x = f [ x 0 ]+ f [ x 0 ;x 1 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 + f [ x 0 ;x 1 ;x 2 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 1 : Herearethenitedierences: xf [ x ] FirstdivideddierenceSeconddivideddierence x 0 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 f [ x 0 ]= )]TJ/F81 10.9091 Tf 8.485 0 Td [(6 f [ x 0 ;x 1 ]= f [ x 1 ] )]TJ/F49 7.9701 Tf 6.587 0 Td [(f [ x 0 ] x 1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 0 =3 x 1 =1 f [ x 1 ]=0 f [ x 0 ;x 1 ;x 2 ]= f [ x 1 ;x 2 ] )]TJ/F49 7.9701 Tf 6.587 0 Td [(f [ x 0 ;x 1 ] x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 0 =1 f [ x 1 ;x 2 ]= f [ x 2 ] )]TJ/F49 7.9701 Tf 6.587 0 Td [(f [ x 1 ] x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 1 =6 x 2 =2 f [ x 2 ]=6 Therefore p 2 x = )]TJ/F81 10.9091 Tf 8.485 0 Td [(6+3 x +1+1 x +1 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 ; whichisthesamepolynomialwehadinExample49. Exercise3.1-4: Considerthefunction f giveninthefollowingtable. x 1246 f x 2359 aConstructadivideddierencetablefor f byhand,andwritetheNewtonformoftheinterpolatingpolynomialusingthedivideddierences. bAssumeyouaregivenanewdatapointforthefunction: x =3 ;y =4 : Findthenew interpolatingpolynomial.Hint:Thinkabouthowtoupdatetheinterpolatingpolynomial youfoundinparta. cIfyouwereworkingwiththeLagrangeformoftheinterpolatingpolynomialinsteadofthe Newtonform,andyouweregivenanadditionaldatapointlikeinpartb,howeasywould itbecomparedtowhatyoudidinpartbtoupdateyourinterpolatingpolynomial? Example57. Beforethewidespreadavailabilityofcomputersandmathematicalsoftware,the valuesofsomeoften-usedmathematicalfunctionsweredisseminatedtoresearchersandengineers viatables.Thefollowingtable,takenfrom[1],displayssomevaluesofthegammafunction, \050 x = R 1 0 t x )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 e )]TJ/F49 7.9701 Tf 6.587 0 Td [(t dt .

PAGE 89

CHAPTER3.INTERPOLATION 88 x 1.7501.7551.7601.765 \050 x 0.919060.920210.921370.92256 Usepolynomialinterpolationtoestimate \0501 : 761 . Solution. Thenitedierences,withve-digitrounding,are: ix i f [ x i ] f [ x i ;x i +1 ] f [ x i )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ;x i ;x i +1 ] f [ x 0 ;x 1 ;x 2 ;x 3 ] 01.7500.91906 0.23 11.7550.920210.2 0.23226.667 21.7600.921370.6 0.238 31.7650.92256 Herearevariousestimatesfor \0501 : 761 usinginterpolatingpolynomialsofincreasingdegrees: p 1 x = f [ x 0 ]+ f [ x 0 ;x 1 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 p 1 : 761=0 : 91906+0 : 23 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 750=0 : 92159 p 2 x = f [ x 0 ]+ f [ x 0 ;x 1 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 + f [ x 0 ;x 1 ;x 2 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 1 p 2 : 761=0 : 92159+ : 2 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 750 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 755=0 : 9216 p 3 x = p 2 x + f [ x 0 ;x 1 ;x 2 ;x 3 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 p 3 : 761=0 : 9216+26 : 667 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 750 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 755 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 760=0 : 9216 Nextwewillchangetheorderingofthedataandrepeatthecalculations.Wewilllistthedata indecreasingorderofthe x -coordinates: ix i f [ x i ] f [ x i ;x i +1 ] f [ x i )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ;x i ;x i +1 ] f [ x 0 ;x 1 ;x 2 ;x 3 ] 01.7650.92256 0.238 11.7600.921370.6 0.23226.667 21.7550.920210.2 0.23 31.7500.91906

PAGE 90

CHAPTER3.INTERPOLATION 89 Thepolynomialevaluationsare: p 1 : 761=0 : 92256+0 : 238 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 765=0 : 92161 p 2 : 761=0 : 92161+0 : 6 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 765 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 760=0 : 92161 p 3 : 761=0 : 92161+26 : 667 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 765 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 760 : 761 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 755=0 : 92161 Summaryofresults: Thefollowingtabledisplaystheresultsforeachorderingofthedata, togetherwiththecorrect \0501 : 761 to7digitsofaccuracy. Ordering : 75 ; 1 : 755 ; 1 : 76 ; 1 : 765 : 765 ; 1 : 76 ; 1 : 755 ; 1 : 75 p 1 : 761 0.921590.92161 p 2 : 761 0.921600.92161 p 3 : 761 0.921600.92161 \0501 : 761 0.92161030.9216103 Exercise3.1-5: Answerthefollowingquestions: aTheorem55statedthattheorderingofthedataindivideddierencesdoesnotmatter.But weseedierencesinthetwotablesabove.Isthisacontradiction? b p 1 : 761 isabetterapproximationto \0501 : 761 inthesecondordering.Isthisexpected? c p 3 : 761 isdierentinthetwoorderings,however,thisdierenceisduetoroundingerror. Inotherwords,ifthecalculationscanbedoneexactly, p 3 : 761 willbethesameineach orderingofthedata.Why? Exercise3.1-6: Considerafunction f x suchthat f =1 : 5713 , f =1 : 5719 ;f = 1 : 5738 ; and f =1 : 5751 : Estimate f usingaseconddegreeinterpolatingpolynomialinterpolatingtherstthreedatapointsandathirddegreeinterpolatingpolynomialinterpolatingthe rstfourdatapoints.Roundthenalresultstofourdecimalplaces.Isthereanyadvantagehere inusingathirddegreeinterpolatingpolynomial? PythoncodeforNewtoninterpolation Considerthefollowingnitedierencetable. Thereare 2+1=3 divideddierencesinthetable,notcountingthe0thdivideddierences. Ingeneral,thenumberofdivideddierencestocomputeis 1+ ::: + n = n n +1 = 2 .However,to constructNewton'sformoftheinterpolatingpolynomial,weneedonly n divideddierencesand the0thdivideddierence y 0 .ThesenumbersaredisplayedinredinTable3.1.Theimportant

PAGE 91

CHAPTER3.INTERPOLATION 90 xf x f [ x i ;x i +1 ] f [ x i )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ;x i ;x i +1 ] x 0 y 0 y 1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(y 0 x 1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 0 = f [ x 0 ;x 1 ] x 1 y 1 f [ x 1 ;x 2 ] )]TJ/F49 7.9701 Tf 6.586 0 Td [(f [ x 0 ;x 1 ] x 2 )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 0 = f [ x 0 ;x 1 ;x 2 ] y 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(y 1 x 2 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 1 = f [ x 1 ;x 2 ] x 2 y 2 Table3.1:Divideddierencesforthreedatapoints observationis,eventhoughallthedivideddierenceshavetobecomputedinordertogettheones neededforNewton'sform,theydonothavetobeallstored.ThefollowingPythoncodeisbasedon anecientalgorithmthatgoesthroughthedivideddierencecalculationsrecursively,andstores anarrayofsize m = n +1 atanygiventime.Inthenaliteration,thisarrayhasthedivided dierencesneededforNewton'sform. Let'sexplaintheideaofthealgorithmusingthesimpleexampleofTable3.1.Thecodecreates anarray a = a 0 ;a 1 ;a 2 ofsize m = n +1 ,whichisthreeinourexample,andsets a 0 = y 0 ;a 1 = y 1 ;a 2 = y 2 : Intherstiterationforloop, j =1 , a 1 and a 2 areupdated: a 1 := a 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 = y 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(y 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 = f [ x 0 ;x 1 ] ;a 2 := a 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 = y 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(y 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 : Inthelastiterationforloop, j =2 ,only a 2 isupdated: a 2 := a 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 = y 2 )]TJ/F49 7.9701 Tf 6.586 0 Td [(y 1 x 2 )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 1 )]TJ/F49 7.9701 Tf 12.686 4.931 Td [(y 1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(y 0 x 1 )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 = f [ x 0 ;x 1 ;x 2 ] : Thenalarray a is a = y 0 ;f [ x 0 ;x 1 ] ;f [ x 0 ;x 1 ;x 2 ] containingthedivideddierencesneededtoconstructtheNewton'sformofthepolynomial. HereisthePythonfunction di thatcomputesthedivideddierences.Thecodeusesafunction wehavenotusedbefore: np.ipnp.arangej,m .Anexampleillustrateswhatitdoesthebest: In[1]:importnumpyasnp In[2]:np.flipnp.arange2,6 Out[2]:array[5,4,3,2] Inthecode di theinputsarethe x -and y -coordinatesofthedata.Thenumberingofthe indicesstartsat0.

PAGE 92

CHAPTER3.INTERPOLATION 91 In[3]:defdiffx,y: m=x.size #heremisthenumberofdatapoints. #thedegreeofthepolynomialism-1 a=np.zerosm foriinrangem: a[i]=y[i] forjinrange1,m: foriinnp.flipnp.arangej,m: a[i]=a[i]-a[i-1]/x[i]-x[i-j] returna Let'scomputethedivideddierencesofExample56: In[4]:diffnp.array[-1,1,2],np.array[-6,0,6] Out[4]:array[-6.,3.,1.] ThesearethedivideddierencesinthesecondorderingofthedatainExample57: In[5]:diffnp.array[1.765,1.760,1.755,1.750], np.array[0.92256,0.92137,0.92021,0.91906] Out[5]:array[0.92256,0.238,0.6,26.66666667] Nowlet'swriteacodefortheNewtonformofpolynomialinterpolation.Theinputstothe function newton arethe x -and y -coordinatesofthedata,andwherewewanttoevaluatethe polynomial: z .Thecodeusesthedivideddierencesfunction di discussedearliertocompute: f [ x 0 ]+ f [ x 0 ;x 1 ] z )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 + ::: + f [ x 0 ;x 1 ;:::;x n ] z )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 z )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 z )]TJ/F82 10.9091 Tf 10.91 0 Td [(x n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 In[6]:defnewtonx,y,z: m=x.size #heremisthenumberofdatapoints,notthedegree #ofthepolynomial a=diffx,y sum=a[0] pr=1.0 forjinrangem-1: pr*=z-x[j] sum+=a[j+1]*pr returnsum Let'sverifythecodebycomputing p 3 : 761 ofExample57:

PAGE 93

CHAPTER3.INTERPOLATION 92 In[7]:newtonnp.array[1.765,1.760,1.755,1.750], np.array[0.92256,0.92137,0.92021,0.91906],1.761 Out[7]:0.92160496 Exercise3.1-7: Thisproblemdiscussesinverseinterpolationwhichgivesanothermethod tondtherootofafunction.Let f beacontinuousfunctionon [ a;b ] withoneroot p inthe interval.Alsoassume f hasaninverse.Let x 0 ;x 1 ;:::;x n be n +1 distinctnumbersin [ a;b ] with f x i = y i ;i =0 ; 1 ;:::;n: Constructaninterpolatingpolynomial P n for f )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 x ; bytakingyourdata pointsas y i ;x i ;i =0 ; 1 ;:::;n: Observethat f )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 = p; therootwearetryingtond.Then, approximatetheroot p ,byevaluatingtheinterpolatingpolynomialfor f )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 at0,i.e., P n p: Usingthismethod,andthefollowingdata,ndanapproximationtothesolutionof log x =0 : x 0.40.81.21.6 log x -0.92-0.220.180.47 3.2Highdegreepolynomialinterpolation Supposeweapproximate f x usingitspolynomialinterpolant p n x obtainedfrom n +1 data points.Wethenincreasethenumberofdatapoints,andupdate p n x accordingly.Thecentral questionwewanttodiscussisthefollowing:asthenumberofnodesdatapointsincreases,does p n x becomeabetterapproximationto f x on [ a;b ]? Wewillinvestigatethisquestionnumerically, usingafamousexample:Runge'sfunction,givenby f x = 1 1+ x 2 . WewillinterpolateRunge'sfunctionusingpolynomialsofvariousdegrees,andplotthefunction,togetherwithitsinterpolatingpolynomialandthedatapoints.Weareinterestedtosee whathappensasthenumberofdatapoints,andhencethedegreeoftheinterpolatingpolynomial, increases. In[8]:importmatplotlib.pyplotasplt %matplotlibinline Westartwithtakingfourequallyspaced x -coordinatesbetween-5and5,andplotthecorrespondinginterpolatingpolynomialandRunge'sfunction.Matplotliballowstypingmathematicsin captionsofaplotusingLatex.Latexisatypesettingprogramthisbookiswrittenwith.Latex commandsneedtobeenclosedbyapairofdollarsigns,inadditiontoapairofquotationmarks.

PAGE 94

CHAPTER3.INTERPOLATION 93 In[9]:f=lambdax:1/1+x**2 xi=np.linspace-5,5,4 #x-coordinatesofthedata,4pointsequallyspacedfrom #-5to5inincrementsof10/3 yi=fxi #thecorrespondingy-coordinates xaxis=np.linspace-5,5,1000 runge=fxaxis #Runge ' sfunctionvalues interp=newtonxi,yi,xaxis plt.plotxaxis,interp,label= ' interpolatingpoly ' plt.plotxaxis,runge,label="$fx=1/+x^2$" plt.plotxi,yi, ' o ' ,label= ' data ' plt.legendloc= ' upperright ' ; Next,weincreasethenumberofdatapointsto6. In[10]:xi=np.linspace-5,5,6 #6equallyspacedvaluesfrom-5to5 yi=fxi #thecorrespondingy-coordinates interp=newtonxi,yi,xaxis plt.plotxaxis,interp,label= ' interpolatingpoly ' plt.plotxaxis,runge,label="$fx=1/+x^2$" plt.plotxi,yi, ' o ' ,label= ' data ' plt.legendloc= ' upperright ' ;

PAGE 95

CHAPTER3.INTERPOLATION 94 Thenexttwographsplotinterpolatingpolynomialson11and21equallyspaceddata. In[11]:xi=np.linspace-5,5,11 #11equallyspacedvaluesfrom-5to5 yi=fxi #thecorrespondingy-coordinates interp=newtonxi,yi,xaxis plt.plotxaxis,interp,label= ' interpolatingpoly ' plt.plotxaxis,runge,label="$fx=1/+x^2$" plt.plotxi,yi, ' o ' ,label= ' data ' plt.legendloc= ' uppercenter ' ;

PAGE 96

CHAPTER3.INTERPOLATION 95 In[12]:xi=np.linspace-5,5,21 #21equallyspacedvaluesfrom-5to5 yi=fxi #thecorrespondingy-coordinates interp=newtonxi,yi,xaxis plt.plotxaxis,interp,label= ' interpolatingpoly ' plt.plotxaxis,runge,label="$fx=1/+x^2$" plt.plotxi,yi, ' o ' ,label= ' data ' plt.legendloc= ' lowercenter ' ; Weobservethatasthedegreeoftheinterpolatingpolynomialincreases,thepolynomialhas largeoscillationstowardtheendpointsoftheinterval.Infact,itcanbeshownthatforany x such that 3 : 64 < j x j < 5 ; sup n 0 j f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n x j = 1 ,where f isRunge'sfunction. Thistroublesomebehaviorofhighdegreeinterpolatingpolynomialsimprovessignicantly,ifwe considerdatawith x -coordinatesthatare not equallyspaced.Considertheinterpolationerrorof Theorem51: f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(p n x = f n +1 n +1! x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n : Perhapssurprisingly,theright-handsideoftheaboveequationisnotminimizedwhenthenodes, x i ,areequallyspaced!Thesetofnodesthatminimizestheinterpolationerroristherootsof theso-calledChebyshevpolynomials.Theplacingofthesenodesissuchthattherearemorenodes towardstheendpointsoftheinterval,thanthemiddle.WewilllearnaboutChebyshevpolynomials inChapter5.UsingChebyshevnodesinpolynomialinterpolationavoidsthedivergingbehavior

PAGE 97

CHAPTER3.INTERPOLATION 96 ofpolynomialinterpolantsasthedegreeincreases,asobservedinthecaseofRunge'sfunction,for sucientlysmoothfunctions. Divideddierencesandderivatives Thefollowingtheoremshowsthesimilaritybetweendivideddierencesandderivatives. Theorem58. Suppose f 2 C n [ a;b ] and x 0 ;x 1 ;:::;x n aredistinctnumbersin [ a;b ] : Thenthere exists 2 a;b suchthat f [ x 0 ;:::;x n ]= f n n ! : Toprovethistheorem,weneedthegeneralizedRolle'stheorem. Theorem59 Rolle'stheorem . Suppose f isadierentiablefunctionon a;b : If f a = f b ; thenthereexists c 2 a;b suchthat f 0 c =0 : Theorem60 GeneralizedRolle'stheorem . Suppose f has n derivativeson a;b : If f x =0 at n +1 distinctnumbers x 0 ;x 1 ;:::;x n 2 [ a;b ] ,thenthereexists c 2 a;b suchthat f n c =0 : ProofofTheorem58. Considerthefunction g x = p n x )]TJ/F82 10.9091 Tf 11.952 0 Td [(f x : Observethat g x i =0 for i =0 ; 1 ;:::;n: FromgeneralizedRolle'stheorem,thereexists 2 a;b suchthat g n =0 ; which implies p n n )]TJ/F82 10.9091 Tf 10.909 0 Td [(f n =0 : Since p n x = f [ x 0 ]+ f [ x 0 ;x 1 ] x )]TJ/F82 10.9091 Tf 11.028 0 Td [(x 0 + ::: + f [ x 0 ;:::;x n ] x )]TJ/F82 10.9091 Tf 11.028 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 11.028 0 Td [(x n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 , p n n x equals n ! timestheleadingcoecient f [ x 0 ;:::;x n ] : Therefore f n = n ! f [ x 0 ;:::;x n ] : 3.3Hermiteinterpolation Inpolynomialinterpolation,ourstartingpointhasbeenthe x -and y -coordinatesofsomedatawe wanttointerpolate.Suppose,inaddition,weknowthederivativeoftheunderlyingfunctionat these x -coordinates.Ournewdatasethasthefollowingform. Data: x 0 ;x 1 ;:::;x n y 0 ;y 1 ;:::;y n ; y i = f x i

PAGE 98

CHAPTER3.INTERPOLATION 97 y 0 0 ;y 0 1 ;:::;y 0 n ; y 0 i = f 0 x i Weseekapolynomialthattsthe y and y 0 values,thatis,weseekapolynomial H x suchthat H x i = y i and H 0 x i = y 0 i , i =0 ; 1 ;:::;n: Thismakes 2 n +2 equations,andifwelet H x = a 0 + a 1 x + ::: + a 2 n +1 x 2 n +1 ; thenthereare 2 n +2 unknowns, a 0 ;:::;a 2 n +1 ,tosolvefor.Thefollowingtheoremshowsthatthere isauniquesolutiontothissystemofequations;aproofcanbefoundinBurden,Faires,Burden[4]. Theorem61. If f 2 C 1 [ a;b ] and x 0 ;:::;x n 2 [ a;b ] aredistinct,thenthereisauniquepolynomial H 2 n +1 x ,ofdegreeatmost 2 n +1 ,agreeingwith f and f 0 at x 0 ;:::;x n .Thepolynomialcanbe writtenas: H 2 n +1 x = n X i =0 y i h i x + n X i =0 y 0 i e h i x where h i x = )]TJ/F81 10.9091 Tf 5 -8.837 Td [(1 )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x i l 0 i x i l i x 2 e h i x = x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x i l i x 2 : Here l i x isthe i thLagrangebasisfunctionforthenodes x 0 ;:::;x n ,and l 0 i x isitsderivative. H 2 n +1 x iscalledtheHermiteinterpolatingpolynomial. TheonlydierencebetweenHermiteinterpolationandpolynomialinterpolationisthatinthe former,wehavethederivativeinformation,whichcangoalongwayincapturingtheshapeofthe underlyingfunction. Example62. Wewanttointerpolatethefollowingdata: x -coordinates : )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 : 5 ; 1 : 6 ; 4 : 7 y -coordinates :0 : 071 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 029 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 012 : Theunderlyingfunctionthedatacomesfromis cos x ,butwepretendwedonotknowthis.Figure 3.2plotstheunderlyingfunction,thedata,andthepolynomialinterpolantforthedata.Clearly, thepolynomialinterpolantdoesnotcomeclosetogivingagoodapproximationtotheunderlying function cos x .

PAGE 99

CHAPTER3.INTERPOLATION 98 Figure3.2 Nowlet'sassumeweknowthederivativeoftheunderlyingfunctionatthesenodes: x -coordinates : )]TJ/F81 10.9091 Tf 8.484 0 Td [(1 : 5 ; 1 : 6 ; 4 : 7 y -coordinates :0 : 071 ; )]TJ/F81 10.9091 Tf 8.484 0 Td [(0 : 029 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 012 y 0 -values :1 ; )]TJ/F81 10.9091 Tf 8.484 0 Td [(1 ; 1 : WethenconstructtheHermiteinterpolatingpolynomial,incorporatingthederivativeinformation.Figure3.3plotstheHermiteinterpolatingpolynomial,togetherwiththepolynomial interpolant,andtheunderlyingfunction. ItisvisuallydiculttoseparatetheHermiteinterpolatingpolynomialfromtheunderlying function cos x inFigure3.3.GoingfrompolynomialinterpolationtoHermiteinterpolationresults inratherdramaticimprovementinapproximatingtheunderlyingfunction.

PAGE 100

CHAPTER3.INTERPOLATION 99 Figure3.3 ComputingtheHermitepolynomial WedonotuseTheorem61tocomputetheHermitepolynomial:thereisamoreecientmethod usingdivideddierencesforthiscomputation. Westartwiththedata: x 0 ;x 1 ;:::;x n y 0 ;y 1 ;:::;y n ; y i = f x i y 0 0 ;y 0 1 ;:::;y 0 n ; y 0 i = f 0 x i anddeneasequence z 0 ;z 1 ;:::;z 2 n +1 by z 0 = x 0 ;z 2 = x 1 ;z 4 = x 2 ;:::;z 2 n = x n z 1 = x 0 ;z 3 = x 1 ;z 5 = x 2 ;:::;z 2 n +1 = x n i.e., z 2 i = z 2 i +1 = x i ; for i =0 ; 1 ;:::;n:

PAGE 101

CHAPTER3.INTERPOLATION 100 ThentheHermitepolynomialcanbewrittenas: H 2 n +1 x = f [ z 0 ]+ f [ z 0 ;z 1 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z 0 + f [ z 0 ;z 1 ;z 2 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z 1 + ::: + f [ z 0 ;z 1 ;:::;z 2 n +1 ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z 2 n = f [ z 0 ]+ 2 n +1 X i =1 f [ z 0 ;::::;z i ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z i )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 : Thereisalittleproblemwithsomeoftherstdivideddierencesabove:theyareundened! Observethat f [ z 0 ;z 1 ]= f [ x 0 ;x 0 ]= f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 or,ingeneral, f [ z 2 i ;z 2 i +1 ]= f [ x i ;x i ]= f x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x i x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x i for i =0 ;:::;n: FromTheorem58,weknow f [ x 0 ;:::;x n ]= f n n ! forsome betweentheminandmaxof x 0 ;:::;x n : FromaclassicalresultbyHermite&GennochiseeAtkinson[3],page144,divided dierencesarecontinuousfunctionsoftheirvariables x 0 ;:::;x n : Thisimplieswecantakethelimit oftheaboveresultas x i ! x 0 forall i; whichresultsin f [ x 0 ;:::;x 0 ]= f n x 0 n ! : ThereforeintheHermitepolynomialcoecientcalculations,wewillput f [ z 2 i ;z 2 i +1 ]= f [ x i ;x i ]= f 0 x i = y 0 i for i =0 ; 1 ;:::;n: Example63. Let'scomputetheHermitepolynomialofExample62.Thedatais: ix i y i y 0 i 0-1.50.0711 11.6-0.029-1 24.7-0.0121 Here n =2 ; and 2 n +1=5 ; sotheHermitepolynomialis H 5 x = f [ z 0 ]+ 5 X i =1 f [ z 0 ;:::;z i ] x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(z i )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 : Thedivideddierencesare:

PAGE 102

CHAPTER3.INTERPOLATION 101 zf z 1stdi2nddi3rddi4thdi5thdi z 0 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 : 5 0.071 f 0 z 0 =1 z 1 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 : 5 0.071 )]TJ/F46 7.9701 Tf 6.587 0 Td [(0 : 032 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 1 : 6+1 : 5 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 33 f [ z 1 ;z 2 ]= )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 032 0.0065 z 2 =1 : 6 -0.029 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1+0 : 032 1 : 6+1 : 5 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 31 0.015 f 0 z 2 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 0.10-0.005 z 3 =1 : 6 -0.029 0 : 0055+1 4 : 7 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 : 6 =0 : 32 -0.016 f [ z 3 ;z 4 ]=0 : 0055 0 z 4 =4 : 7 -0.012 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(0 : 0055 4 : 7 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 : 6 =0 : 32 f 0 z 4 =1 z 5 =4 : 7 -0.012 Therefore,theHermitepolynomialis: H 5 x =0 : 071+1 x +1 : 5 )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 33 x +1 : 5 2 +0 : 0065 x +1 : 5 2 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 6+ +0 : 015 x +1 : 5 2 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 6 2 )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 005 x +1 : 5 2 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 : 6 2 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(4 : 7 : PythoncodeforcomputingHermiteinterpolatingpolynomial In[1]:importnumpyasnp importmatplotlib.pyplotasplt %matplotlibinline Thefollowingfunction hdi computesthedivideddierencesneededforHermiteinterpolation. Itisbasedonthefunction di forcomputingdivideddierencesforNewtoninterpolation.The inputsto hdi arethe x -coordinates,the y -coordinates,andthederivatives yprime . In[2]:defhdiffx,y,yprime: m=x.size #heremisthenumberofdatapoints.Noten=m-1 #and2n+1=2m-1 l=2*m z=np.zerosl a=np.zerosl foriinrangem: z[2*i]=x[i] z[2*i+1]=x[i] foriinrangem: a[2*i]=y[i]

PAGE 103

CHAPTER3.INTERPOLATION 102 a[2*i+1]=y[i] foriinnp.flipnp.arange1,m: #computesthefirstdivided #differencesusingderivatives a[2*i+1]=yprime[i] a[2*i]=a[2*i]-a[2*i-1]/z[2*i]-z[2*i-1] a[1]=yprime[0] forjinrange2,l: #computestherestofthedivideddifferences foriinnp.flipnp.arangej,l: a[i]=a[i]-a[i-1]/z[i]-z[i-j] returna Let'scomputethedivideddierencesofExample62. In[3]:hdiffnp.array[-1.5,1.6,4.7], np.array[0.071,-0.029,-0.012], np.array[1,-1,1] Out[3]:array[0.071,1.,-0.33298647,0.00671344,0.0154761, -0.00519663] Notethatinthehand-calculationsofExample63,wheretwo-digitroundingwasused,we obtained0.0065astherstthirddivideddierence.InthePythonoutputabove,thisdivided dierenceis0.0067. ThefollowingfunctioncomputestheHermiteinterpolatingpolynomial,usingthedivideddifferencesobtainedfrom hdi ,andthenevaluatesthepolynomialat w . In[4]:defhermitex,y,yprime,w: m=x.size #heremisthenumberofdatapoints.notthe #degreeofthepolynomial a=hdiffx,y,yprime z=np.zeros2*m foriinrangem: z[2*i]=x[i] z[2*i+1]=x[i] sum=a[0] pr=1.0 forjinrange2*m-1: pr*=w-z[j] sum+=a[j+1]*pr returnsum

PAGE 104

CHAPTER3.INTERPOLATION 103 Let'srecreatetheHermiteinterpolatingpolynomialplotofExample62. In[5]:xaxis=np.linspace-np.pi/2,3*np.pi/2,120 x=np.array[-1.5,1.6,4.7] y=np.array[0.071,-0.029,-0.012] yprime=np.array[1,-1,1] funct=np.cosxaxis interp=hermitex,y,yprime,xaxis plt.plotxaxis,interp,label= ' Hermiteinterpolation ' plt.plotxaxis,funct,label="cosx" plt.plotx,y, ' o ' ,label= ' data ' plt.legendloc= ' upperright ' ; Exercise3.3-1: Thefollowingtablegivesthevaluesof y = f x and y 0 = f 0 x where f x = e x +sin10 x .ComputetheHermiteinterpolatingpolynomialandthepolynomialinterpolant forthedatainthetable.Plotthetwointerpolatingpolynomialstogetherwith f x = e x +sin10 x on ; 3 . x 00.4122.63 y 10.7352.178.3014.219.1 y 0 11-5.04-5.6711.519.921.6

PAGE 105

CHAPTER3.INTERPOLATION 104 3.4Piecewisepolynomials:splineinterpolation AsweobservedinSection3.2,apolynomialinterpolantofhighdegreecanhavelargeoscillations, andthusprovideanoverallpoorapproximationtotheunderlyingfunction.Recallthatthedegree oftheinterpolatingpolynomialisdirectlylinkedtothenumberofdatapoints:wedonothavethe freedomtochoosethedegreeofthepolynomial. Insplineinterpolation,wetakeaverydierentapproach:insteadofndingasinglepolynomial thattsthegivendata,wendonelow-degreepolynomialthattsevery pair ofdata.Thisresults inseveralpolynomialpiecesjoinedtogether,andwetypicallyimposesomesmoothnessconditions ondierentpieces.Theterm splinefunction meansafunctionthatconsistsofpolynomialpieces joinedtogetherwithsomesmoothnessconditions. In linearsplineinterpolation ,wesimplyjoindatapointsthenodes,bylinesegments, thatis,linearpolynomials.Forexample,considerthefollowinggurethatplotsthreedata points x i )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 ;y i )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 ; x i ;y i ; x i +1 ;y i +1 .Wetalinearpolynomial P x totherstpairofdata points x i )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 ;y i )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 ; x i ;y i ,andanotherlinearpolynomial Q x tothesecondpairofdatapoints x i ;y i ; x i +1 ;y i +1 . Figure3.4:Linearspline Let P x = ax + b and Q x = cx + d .Wendthecoecients a;b;c;d bysolving P x i )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 = y i )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 P x i = y i Q x i = y i Q x i +1 = y i +1 whichisasystemoffourequationsandfourunknowns.Wethenrepeatthisprocedureforalldata points, x 0 ;y 0 ; x 1 ;y 1 ;:::; x n ;y n ,todetermineallofthelinearpolynomials.

PAGE 106

CHAPTER3.INTERPOLATION 105 Onedisadvantageoflinearsplineinterpolationisthelackofsmoothness.Therstderivative ofthesplineisnotcontinuousatthenodesunlessthedatafallonaline.Wecanobtainbetter smoothnessbyincreasingthedegreeofthepiecewisepolynomials.In quadraticsplineinterpolation ,weconnectthenodesviaseconddegreepolynomials. Figure3.5:Quadraticspline Let P x = a 0 + a 1 x + a 2 x 2 and Q x = b 0 + b 1 x + b 2 x 2 : Therearesixunknownstodetermine, butonlyfourequationsfromtheinterpolationconditions: P x i )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 = y i )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 ;P x i = y i ;Q x i = y i ;Q x i +1 = y i +1 .Wecanndextratwoconditionsbyrequiringsomesmoothness, P 0 x i = Q 0 x i , andanotherequationbyrequiring P 0 or Q 0 takeacertainvalueatoneoftheendpoints. Cubicsplineinterpolation Thisisthemostcommonsplineinterpolation.Itusescubicpolynomialstoconnectthenodes. Considerthedata x 0 ;y 0 ; x 1 ;y 1 ;:::; x n ;y n ; where x 0
PAGE 107

CHAPTER3.INTERPOLATION 106 Figure3.6:Cubicspline Thepolynomial S i interpolatesthenodes x i ;y i ; x i +1 ;y i +1 .Let S i x = a i + b i x + c i x 2 + d i x 3 for i =0 ; 1 ;:::;n )]TJ/F81 10.9091 Tf 10.92 0 Td [(1 : Thereare 4 n unknownstodetermine: a i ;b i ;c i ;d i ,as i takesonvaluesfrom0 to n )]TJ/F81 10.9091 Tf 11.13 0 Td [(1 .Let'sdescribetheequations S i mustsatisfy.First,theinterpolationconditions,thatis, therequirementthat S i passesthroughthenodes x i ;y i ; x i +1 ;y i +1 : S i x i = y i S i x i +1 = y i +1 for i =0 ; 1 ;:::;n )]TJ/F81 10.9091 Tf 10.405 0 Td [(1 ,whichgives 2 n equations.Thenextgroupofequationsareaboutsmoothness: S 0 i )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x i = S 0 i x i S 00 i )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x i = S 00 i x i for i =1 ; 2 ;:::;n )]TJ/F81 10.9091 Tf 11.209 0 Td [(1 ,whichgives 2 n )]TJ/F81 10.9091 Tf 11.208 0 Td [(1=2 n )]TJ/F81 10.9091 Tf 11.209 0 Td [(2 equations.Lasttwoequationsarecalledthe boundaryconditions.Therearetwochoices: Freeornaturalboundary: S 00 0 x 0 = S 00 n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x n =0 Clampedboundary: S 0 0 x 0 = f 0 x 0 and S 0 n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x n = f 0 x n

PAGE 108

CHAPTER3.INTERPOLATION 107 Eachboundarychoicegivesanothertwoequations,bringingthetotalnumberofequationsto 4 n . Thereare 4 n unknownsaswell.Dothesesystemsofequationshaveauniquesolution?Theanswer isyes,andaproofcanbefoundinBurden,Faires,Burden[4].Thesplineobtainedfromtherst boundarychoiceiscalleda naturalspline ,andtheotheroneiscalleda clampedspline . Example64. Findthenaturalcubicsplinethatinterpolatesthedata ; 0 ; ; 1 ; ; 0 . Solution. Wehavetwocubicpolynomialstodetermine: S 0 x = a 0 + b 0 x + c 0 x 2 + d 0 x 3 S 1 x = a 1 + b 1 x + c 1 x 2 + d 1 x 3 Theinterpolationequationsare: S 0 =0 a 0 =0 S 0 =1 a 0 + b 0 + c 0 + d 0 =1 S 1 =1 a 1 + b 1 + c 1 + d 1 =1 S 1 =0 a 1 +2 b 1 +4 c 1 +8 d 1 =0 Weneedthederivativesofthepolynomialsfortheotherequations: S 0 0 x = b 0 +2 c 0 x +3 d 0 x 2 S 0 1 x = b 1 +2 c 1 x +3 d 1 x 2 S 00 0 x =2 c 0 +6 d 0 x S 00 1 x =2 c 1 +6 d 1 x Thesmoothnessconditionsare: S 0 0 = S 0 1 b 0 +2 c 0 +3 d 0 = b 1 +2 c 1 +3 d 1 S 00 0 = S 00 1 2 c 0 +6 d 0 =2 c 1 +6 d 1 Thenaturalboundaryconditionsare: S 00 0 =0 2 c 0 =0 S 00 1 =0 2 c 1 +12 d 1 =0 Thereareeightequationsandeightunknowns.However, a 0 = c 0 =0 ,sothatreducesthenumber ofequationsandunknownstosix.Werewritetheequationsbelow,substituting a 0 = c 0 =0 ,and

PAGE 109

CHAPTER3.INTERPOLATION 108 simplifyingwhenpossible: b 0 + d 0 =1 a 1 + b 1 + c 1 + d 1 =1 a 1 +2 b 1 +4 c 1 +8 d 1 =0 b 0 +3 d 0 = b 1 +2 c 1 +3 d 1 3 d 0 = c 1 +3 d 1 c 1 +6 d 1 =0 WewillusePythontosolvethissystemofequations.Todothat,werstrewritethesystemof equationsasamatrixequation Ax = v where A = 2 6 6 6 6 6 6 6 6 6 4 110000 001111 001248 130 )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 )]TJ/F81 10.9091 Tf 8.485 0 Td [(2 )]TJ/F81 10.9091 Tf 8.485 0 Td [(3 0300 )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 )]TJ/F81 10.9091 Tf 8.485 0 Td [(3 000016 3 7 7 7 7 7 7 7 7 7 5 ;x = 2 6 6 6 6 6 6 6 6 6 4 b 0 d 0 a 1 b 1 c 1 d 1 3 7 7 7 7 7 7 7 7 7 5 ;v = 2 6 6 6 6 6 6 6 6 6 4 1 1 0 0 0 0 3 7 7 7 7 7 7 7 7 7 5 : Weenterthematrices A;v inPythonandsolvetheequation Ax = v usingthecommand np:linalg:solve . In[1]:A=np.array[[1,1,0,0,0,0],[0,0,1,1,1,1], [0,0,1,2,4,8],[1,3,0,-1,-2,-3], [0,3,0,0,-1,-3],[0,0,0,0,1,6]] A Out[1]:array[[1,1,0,0,0,0], [0,0,1,1,1,1], [0,0,1,2,4,8], [1,3,0,-1,-2,-3], [0,3,0,0,-1,-3], [0,0,0,0,1,6]] In[2]:v=np.array[1,1,0,0,0,0] v Out[2]:array[1,1,0,0,0,0] In[3]:np.linalg.solveA,v

PAGE 110

CHAPTER3.INTERPOLATION 109 Out[3]:array[1.5,-0.5,-1.,4.5,-3.,0.5] Therefore,thepolynomialsare: S 0 x =1 : 5 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(0 : 5 x 3 S 1 x = )]TJ/F81 10.9091 Tf 8.484 0 Td [(1+4 : 5 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(3 x 2 +0 : 5 x 3 Solvingtheequationsofasplineevenforthreedatapointscanbetedious.Fortunately,there isageneralapproachtosolvingtheequationsfornaturalandclampedsplines,foranynumberof datapoints.WewillusethisapproachwhenwewritePythoncodesforsplinesnext. Exercise3.4-1: Findthenaturalcubicsplineinterpolantforthefollowingdata: x -101 y 120 Exercise3.4-2: Thefollowingisaclampedcubicsplineforafunction f denedon [1 ; 3] : s x = 8 < : s 0 x = x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1+ x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [( x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 3 ; if 1 x< 2 s 1 x = a + b x )]TJ/F81 10.9091 Tf 10.91 0 Td [(2+ c x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 2 + d x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 3 if 2 x< 3 : Find a;b;c; and d; if f 0 =1 and f 0 =2 : Pythoncodeforsplineinterpolation In[1]:importnumpyasnp importmatplotlib.pyplotasplt %matplotlibinline Thefunction CubicNatural takesthe x -and y -coordinatesofthedataasinput,andcomputes thenaturalcubicsplineinterpolatingthedata,bysolvingtheresultingmatrixequation.Thecode isbasedonAlgorithm3.4ofBurden,Faires,Burden[4].Theoutputisthecoecientsofthe m )]TJ/F81 10.9091 Tf 10.935 0 Td [(1 cubicpolynomials, a i ;b i ;c i ;d i ;i =0 ;:::;m )]TJ/F81 10.9091 Tf 10.935 0 Td [(2 where m isthenumberofdatapoints.These coecientsarestoredinthearrays a;b;c;d andreturnedattheendofthefunction,sothatwecan accessthesearrayslatertoevaluatethesplineforagivenvalue w . In[2]:defCubicNaturalx,y: m=x.size #misthenumberofdatapoints n=m-1

PAGE 111

CHAPTER3.INTERPOLATION 110 a=np.zerosm b=np.zerosn c=np.zerosm d=np.zerosn foriinrangem: a[i]=y[i] h=np.zerosn foriinrangen: h[i]=x[i+1]-x[i] u=np.zerosn u[0]=0 foriinrange1,n: u[i]=3*a[i+1]-a[i]/h[i]-3*a[i]-a[i-1]/h[i-1] s=np.zerosm z=np.zerosm t=np.zerosn s[0]=1 z[0]=0 t[0]=0 foriinrange1,n: s[i]=2*x[i+1]-x[i-1]-h[i-1]*t[i-1] t[i]=h[i]/s[i] z[i]=u[i]-h[i-1]*z[i-1]/s[i] s[m-1]=1 z[m-1]=0 c[m-1]=0 foriinnp.flipnp.arangen: c[i]=z[i]-t[i]*c[i+1] b[i]=a[i+1]-a[i]/h[i]-h[i]*c[i+1]+2*c[i]/3 d[i]=c[i+1]-c[i]/3*h[i] returna,b,c,d Oncethematrixequationissolved,andthecoecientsofthecubicpolynomialsarecomputed by CubicNatural ,thenextstepistoevaluatethesplineatagivenvalue.Thisisdonebythe followingfunction CubicNaturalEval .Theinputsarethevalueatwhichthesplineisevaluated, w ,the x -coordinatesofthedataandthecoecientscomputedby CubicNatural .Thefunction rstndstheinterval [ x i ;x i +1 ] ;i =0 ;:::;m )]TJ/F81 10.9091 Tf 11.195 0 Td [(2 ;w belongsto,andthenevaluatesthesplineat w usingthecorrespondingcubicpolynomial. In[3]:defCubicNaturalEvalw,x,coeff:

PAGE 112

CHAPTER3.INTERPOLATION 111 m=x.size ifwx[m-1]: print ' error:splineevaluatedoutsideitsdomain ' return n=m-1 p=0 foriinrangen: ifw<=x[i+1]: break else: p+=1 #pisthenumberofthesubintervalwfallsinto,i.e.,p=imeans #wfallsintotheithsubinterval$x_i,x_{i+1},andtherefore #thevalueofthesplineatwis #a_i+b_i*w-x_i+c_i*w-x_i^2+d_i*w-x_i^3. a=coeff[0] b=coeff[1] c=coeff[2] d=coeff[3] returna[p]+b[p]*w-x[p]+c[p]*w-x[p]**2+d[p]*w-x[p]**3 NextwewillcompareNewtonandnaturalcubicsplineinterpolationwhenappliedtoRunge's function.WeimportthefunctionsforNewtoninterpolationrst. In[4]:defdiffx,y: m=x.size #heremisthenumberofdatapoints. a=np.zerosm foriinrangem: a[i]=y[i] forjinrange1,m: foriinnp.flipnp.arangej,m: a[i]=a[i]-a[i-1]/x[i]-x[i-j] returna In[5]:defnewtonx,y,z: m=x.size #heremisthenumberofdatapoints,notthedegree #ofthepolynomial a=diffx,y sum=a[0] pr=1.0

PAGE 113

CHAPTER3.INTERPOLATION 112 forjinrangem-1: pr*=z-x[j] sum+=a[j+1]*pr returnsum Hereisthecodethatcomputesthecubicspline,Newtoninterpolation,andplotthem. In[6]:xaxis=np.linspace-5,5,1000 f=lambdax:1/1+x**2 runge=fxaxis xi=np.arange-5,6 yi=fxi coeff=CubicNaturalxi,yi naturalspline=np.arraylistmaplambdax:CubicNaturalEvalx,xi,coeff,xaxis interp=newtonxi,yi,xaxis #Interpolatingpolynomialfor #thedata plt.plotxaxis,runge,label= ' $1/+x^2$ ' plt.plotxaxis,interp,label= ' Interpolatingpoly ' plt.plotxaxis,naturalspline,label= ' Naturalcubicspline ' plt.plotxi,yi, ' o ' ,label= ' Data ' plt.legendloc= ' uppercenter ' ; ThecubicsplinegivesanexcellentttoRunge'sfunctiononthisscale:wecannotvisually separateitfromthefunctionitself. Thefollowingfunction CubicClamped computestheclampedcubicspline;thecodeisbased onAlgorithm3.5ofBurden,Faires,Burden[4].Thefunction CubicClampedEval evaluatesthe splineatagivenvalue.

PAGE 114

CHAPTER3.INTERPOLATION 113 In[7]:defCubicClampedx,y,yprime_left,yprime_right: m=x.size #misthenumberofdatapoints n=m-1 A=np.zerosm B=np.zerosn C=np.zerosm D=np.zerosn foriinrangem: A[i]=y[i] h=np.zerosn foriinrangen: h[i]=x[i+1]-x[i] u=np.zerosm u[0]=3*A[1]-A[0]/h[0]-3*yprime_left u[m-1]=3*yprime_right-3*A[m-1]-A[m-2]/h[m-2] foriinrange1,n: u[i]=3*A[i+1]-A[i]/h[i]-3*A[i]-A[i-1]/h[i-1] s=np.zerosm z=np.zerosm t=np.zerosn s[0]=2*h[0] t[0]=0.5 z[0]=u[0]/s[0] foriinrange1,n: s[i]=2*x[i+1]-x[i-1]-h[i-1]*t[i-1] t[i]=h[i]/s[i] z[i]=u[i]-h[i-1]*z[i-1]/s[i] s[m-1]=h[m-2]*2-t[m-2] z[m-1]=u[m-1]-h[m-2]*z[m-2]/s[m-1] C[m-1]=z[m-1] foriinnp.flipnp.arangen: C[i]=z[i]-t[i]*C[i+1] B[i]=A[i+1]-A[i]/h[i]-h[i]*C[i+1]+2*C[i]/3 D[i]=C[i+1]-C[i]/3*h[i] returnA,B,C,D In[8]:defCubicClampedEvalw,x,coeff: m=x.size ifwx[m-1]:

PAGE 115

CHAPTER3.INTERPOLATION 114 print ' error:splineevaluatedoutsideitsdomain ' return n=m-1 p=0 foriinrangen: ifw<=x[i+1]: break else: p+=1 A=coeff[0] B=coeff[1] C=coeff[2] D=coeff[3] returnA[p]+B[p]*w-x[p]+C[p]*w-x[p]**2+D[p]*w-x[p]**3 Inthefollowing,weusenaturalandclampedcubicsplinestointerpolatedatacomingfrom sin x atthex-coordinates: 0 ;; 3 = 2 ; 2 .Thederivativesattheendpointsarebothequalto1. In[9]:xaxis=np.linspace0,2*np.pi,600 f=lambdax:np.sinx funct=fxaxis xi=np.array[0,np.pi,3*np.pi/2,2*np.pi] yi=fxi coeff=CubicNaturalxi,yi naturalspline=np.arraylistmaplambdax:CubicNaturalEvalx,xi,coeff,xaxis coeff=CubicClampedxi,yi,1,1 clampedspline=np.arraylistmaplambdax:CubicClampedEvalx,xi,coeff,xaxis plt.plotxaxis,funct,label= ' sinx ' plt.plotxaxis,naturalspline,linestyle= ' -' ,label= ' Naturalcubicspline ' plt.plotxaxis,clampedspline,label= ' Clampedcubicspline ' plt.plotxi,yi, ' o ' ,label= ' Data ' plt.legendloc= ' upperright ' ;

PAGE 116

CHAPTER3.INTERPOLATION 115 Especiallyontheinterval ; ,theclampedsplinegivesamuchbetterapproximationto sin x thanthenaturalspline.However,addinganextradatapointbetween0and removesthevisual dierencesbetweenthesplines. In[10]:xaxis=np.linspace0,2*np.pi,600 f=lambdax:np.sinx funct=fxaxis xi=np.array[0,np.pi/2,np.pi,3*np.pi/2,2*np.pi] yi=fxi coeff=CubicNaturalxi,yi naturalspline=np.arraylistmaplambdax:CubicNaturalEvalx,xi,coeff,xaxis coeff=CubicClampedxi,yi,1,1 clampedspline=np.arraylistmaplambdax:CubicClampedEvalx,xi,coeff,xaxis plt.plotxaxis,funct,label= ' sinx ' plt.plotxaxis,naturalspline,linestyle= ' -' ,label= ' Naturalcubicspline ' plt.plotxaxis,clampedspline,label= ' Clampedcubicspline ' plt.plotxi,yi, ' o ' ,label= ' Data ' plt.legendloc= ' upperright ' ;

PAGE 117

CHAPTER3.INTERPOLATION 116

PAGE 118

CHAPTER3.INTERPOLATION 117 AryaandtheletterNUH AryalovesDr.Seusswhodoesn't?,andshe iswritinghertermpaperinanEnglishclasson OnBeyondZebra! 2 .InthisbookDr.Seussinventsnewletters,oneofwhichiscalledNUH.He writes: AndNUHistheletterIusetospell Nutches Wholiveinsmallcaves,knownas Nitches,forhutches. TheseNutcheshavetroubles,the biggestofwhichis Thefacttherearemanymore NutchesthanNitches. Whatdoesthisletterlooklike?Well,something likethis. WhatAryawantsisadigitizedversionofthesketch;agurethatissmoothandcanbe manipulatedusinggraphicssoftware.TheletterNUHisalittlecomplicatedtoapplyaspline interpolationdirectly,sinceithassomecusps.Forsuchplanarcurves,wecanusetheirparametric representation,anduseacubicsplineinterpolationfor x -and y -coordinatesseparately.Tothisend, AryapickseightpointsontheletterNUH,andlabelsthemas t =1 ; 2 ;:::; 8 ;seethegurebelow. 2 Seuss,1955.OnBeyondZebra!RandomHouseforYoungReaders.

PAGE 119

CHAPTER3.INTERPOLATION 118 Thenforeachpointsheeyeballsthe x and y -coordinateswiththehelpofagraphpaper.The resultsaredisplayedinthetablebelow. t 12345678 x 00-0.050.10.40.650.70.76 y 01.252.510.30.91.50 Thenextstepistotacubicsplinetothedata t 1 ;x 1 ;:::; t 8 ;x 8 ,andanothercubicspline tothedata t 1 ;y 1 ;:::; t 8 ;y 8 .Let'scallthesesplinesxspline t ; yspline t ,respectively,sincethey representthe x -and y -coordinatesasfunctionsoftheparameter t .Plottingxspline t ; yspline t willproducetheletterNUH,aswecanseeinthefollowingPythoncodes. First,loadtheNumPyandMatplotlibpackages,andcopyandevaluatethefunctions CubicNatural and CubicNaturalEval thatwediscussedearlier.HereistheletterNUH,obtainedby splineinterpolation: In[1]:t=np.array[1,2,3,4,5,6,7,8] x=np.array[0,0,-0.05,0.1,0.4,0.65,0.7,0.76] y=np.array[0,1.25,2.5,1,0.3,0.9,1.5,0] taxis=np.linspace1,8,700 coeff=CubicNaturalt,x xspline=np.arraylistmaplambdax:CubicNaturalEvalx,t,coeff,taxis coeff=CubicNaturalt,y yspline=np.arraylistmaplambdax:CubicNaturalEvalx,t,coeff,taxis plt.plotxspline,yspline,linewidth=5; Thislookslikeitneedstobesqueezed!Adjustingtheaspectratiogivesabetterimage.Inthe following,weusethecommands In[2]:w,h=plt.figaspect2; plt.figurefigsize=w,h;

PAGE 120

CHAPTER3.INTERPOLATION 119 toadjusttheaspectratio. In[3]:t=np.array[1,2,3,4,5,6,7,8] x=np.array[0,0,-0.05,0.1,0.4,0.65,0.7,0.76] y=np.array[0,1.25,2.5,1,0.3,0.9,1.5,0] taxis=np.linspace1,8,700 coeff=CubicNaturalt,x xspline=np.arraylistmaplambdax:CubicNaturalEvalx,t,coeff,taxis coeff=CubicNaturalt,y yspline=np.arraylistmaplambdax:CubicNaturalEvalx,t,coeff,taxis w,h=plt.figaspect2 plt.figurefigsize=w,h plt.plotxspline,yspline,linewidth=5; Exercise3.4-3: Limaonisacurve,namedafteraFrenchwordforsnail,whichappearsin thestudyofplanetarymotion.Thepolarequationforthecurveis r =1+ c sin where c isa constant.Belowisaplotofthecurvewhen c =1 .

PAGE 121

CHAPTER3.INTERPOLATION 120 The x;y coordinatesofthedotsonthecurvearedisplayedinthefollowingtable: x 00.511.30-1.3-1-0.50 y 0-0.2500.7120.710-0.250 Recreatethelimaonabove,byapplyingthesplineinterpolationforplanecurvesapproachused in AryaandtheletterNUH exampletothepointsgiveninthetable.

PAGE 122

Chapter4 NumericalQuadratureand Dierentiation Estimating R b a f x dx usingsumsoftheform P n i =0 w i f x i isknownasthequadratureproblem. Here w i arecalledweights,and x i arecallednodes.Theobjectiveistodeterminethenodesand weightstominimizeerror. 4.1Newton-Cotesformulas Theideaistoconstructthepolynomialinterpolant P x andcompute R b a P x dx asanapproximationto R b a f x dx .Givennodes x 0 ;x 1 ;:::;x n ,theLagrangeformoftheinterpolantis P n x = n X i =0 f x i l i x andfromtheinterpolationerrorformulaTheorem51,wehave f x = P n x + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n f n +1 x n +1! ; where x 2 [ a;b ] .Wehavewritten x insteadof toemphasizethat dependsonthevalueof x . Takingtheintegralofbothsidesyields Z b a f x dx = Z b a P n x dx | {z } quadraturerule + 1 n +1! Z b a n Y i =0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x i f n +1 x dx: | {z } errorterm .1 121

PAGE 123

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 122 Therstintegralontheright-handsidegivesthequadraturerule: Z b a P n x dx = Z b a n X i =0 f x i l i x ! dx = n X i =0 0 B B @ Z b a l i x dx | {z } w i 1 C C A f x i = n X i =0 w i f x i ; andthesecondintegralgivestheerrorterm. Weobtaindierentquadraturerulesbytakingdierentnodes,ornumberofnodes.Thefollowing resultisusefulinthetheoreticalanalysisofNewton-Cotesformulas. Theorem65 Weightedmeanvaluetheoremforintegrals . Suppose f 2 C 0 [ a;b ] ; theRiemann integralof g x existson [ a;b ] ; and g x doesnotchangesignon [ a;b ] : Thenthereexists 2 a;b with R b a f x g x dx = f R b a g x dx: Twowell-knownnumericalquadraturerules,trapezoidalruleandSimpson'srule,areexamples ofNewton-Cotesformulas: Trapezoidalrule Let f 2 C 2 [ a;b ] .Taketwonodes, x 0 = a;x 1 = b; andusethelinearLagrangepolynomial P 1 x = x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 0 )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 1 f x 0 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 f x 1 toestimate f x .Substitute n =1 inEquation4.1toget Z b a f x dx = Z b a P 1 x dx + 1 2 Z b a 1 Y i =0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x i f 00 x dx; andthensubstitutefor P 1 toobtain Z b a f x dx = Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 f x 0 dx + Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 f x 1 dx + 1 2 Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 f 00 x dx: Thersttwointegralsontheright-handsideare h 2 f x 0 and h 2 f x 1 .Let'sevaluate Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 f 00 x dx: WewilluseTheorem65forthiscomputation.Notethatthefunction x )]TJ/F82 10.9091 Tf 11.453 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 11.453 0 Td [(x 1 = x )]TJ/F82 10.9091 Tf 10.725 0 Td [(a x )]TJ/F82 10.9091 Tf 10.726 0 Td [(b doesnotchangesignontheinterval [ a;b ] anditisintegrable:sothisfunction servestheroleof g x inTheorem65.Theotherterm, f 00 x ,servestheroleof f x . Applyingthetheorem,weget Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(b f 00 x dx = f 00 Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(b dx

PAGE 124

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 123 wherewekeptthesamenotation ,somewhatinappropriately,aswemoved f 00 x from insidetotheoutsideoftheintegral.Finally,observethat Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(b dx = a )]TJ/F82 10.9091 Tf 10.909 0 Td [(b 3 6 = )]TJ/F82 10.9091 Tf 8.485 0 Td [(h 3 6 ; where h = b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a .Puttingallthepiecestogether,wehaveobtained Z b a f x dx = h 2 [ f x 0 + f x 1 ] )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(h 3 12 f 00 : Simpson'srule Let f 2 C 4 [ a;b ] .Takethreeequally-spacednodes, x 0 = a , x 1 = a + h , x 2 = b; where h = b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a 2 ; andusetheseconddegreeLagrangepolynomial P 2 x = x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 2 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 f x 0 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 f x 1 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 f x 2 toestimate f x .Substitute n =2 inEquation4.1toget Z b a f x dx = Z b a P 2 x dx + 1 3! Z b a 2 Y i =0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x i f x dx; andthensubstitutefor P 2 toobtain Z b a f x dx = Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 2 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 f x 0 dx + Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 f x 1 dx + + Z b a x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 f x 2 dx + 1 6 Z b a x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 f x dx: Thesumoftherstthreeintegralsontheright-handsidesimplifyas: h 3 [ f x 0 +4 f x 1 + f x 2 ] .ThelastintegralcannotbeevaluatedusingTheorem65 directly,likeinthetrapezoidalrule,sincethefunction x )]TJ/F82 10.9091 Tf 11.828 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 11.828 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 11.829 0 Td [(x 2 changes signon [ a;b ] .However,acleverapplicationofintegrationbypartstransformstheintegral toanintegralwhereTheorem65isapplicableseeAtkinson[3]fordetails,andtheintegral simpliesas )]TJ/F49 7.9701 Tf 9.68 4.296 Td [(h 5 90 f forsome 2 a;b .Insummary,weobtain Z b a f x dx = h 3 [ f x 0 +4 f x 1 + f x 2 ] )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(h 5 90 f where 2 a;b :

PAGE 125

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 124 Exercise4.1-1: ProvethatthesumoftheweightsinNewton-Cotesrulesis b )]TJ/F82 10.9091 Tf 10.066 0 Td [(a ,forany n: Denition66. Thedegreeofaccuracy,orprecision,ofaquadratureformulaisthelargestpositive integer n suchthattheformulaisexactfor f x = x k ; when k =0 ; 1 ;:::;n; orequivalently,forany polynomialofdegreelessthanorequalto n . ObservethatthetrapezoidalandSimpson'sruleshavedegreesofaccuracyofoneandthree. ThesetworulesareexamplesofclosedNewton-Cotesformulas;closedreferstothefactthattheend points a;b oftheintervalareusedasnodesinthequadraturerule.Hereisthegeneraldenition. Denition67 ClosedNewton-Cotes . The n +1 -pointclosedNewton-Cotesformulausesnodes x i = x 0 + ih ,for i =0 ; 1 ;:::;n; where x 0 = a;x n = b;h = b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a n ; and w i = Z x n x 0 l i x dx = Z x n x 0 n Y j =0 ;j 6 = i x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j dx: ThefollowingtheoremprovidesanerrorformulafortheclosedNewton-Cotesformula.Aproof canbefoundinIsaacsonandKeller[12]. Theorem68. Forthe n +1 -pointclosedNewton-Cotesformula,wehave: if n isevenand f 2 C n +2 [ a;b ] Z b a f x dx = n X i =0 w i f x i + h n +3 f n +2 n +2! Z n 0 t 2 t )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 t )]TJ/F82 10.9091 Tf 10.909 0 Td [(n dt; if n isoddand f 2 C n +1 [ a;b ] Z b a f x dx = n X i =0 w i f x i + h n +2 f n +1 n +1! Z n 0 t t )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 t )]TJ/F82 10.9091 Tf 10.909 0 Td [(n dt where issomenumberin a;b : SomewellknownexamplesofclosedNewton-Cotesformulasarethetrapezoidalrule n =1 ; Simpson'srule n =2 ,andSimpson'sthree-eighthrule n =3 : Observethatinthe n +1 -point closedNewton-Cotesformula,if n iseven,thenthedegreeofaccuracyis n +1 ,althoughthe interpolatingpolynomialisofdegree n: TheopenNewton-Cotesformulasexcludetheendpointsof theinterval. Denition69 OpenNewton-Cotes . The n +1 -pointopenNewton-Cotesformulausesnodes x i = x 0 + ih ,for i =0 ; 1 ;:::;n; where x 0 = a + h;x n = b )]TJ/F82 10.9091 Tf 10.909 0 Td [(h;h = b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a n +2 and w i = Z b a l i x dx = Z b a n Y j =0 ;j 6 = i x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j dx:

PAGE 126

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 125 Weput a = x )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 and b = x n +1 . TheerrorformulafortheopenNewton-Cotesformulaisgivennext;foraproofseeIsaacsonand Keller[12]. Theorem70. Forthe n +1 -pointopenNewton-Cotesformula,wehave: if n isevenand f 2 C n +2 [ a;b ] Z b a f x dx = n X i =0 w i f x i + h n +3 f n +2 n +2! Z n +1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 t 2 t )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 t )]TJ/F82 10.9091 Tf 10.909 0 Td [(n dt; if n isoddand f 2 C n +1 [ a;b ] Z b a f x dx = n X i =0 w i f x i + h n +2 f n +1 n +1! Z n +1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 t t )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 t )]TJ/F82 10.9091 Tf 10.909 0 Td [(n dt; where issomenumberin a;b : Thewell-knownmidpointruleisanexampleofopenNewton-Cotesformula: Midpointrule Takeonenode, x 0 = a + h ,whichcorrespondsto n =0 intheabovetheoremtoobtain Z b a f x dx =2 hf x 0 + h 3 f 00 3 where h = b )]TJ/F82 10.9091 Tf 10.221 0 Td [(a = 2 : Thisruleinterpolates f byaconstantthevaluesof f atthemidpoint, thatis,apolynomialofdegree0,butithasdegreeofaccuracy1. Remark71. BothclosedandopenNewton-Cotesformulasusingoddnumberofnodes n iseven, gainanextradegreeofaccuracybeyondthatofthepolynomialinterpolantonwhichtheyarebased. Thisisduetocancellationofpositiveandnegativeerror. TherearesomedrawbacksofNewton-Cotesformulas: Ingeneral,theserulesarenotofthehighestdegreeofaccuracypossibleforthenumberof nodesused. Theuseoflargenumberofequallyspacednodesmayincurtheerraticbehaviorassociated withhigh-degreepolynomialinterpolation.Weightsforahigh-orderrulemaybenegative, potentiallyleadingtolossofsignicanceerrors. Let I n denotetheNewton-Cotesestimateofanintegralbasedon n nodes. I n maynotconverge tothetrueintegralas n !1 forperfectlywell-behavedintegrands. Example72. Estimate R 1 0 : 5 x x dx usingthemidpoint,trapezoidal,andSimpson'srules.

PAGE 127

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 126 Solution. Let f x = x x .Themidpointestimatefortheintegralis 2 hf x 0 where h = b )]TJ/F82 10.9091 Tf 12.174 0 Td [(a = 2=1 = 4 and x 0 =0 : 75 : Thenthemidpointestimate,using6digits,is f : 75 = 2= 0 : 805927 = 2=0 : 402964 .Thetrapezoidalestimateis h 2 [ f : 5+ f ] where h =1 = 2 ,whichresultsin 1 : 707107 = 4=0 : 426777 : Finally,forSimpson'srule, h = b )]TJ/F82 10.9091 Tf 11.605 0 Td [(a = 2=1 = 4 ; andthusthe estimateis h 3 [ f : 5+4 f : 75+ f ]= 1 12 [0 : 707107+4 : 805927+1]=0 : 410901 : Hereisasummaryoftheresults: MidpointTrapezoidalSimpson's 0.4029640.4267770.410901 Example73. Findtheconstants c 0 ;c 1 ;x 1 sothatthequadratureformula Z 1 0 f x dx = c 0 f + c 1 f x 1 hasthehighestpossibledegreeofaccuracy. Solution. Wewillndhowmanyofthepolynomials 1 ;x;x 2 ;::: therulecanintegrateexactly.If p x =1 ; then Z 1 0 p x dx = c 0 p + c 1 p x 1 1= c 0 + c 1 : If p x = x; weget Z 1 0 p x dx = c 0 p + c 1 p x 1 1 2 = c 1 x 1 and p x = x 2 implies Z 1 0 p x dx = c 0 p + c 1 p x 1 1 3 = c 1 x 2 1 : Wehavethreeunknownsandthreeequations,sowehavetostophere.Solvingthethreeequations weget: c 0 =1 = 4 ;c 1 =3 = 4 ;x 1 =2 = 3 : Sothequadratureruleisofprecisiontwoanditis: Z 1 0 f x dx = 1 4 f + 3 4 f 2 3 : Exercise4.1-2: Find c 0 ;c 1 ;c 2 sothatthequadraturerule R 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 f x dx = c 0 f )]TJ/F81 10.9091 Tf 8.485 0 Td [(1+ c 1 f + c 2 f hasdegreeofaccuracy2.

PAGE 128

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 127 4.2CompositeNewton-Cotesformulas Iftheinterval [ a;b ] inthequadratureislarge,thentheNewton-Cotesformulaswillgivepoor approximations.Thequadratureerrordependson h = b )]TJ/F82 10.9091 Tf 11.03 0 Td [(a =n closedformulas,andif b )]TJ/F82 10.9091 Tf 11.029 0 Td [(a is large,thensois h; henceerror.Ifweraise n tocompensateforlargeinterval,thenwefaceaproblem discussedearlier:errorduetotheoscillatorybehaviorofhigh-degreeinterpolatingpolynomialsthat useequally-spacednodes.Asolutionistobreakupthedomainintosmallerintervalsandusea Newton-Cotesrulewithasmaller n oneachsubinterval:thisisknownasacompositerule. Example74. Let'scompute R 2 0 e x sin xdx: Theantiderivativecanbecomputedusingintegration byparts,andthetruevalueoftheintegralto6digitsis5.39689.IfweapplytheSimpson'srulewe get: Z 2 0 e x sin xdx 1 3 e 0 sin0+4 e sin1+ e 2 sin2=5 : 28942 : Ifwepartitiontheintegrationdomain ; 2 into ; 1 and ; 2 ,andapplySimpson'sruletoeach domainseparately,weget Z 2 0 e x sin xdx = Z 1 0 e x sin xdx + Z 2 1 e x sin xdx 1 6 e 0 sin0+4 e 0 : 5 sin : 5+ e sin1+ 1 6 e sin1+4 e 1 : 5 sin : 5+ e 2 sin2 =5 : 38953 ; improvingtheaccuracysignicantly.Notethatwehaveusedvenodes, 0 ; 0 : 5 ; 1 ; 1 : 5 ; 2 ,whichsplit thedomain ; 2 intofoursubintervals. Thecompositerulesformidpoint,trapezoidal,andSimpson'srule,withtheirerrorterms,are: CompositeMidpointrule Let f 2 C 2 [ a;b ] ;n beeven, h = b )]TJ/F49 7.9701 Tf 6.587 0 Td [(a n +2 ; and x j = a + j +1 h for j = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 0 ;:::;n +1 : The compositeMidpointrulefor n +2 subintervalsis Z b a f x dx =2 h n= 2 X j =0 f x 2 j + b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 6 h 2 f 00 .2 forsome 2 a;b : CompositeTrapezoidalrule Let f 2 C 2 [ a;b ] ;h = b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a n ; and x j = a + jh for j =0 ; 1 ;:::;n: ThecompositeTrapezoidalrule for n subintervalsis Z b a f x dx = h 2 2 4 f a +2 n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 X j =1 f x j + f b 3 5 )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 12 h 2 f 00 .3

PAGE 129

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 128 forsome 2 a;b : CompositeSimpson'srule Let f 2 C 4 [ a;b ] ;n beeven, h = b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a n ; and x j = a + jh for j =0 ; 1 ;:::;n: Thecomposite Simpson'srulefor n subintervalsis Z b a f x dx = h 3 2 4 f a +2 n 2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 X j =1 f x 2 j +4 n= 2 X j =1 f x 2 j )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 + f b 3 5 )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 180 h 4 f .4 forsome 2 a;b : Exercise4.2-1: ShowthatthequadratureruleinExample74correspondstotaking n =4 inthecompositeSimpson'sformula4.4. Exercise4.2-2: Showthattheabsoluteerrorforthecompositetrapezoidalruledecaysat therateof 1 =n 2 ,andtheabsoluteerrorforthecompositeSimpson'sruledecaysattherateof 1 =n 4 , where n isthenumberofsubintervals. Example75. Determine n thatensuresthecompositeSimpson'sruleapproximates R 2 1 x log xdx withanabsoluteerrorofatmost 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 . Solution. TheerrortermforthecompositeSimpson'sruleis b )]TJ/F49 7.9701 Tf 6.586 0 Td [(a 180 h 4 f where issomenumber between a =1 and b =2 ,and h = b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a =n: Dierentiatetoget f x = 2 x 3 : Then b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a 180 h 4 f = 1 180 h 4 2 3 h 4 90 whereweusedthefactthat 2 3 2 1 =2 when 2 ; 2 .Nowmaketheupperboundlessthan 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 ,thatis, h 4 90 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 1 n 4 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 n 4 10 6 90 11111 : 11 whichimplies n 10 : 27 : Since n mustbeevenforSimpson'srule,thismeansthesmallestvalueof n toensureanerrorofatmost 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 is12. UsingthePythoncodeforthecompositeSimpson'srulethatwillbeintroducednext,weget 0.6362945608astheestimate,using10digits.Thecorrectintegralto10digitsis 0 : 6362943611 , whichmeansanabsoluteerrorof 2 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(7 ,betterthantheexpected 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 :

PAGE 130

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 129 PythoncodesforNewton-Cotesformulas WewritecodesforthetrapezoidalandSimpson'srules,andthecompositeSimpson'srule.Coding trapezoidalandSimpson'sruleisstraightforward. Trapezoidalrule In[1]:deftrapf,a,b: returnfa+fb*b-a/2 Let'sverifythecalculationsofExample72: In[2]:trapx->x^x,0.5,1 Out[2]:0.42677669529663687 Simpson'srule In[3]:defsimpsonf,a,b: returnfa+4*fa+b/2+fb*b-a/6 In[4]:simpsonlambdax:x**x,0.5,1 Out[4]:0.4109013813880978 RecallthatthedegreeofaccuracyofSimpson'sruleis3.Thismeanstheruleintegratespolynomials 1 ;x;x 2 ;x 3 exactly,butnot x 4 .Wecanusethisasawaytoverifyourcode: In[5]:simpsonlambdax:x,0,1 Out[5]:0.5 In[6]:simpsonlambdax:x**2,0,1 Out[6]:0.3333333333333333 In[7]:simpsonlambdax:x**3,0,1 Out[7]:0.25 In[8]:simpsonlambdax:x**4,0,1 Out[8]:0.20833333333333334

PAGE 131

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 130 CompositeSimpson'srule NextwecodethecompositeSimpson'srule,andverifytheresultofExample75. In[9]:defcompsimpsonf,a,b,n: h=b-a/n nodes=np.zerosn+1 foriinrangen+1: nodes[i]=a+i*h sum=fa+fb foriinrange2,n-1,2: sum+=2*fnodes[i] foriinrange1,n,2: sum+=4*fnodes[i] returnsum*h/3 In[10]:compsimpsonlambdax:x*np.logx,1,2,12 Out[10]:0.636294560831306 Exercise4.2-3: Determinethevalueof n requiredtoapproximate Z 2 0 1 x +1 dx towithin 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(4 ,andcomputetheapproximation,usingthecompositetrapezoidalandcomposite Simpson'srule. Compositerulesandroundoerror Asweincrease n inthecompositerulestolowererror,thenumberoffunctionevaluationsincreases, andanaturalquestiontoaskwouldbewhetherroundoerrorcouldaccumulateandcauseproblems. Somewhatremarkably,theanswerisno.Let'sassumetheroundoerrorassociatedwithcomputing f x isboundedforall x; bysomepositiveconstant .Andlet'strytocomputetheroundoerror incompositeSimpsonrule.Sinceeachfunctionevaluationinthecompositeruleincorporatesan errorofatmost ,thetotalerrorisboundedby h 3 2 4 +2 n 2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 X j =1 +4 n= 2 X j =1 + 3 5 h 3 h +2 n 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 +4 n 2 + i = h 3 n = hn:

PAGE 132

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 131 However,since h = b )]TJ/F82 10.9091 Tf 9.886 0 Td [(a =n ,theboundsimpliesas hn = b )]TJ/F82 10.9091 Tf 9.887 0 Td [(a : Thereforenomatterhowlarge n is,thatis,howlargethenumberoffunctionevaluationsis,theroundoerrorisboundedbythe sameconstant b )]TJ/F82 10.9091 Tf 10.91 0 Td [(a whichonlydependsonthesizeoftheinterval. Exercise4.2-4: Thisproblemshowsthatnumericalquadratureisstablewithrespectto errorinfunctionvalues.Assumethefunctionvalues f x i areapproximatedby ~ f x i ,sothat j f x i )]TJ/F81 10.9091 Tf 13.466 2.879 Td [(~ f x i j < forany x i 2 a;b : Findanupperboundontheerrorofnumericalquadrature P w i f x i whenitisactuallycomputedas P w i ~ f x i : 4.3Gaussianquadrature Newton-Cotesformulaswereobtainedbyintegratinginterpolatingpolynomialswithequally-spaced nodes.Theequalspacingisconvenientinderivingsimpleexpressionsforthecompositerules. However,thisplacementofnodesisnotnecessarilytheoptimalplacement.Forexample,the trapezoidalruleapproximatestheintegralbyintegratingalinearfunctionthatjoinstheendpoints ofthefunction.Thefactthatthisisnottheoptimalchoicecanbeseenbysketchingasimple parabola. TheideaofGaussianquadratureisthefollowing:inthenumericalquadraturerule Z b a f x dx n X i =1 w i f x i choose x i and w i insuchawaythatthequadraturerulehasthehighestpossibleaccuracy.Note thatunlikeNewton-Cotesformulaswherewestartedlabelingthenodeswith x 0 ,intheGaussian quadraturetherstnodeis x 1 .Thisdierenceinnotationiscommonintheliterature,andeach choicemakesthesubsequentequationsinthecorrespondingtheoryeasiertoread. Example76. Let a;b = )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1 ; and n =2 : Findthebest x i and w i . Therearefourparameterstodetermine: x 1 ;x 2 ;w 1 ;w 2 : Weneedfourconstraints.Let'srequire theruletointegratethefollowingfunctionsexactly: f x =1 ;f x = x;f x = x 2 ,and f x = x 3 : Iftheruleintegrates f x =1 exactly,then R 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 dx = P 2 i =1 w i ,i.e., w 1 + w 2 =2 : Iftherule integrates f x = x exactly,then R 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 xdx = P 2 i =1 w i x i ,i.e., w 1 x 1 + w 2 x 2 =0 : Continuingthisfor

PAGE 133

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 132 f x = x 2 ,and f x = x 3 ; weobtainthefollowingequations: w 1 + w 2 =2 w 1 x 1 + w 2 x 2 =0 w 1 x 2 1 + w 2 x 2 2 = 2 3 w 1 x 3 1 + w 2 x 3 2 =0 : Solvingtheequationsgives: w 1 = w 2 =1 ;x 1 = )]TJ 6.587 6.598 Td [(p 3 3 ;x 2 = p 3 3 : Thereforethequadratureruleis: Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 f x dx f )]TJ 8.485 9.025 Td [(p 3 3 ! + f p 3 3 ! : Observethat: Thetwonodesarenotequallyspacedon )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1 : Theaccuracyoftheruleisthree,anditusesonlytwonodes.Recallthattheaccuracyof Simpson'sruleisalsothreebutitusesthreenodes.IngeneralGaussianquadraturegivesa degreeofaccuracyof 2 n )]TJ/F81 10.9091 Tf 10.91 0 Td [(1 usingonly n nodes. Wewereabletosolveforthenodesandweightsinthesimpleexampleabove,however,as thenumberofnodesincreases,theresultingnon-linearsystemofequationswillbeverydiculttosolve.Thereisanalternativeapproachusingthetheoryoforthogonalpolynomials,a topicwewilldiscussindetaillater.Here,wewilluseaparticularsetoforthogonalpolynomials f L 0 x ;L 1 x ;:::;L n x ;::: g calledLegendrepolynomials.Wewillgiveadenitionofthesepolynomialsinthenextchapter.Forthisdiscussion,wejustneedthefollowingpropertiesofthese polynomials: L n x isamonicpolynomialofdegree n foreach n . R 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 P x L n x dx =0 foranypolynomial P x ofdegreelessthan n . TherstfewLegendrepolynomialsare L 0 x =1 ;L 1 x = x;L 2 x = x 2 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(1 3 ;L 3 x = x 3 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(3 5 x;L 4 x = x 4 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(6 7 x 2 + 3 35 : HowdothesepolynomialshelpwithndingthenodesandtheweightsoftheGaussianquadrature rule?Theanswerisshort:the roots oftheLegendrepolynomialsarethe nodes ofthequadrature rule! Tosummarize,the Gauss-Legendrequadraturerule fortheintegralof f over )]TJ/F81 10.9091 Tf 8.484 0 Td [(1 ; 1 is Z 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 f x dx = n X i =1 w i f x i

PAGE 134

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 133 where x 1 ;x 2 ;:::;x n aretherootsofthe n thLegendrepolynomial,andtheweightsarecomputed usingthefollowingtheorem. Theorem77. Supposethat x 1 ;x 2 ;:::;x n aretherootsofthe n thLegendrepolynomial L n x and theweightsaregivenby w i = Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 n Y j =1 ;j 6 = i x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j dx: ThentheGauss-Legendrequadraturerulehasdegreeofaccuracy 2 n )]TJ/F81 10.9091 Tf 10.833 0 Td [(1 .Inotherwords,if P x is anypolynomialofdegreelessthanorequalto 2 n )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 ,then Z 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 P x dx = n X i =1 w i P x i : Proof. Let'sstartwithapolynomial P x withdegreelessthan n .ConstructtheLagrangeinterpolantfor P x ; usingthenodesas x 1 ;:::;x n P x = n X i =1 P x i l i x = n X i =1 n Y j =1 ;j 6 = i x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j P x i : Thereisnoerrortermabovebecausetheerrortermdependsonthe n thderivativeof P x ,but P x isapolynomialofdegreelessthan n ,sothatderivativeiszero.Integratebothsidestoget Z 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 P x dx = Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 2 4 n X i =1 n Y j =1 ;j 6 = i x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j P x i 3 5 dx = Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 2 4 n X i =1 P x i n Y j =1 ;j 6 = i x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j 3 5 dx = n X i =1 2 4 Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 P x i n Y j =1 ;j 6 = i x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j dx 3 5 = n X i =1 2 4 Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 n Y j =1 ;j 6 = i x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j x i )]TJ/F82 10.9091 Tf 10.909 0 Td [(x j dx 3 5 P x i = n X i =1 w i P x i : Thereforethetheoremiscorrectforpolynomialsofdegreelessthan n .Nowlet P x beapolynomial ofdegreegreaterthanorequalto n ,butlessthanorequalto 2 n )]TJ/F81 10.9091 Tf 10.526 0 Td [(1 : Divide P x bytheLegendre polynomial L n x toget P x = Q x L n x + R x : Notethat P x i = Q x i L n x i + R x i = R x i since L n x i =0 ,for i =1 ; 2 ;:::;n: Alsoobservethefollowingfacts:

PAGE 135

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 134 1. R 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 Q x L n x dx =0 ; since Q x isapolynomialofdegreelessthan n; andfromthesecond propertyofLegendrepolynomials. 2. R 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 R x dx = P n i =1 w i R x i ; since R x isapolynomialofdegreelessthan n; andfromthe rstpartoftheproof. Puttingthesefactstogetherweget: Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 P x dx = Z 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 [ Q x L n x + R x ] dx = Z 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 R x dx = n X i =1 w i R x i = n X i =1 w i P x i : Table4.1displaystherootsoftheLegendrepolynomials L 2 ;L 3 ;L 4 ;L 5 andthecorresponding weights. n RootsWeights 2 1 p 3 =0 : 5773502692 1 )]TJ/F46 7.9701 Tf 14.022 4.708 Td [(1 p 3 = )]TJ/F15 11.9552 Tf 9.298 0 Td [(0 : 5773502692 1 3 )]TJ/F15 11.9552 Tf 9.298 0 Td [( 3 5 1 = 2 = )]TJ/F15 11.9552 Tf 9.298 0 Td [(0 : 7745966692 5 9 =0 : 5555555556 0.0 8 9 =0 : 8888888889 3 5 1 = 2 =0 : 7745966692 5 9 =0 : 5555555556 40.86113631160.3478548451 0.33998104360.6521451549 -0.33998104360.6521451549 -0.86113631160.3478548451 50.90617984590.2369268850 0.53846931010.4786286705 0.00.5688888889 -0.53846931010.4786286705 -0.90617984590.2369268850 Table4.1:RootsofLegendrepolynomials L 2 through L 5

PAGE 136

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 135 Example78. Approximate R 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 cos xdx usingGauss-Legendrequadraturewith n =3 nodes. Solution. FromTable4.1,andusingtwo-digitrounding,wehave Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 cos xdx 0 : 56cos )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 77+0 : 89cos0+0 : 56cos : 77=1 : 69 andthetruesolutionis sin )]TJ/F81 10.9091 Tf 10.909 0 Td [(sin )]TJ/F81 10.9091 Tf 8.485 0 Td [(1=1 : 68 : Sofarwediscussedintegratingfunctionsovertheinterval )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1 : Whatifwehaveadierent integrationdomain?Theanswerissimple:changeofvariables!Tocompute R b a f x dx forany a
PAGE 137

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 136 Nowwecompute 1 4 R 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 )]TJ/F49 7.9701 Tf 6.195 -4.541 Td [(t +3 4 t +3 4 dt usingthecode: In[2]:0.25*gausslambdat:t/4+3/4**t/4+3/4 Out[2]:0.41081564812239885 ThenexttheoremisabouttheerroroftheGauss-Legendrerule.Itsproofcanbefoundin Atkinson[3].Thetheoremshows,inparticular,thatthedegreeofaccuracyofthequadraturerule, using n nodes,is 2 n )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 . Theorem80. Let f 2 C 2 n [ )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1] : TheerrorofGauss-Legendrerulesatises Z b a f x dx )]TJ/F49 7.9701 Tf 16.219 13.636 Td [(n X i =1 w i f x i = 2 2 n +1 n ! 4 n +1[ n !] 2 f n n ! forsome 2 )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1 . UsingStirling'sformula n ! e )]TJ/F49 7.9701 Tf 6.586 0 Td [(n n n n 1 = 2 ,wherethesymbol meanstheratioofthetwo sidesconvergesto1as n !1 ,itcanbeshownthat 2 2 n +1 n ! 4 n +1[ n !] 2 4 n : ThismeanstheerrorofGauss-Legendreruledecaysatanexponentialrateof 1 = 4 n asopposedto, forexample,thepolynomialrateof 1 =n 4 forcompositeSimpson'srule. Exercise4.3-1: ProvethatthesumoftheweightsinGauss-Legendrequadratureis2,for any n: Exercise4.3-2: Approximate R 1 : 5 1 x 2 log xdx usingGauss-Legendrerulewith n =2 and n =3 .Comparetheapproximationstotheexactvalueoftheintegral. Exercise4.3-3: AcompositeGauss-LegendrerulecanbeobtainedsimilartocompositeNewton-Cotesformulas.Consider R 2 0 e x dx .Dividetheinterval ; 2 intotwosubintervals ; 1 ; ; 2 andapplytheGauss-Legendrerulewithtwo-nodestoeachsubinterval.Comparethe estimatewiththeestimateobtainedfromtheGauss-Legendrerulewithfour-nodesappliedtothe wholeinterval ; 2 .

PAGE 138

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 137 4.4Multipleintegrals Thenumericalquadraturemethodswehavediscussedcanbegeneralizedtohigherdimensional integrals.Wewillconsiderthetwo-dimensionalintegral ZZ R f x;y dA: Thedomain R determinesthedicultyingeneralizingtheone-dimensionalformulaswelearned before.Thesimplestcasewouldbearectangulardomain R = f x;y j a x b;c y d g .We canthenwritethedoubleintegralastheiteratedintegral ZZ R f x;y dA = Z b a Z d c f x;y dy dx: Consideranumericalquadraturerule Z b a f x dx n X i =1 w i f x i : Applytheruleusing n 2 nodestotheinnerintegraltogettheapproximation Z b a 0 @ n 2 X j =1 w j f x;y j 1 A dx wherethe y j 'sarethenodes.Rewrite,byinterchangingtheintegralandsummation,toget n 2 X j =1 w j Z b a f x;y j dx andapplythequadratureruleagain,using n 1 nodes,togettheapproximation n 2 X j =1 w j n 1 X i =1 w i f x i ;y j ! : Thisgivesthetwo-dimensionalrule Z b a Z d c f x;y dy dx n 2 X j =1 n 1 X i =1 w i w j f x i ;y j : Forsimplicity,weignoredtheerrortermintheabovederivation;however,itsinclusionisstraightforward.

PAGE 139

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 138 Foranexample,let'sderivethetwo-dimensionalGauss-Legendrerulefortheintegral Z 1 0 Z 1 0 f x;y dydx .5 usingtwonodesforeachaxis.Notethateachintegralhastobetransformedto )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1 : Startwith theinnerintegral R 1 0 f x;y dy anduse t =2 y )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 ;dt =2 dy totransformitto 1 2 Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 f x; t +1 2 dt andapplyGauss-Legendrerulewithtwonodestogettheapproximation 1 2 f x; )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 = p 3+1 2 ! + f x; 1 = p 3+1 2 !! : Substitutethisapproximationin4.5fortheinnerintegraltoget Z 1 0 1 2 f x; )]TJ/F81 10.9091 Tf 8.484 0 Td [(1 = p 3+1 2 ! + f x; 1 = p 3+1 2 !! dx: Nowtransformthisintegraltothedomain )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1 using s =2 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 ;ds =2 dx toget 1 4 Z 1 0 f s +1 2 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 = p 3+1 2 ! + f s +1 2 ; 1 = p 3+1 2 !! ds: ApplytheGauss-Legendreruleagaintoget 1 4 " f )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 = p 3+1 2 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 = p 3+1 2 ! + f )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 = p 3+1 2 ; 1 = p 3+1 2 ! + f 1 = p 3+1 2 ; )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 = p 3+1 2 ! + f 1 = p 3+1 2 ; 1 = p 3+1 2 !# : .6 Figure4.1displaysthenodesusedinthiscalculation. Nextwederivethetwo-dimensionalSimpson'sruleforthesameintegral, R 1 0 R 1 0 f x;y dydx; using n =2 ,whichcorrespondstothreenodesintheSimpson'srulerecallthat n isthenumberof nodesinGauss-Legendrerule,but n +1 isthenumberofnodesinNewton-Cotesformulas.

PAGE 140

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 139 Figure4.1:NodesofdoubleGauss-Legendrerule Theinnerintegralisapproximatedas Z 1 0 f x;y dy 1 6 f x; 0+4 f x; 0 : 5+ f x; 1 : Substitutethisapproximationfortheinnerintegralin R 1 0 R 1 0 f x;y dy dx toget 1 6 Z 1 0 f x; 0+4 f x; 0 : 5+ f x; 1 dx: ApplySimpson'sruleagaintothisintegralwith n =2 toobtainthenalapproximation: 1 6 " 1 6 f ; 0+4 f ; 0 : 5+ f ; 1+4 f : 5 ; 0+4 f : 5 ; 0 : 5+ f : 5 ; 1 + f ; 0+4 f ; 0 : 5+ f ; 1 # : .7 Figure4.2displaysthenodesusedintheabovecalculation. Figure4.2:NodesofdoubleSimpson'srule

PAGE 141

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 140 Foraspecicexample,considertheintegral Z 1 0 Z 1 0 2 sin x 2 sin y dydx: Thisintegralcanbeevaluatedexactly,anditsvalueis1.Itisusedasatestintegralfornumerical quadraturerules.Evaluatingequations4.6and4.7with f x;y = )]TJ/F49 7.9701 Tf 6.195 -4.541 Td [( 2 sin x )]TJ/F49 7.9701 Tf 13.013 -4.541 Td [( 2 sin y ,we obtaintheapproximationsgiveninthetablebelow: Simpson'srulenodesGauss-LegendrenodesExactintegral 1.09660.936851 TheGauss-LegendrerulegivesaslightlybetterestimatethanSimpson'srule,butusinglessthan halfthenumberofnodes. Theapproachwehavediscussedcanbeextendedtoregionsthatarenotrectangular,andto higherdimensions.Moredetails,includingalgorithmsfordoubleandtripleintegralsusingSimpson's andGauss-Legendrerule,canbefoundinBurden,Faires,Burden[4]. Thereishowever,anobviousdisadvantageofthewaywehavegeneralizedone-dimensional quadraturerulestohigherdimensions.Imagineanumericalintegrationproblemwherethedimensionis360;suchhighdimensionsappearinsomeproblemsfromnancialengineering.Evenifwe usedtwonodesforeachdimension,thetotalnumberofnodeswouldbe 2 360 ,whichisabout 10 100 ; anumbertoolarge.Inverylargedimensions,theonlygeneralmethodfornumericalintegrationis theMonteCarlomethod.InMonteCarlo,wegeneratepseudorandomnumbersfromthedomain, andevaluatethefunctionaverageatthosepoints.Inotherwords, Z R f x d x 1 n n X i =1 f x i where x i arepseudorandomvectorsuniformlydistributedin R: Forthetwo-dimensionalintegral wediscussedbefore,theMonteCarloestimateis Z b a Z d c f x;y dydx b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a d )]TJ/F82 10.9091 Tf 10.91 0 Td [(c n n X i =1 f a + b )]TJ/F82 10.9091 Tf 10.909 0 Td [(a x i ;c + d )]TJ/F82 10.9091 Tf 10.909 0 Td [(c y i .8 where x i ;y i arepseudorandomnumbersfrom ; 1 : InPython,thefunction np.random.rand generatesapseudorandomnumberfromtheuniformdistributionbetween0and1.Thefollowing codetakestheendpointsoftheintervals a;b;c;d ,andthenumberofnodes n ,whichiscalledthe samplesizeintheMonteCarloliterature,andreturnstheestimatefortheintegralusingEquation 4.8. In[1]:importnumpyasnp In[2]:defmcf,a,b,c,d,n:

PAGE 142

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 141 sum=0. foriinrangen: sum+=fa+b-a*np.random.rand, c+d-c*np.random.rand*b-a*d-c returnsum/n NowweuseMonteCarlotoestimatetheintegral R 1 0 R 1 0 )]TJ/F49 7.9701 Tf 6.195 -4.542 Td [( 2 sin x )]TJ/F49 7.9701 Tf 13.013 -4.542 Td [( 2 sin y dydx : In[2]:mcx,y->pi^2/4*sinpi*x*sinpi*y,0,1,0,1,500 Out[2]:0.9441778334708931 With n =500 ,weobtainaMonteCarloestimateof0.944178.AnadvantageoftheMonte Carlomethodisitssimplicity:theabovecodecanbeeasilygeneralizedtohigherdimensions.A disadvantageofMonteCarloisitsslowrateofconvergence,whichis O = p n .Figure4.3displays 500pseudorandomvectorsfromtheunitsquare. Figure4.3:MonteCarlo:500pseudorandomvectors Example81. Capstick&Keister[5]discusssomehighdimensionaltestintegrals,somewithanalyticalsolutions,motivatedbyphysicalapplicationssuchasthecalculationofquantummechanical matrixelementsinatomic,nuclear,andparticlephysics.Oneoftheintegralswithaknownsolution is Z R s cos k t k e k t k 2 dt 1 dt 2 dt s where k t k = t 2 1 + ::: + t 2 s 1 = 2 .Thisintegralcanbetransformedtoanintegraloverthe s -dimensional

PAGE 143

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 142 unitcubeas s= 2 Z ; 1 s cos " F )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x 1 2 + ::: + F )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x s 2 2 1 = 2 # dx 1 dx 2 dx s .9 where F )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 istheinverseofthecumulativedistributionfunctionofthestandardnormaldistribution: F x = 1 1 = 2 Z x e )]TJ/F49 7.9701 Tf 6.586 0 Td [(s 2 = 2 ds: Wewillestimatetheintegral4.9byMonteCarloas s= 2 n n X i =1 cos 2 4 F )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x i 1 2 + ::: + F )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x i s 2 2 ! 1 = 2 3 5 where x i = x i 1 ;:::;x i s isan s -dimensionalvectorofuniformrandomnumbersbetween0and 1. In[1]:importnumpyasnp In[2]:importmatplotlib.pyplotasplt %matplotlibinline Thefollowingalgorithm,knownastheBeasley-Springer-Moroalgorithm[8],givesanapproximationto F )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 x . In[3]:definvNormalu: #Beasley-Springer-Moroalgorithm a0=2.50662823884 a1=-18.61500062529 a2=41.39119773534 a3=-25.44106049637 b0=-8.47351093090 b1=23.08336743743 b2=-21.06224101826 b3=3.13082909833 c0=0.3374754822726147 c1=0.9761690190917186 c2=0.1607979714918209 c3=0.0276438810333863 c4=0.0038405729373609 c5=0.0003951896511919

PAGE 144

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 143 c6=0.0000321767881768 c7=0.0000002888167364 c8=0.0000003960315187 y=u-0.5 ifnp.absy<0.42: r=y*y x=y*a3*r+a2*r+a1*r+a0/b3*r+b2*r+b1*r+b0*r+1 else: r=u ify>0: r=1-u r=np.log-np.logr x=c0+r*c1+r*c2+r*c3+r*c4+r*c5+r*c6+r*c7+r*c8 ify<0: x=-x returnx ThefollowingistheMonteCarloestimateoftheintegral.Ittakesthedimension s andthe samplesize n asinputs. In[4]:defmcs,n: est=0 forjinrangen: sum=0 foriinranges: sum+=invNormalnp.random.rand**2 est+=np.cossum/2**0.5 returnnp.pi**s/2*est/n Theexactvalueoftheintegralfor s =25 is 1 : 356914 10 6 .Thefollowingcodecomputesthe relativeerroroftheMonteCarloestimatewithsamplesize n . In[5]:relerror=lambdan:np.absmc25,n+1.356914*10**6/1.356914*10**6 Let'splottherelativeerrorofsomeMonteCarloestimates.First,wegeneratesamplesizesfrom 50,000to1,000,000inincrementsof50,000. In[6]:samples=[nforninrange50000,1000001,50000] Foreachsamplesize,wecomputetherelativeerror,andthenplottheresults.

PAGE 145

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 144 In[7]:error=[relerrornforninsamples] In[8]:plt.plotsamples,error plt.xlabel ' Samplesizen ' plt.ylabel ' Relativeerror ' ; Figure4.4:MonteCarlorelativeerrorfortheintegral4.9 4.5Improperintegrals Thequadratureruleswehavelearnedsofarcannotbeappliedorappliedwithapoorperformance tointegralssuchas R b a f x dx if a;b = orif a;b arenitebut f isnotcontinuousatoneor bothoftheendpoints:recallthatbothNewton-CotesandGauss-Legendreerrorboundtheorems requiretheintegrandtohaveanumberofcontinuousderivativesontheclosedinterval [ a;b ] .For example,anintegralintheform Z 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 f x p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 dx clearlycannotbeapproximatedusingthetrapezoidalorSimpson'srulewithoutanymodications, sincebothrulesrequirethevaluesoftheintegrandattheendpointswhichdonotexist.Onecould tryusingtheGauss-Legendrerule,butthefactthattheintegranddoesnotsatisfythesmoothness conditionsrequiredbytheGauss-Legendreerrorboundmeanstheerroroftheapproximationmight belarge. Asimpleremedytotheproblemofimproperintegralsistochangethevariableofintegration andtransformtheintegral,ifpossible,toonethatbehaveswell. Example82. Considerthepreviousintegral R 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 f x p 1 )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 2 dx: Trythetransformation =cos )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 x:

PAGE 146

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 145 Then d = )]TJ/F82 10.9091 Tf 8.485 0 Td [(dx= p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 and Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 f x p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 dx = )]TJ/F54 10.9091 Tf 10.303 14.849 Td [(Z 0 f cos d = Z 0 f cos d: Thelatterintegralcanbeevaluatedusing,forexample,Simpson'srule,provided f issmoothon [0 ; ] : Iftheintervalofintegrationisinnite,anotherapproachthatmightworkistruncationofthe intervaltoaniteone.Thesuccessofthisapproachdependsonwhetherwecanestimatethe resultingerror. Example83. Considertheimproperintegral R 1 0 e )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 2 dx .Writetheintegralas Z 1 0 e )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 2 dx = Z t 0 e )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 2 dx + Z 1 t e )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 2 dx; where t isthe"leveloftruncation"todetermine.Wecanestimatetherstintegralontherighthandsideusingaquadraturerule.Thesecondintegralontheright-handsideistheerrordueto approximating R 1 0 e )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 2 dx by R t 0 e )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 2 dx .Anupperboundfortheerrorcanbefoundeasilyforthis example:notethatwhen x t , x 2 = xx tx ,thus Z 1 t e )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 2 dx Z 1 t e )]TJ/F49 7.9701 Tf 6.586 0 Td [(tx dx = e )]TJ/F49 7.9701 Tf 6.587 0 Td [(t 2 =t: When t =5 , e )]TJ/F49 7.9701 Tf 6.587 0 Td [(t 2 =t 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(12 ,therefore,approximatingtheintegral R 1 0 e )]TJ/F49 7.9701 Tf 6.586 0 Td [(x 2 dx by R 5 0 e )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 2 dx willbe accuratewithin 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(12 .Additionalerrorwillcomefromestimatingthelatterintegralbynumerical quadrature. 4.6Numericaldierentiation Thederivativeof f at x 0 is f 0 x 0 =lim h ! 0 f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 h : Thisformulagivesanobviouswaytoestimatethederivativeby f 0 x 0 f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 h forsmall h .Whatthisformulalacks,however,isitdoesnotgiveanyinformationabouttheerror oftheapproximation. Wewilltryanotherapproach.SimilartoNewton-Cotesquadrature,wewillconstructthe interpolatingpolynomialfor f ,andthenusethederivativeofthepolynomialasanapproximation forthederivativeof f .

PAGE 147

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 146 Let'sassume f 2 C 2 a;b ;x 0 2 a;b ; and x 0 + h 2 a;b : ConstructthelinearLagrange interpolatingpolynomial p 1 x forthedata x 0 ;f x 0 ; x 1 ;f x 1 = x 0 + h;f x 0 + h .From Theorem51,wehave f x = x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 f x 0 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 f x 1 | {z } p 1 x + f 00 x 2! x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 | {z } interpolationerror = x )]TJ/F81 10.9091 Tf 10.909 0 Td [( x 0 + h x 0 )]TJ/F81 10.9091 Tf 10.909 0 Td [( x 0 + h f x 0 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 f x 0 + h + f 00 x 2! x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h = x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h )]TJ/F82 10.9091 Tf 8.485 0 Td [(h f x 0 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 h f x 0 + h + f 00 x 2! x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h : Nowlet'sdierentiate f x : f 0 x = )]TJ/F82 10.9091 Tf 9.68 7.38 Td [(f x 0 h + f x 0 + h h + f 00 x d dx x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h 2 + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h 2 d dx f 00 x = f x 0 + h )]TJ/F82 10.9091 Tf 10.91 0 Td [(f x 0 h + 2 x )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h 2 f 00 x + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h 2 f 000 x 0 x : Intheaboveequation,weknow isbetween x 0 and x 0 + h ;however,wehavenoknowledgeabout 0 x ,whichappearsinthelastterm.Fortunately,ifweset x = x 0 ; thetermwith 0 x vanishes andweget: f 0 x 0 = f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 h )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(h 2 f 00 x 0 : Thisformulaiscalledthe forward-dierence formulaif h> 0 and backward-dierence formula if h< 0 : Notethatfromthisformulawecanobtainaboundontheerror f 0 x 0 )]TJ/F82 10.9091 Tf 12.104 7.38 Td [(f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 h h 2 sup x 2 x 0 ;x 0 + h f 00 x : Toobtaintheforward-dierenceandbackward-dierenceformulaswestartedwithalinear polynomialinterpolantontwopoints.Usingmorepointsandahigherorderinterpolantgivesmore accuracy,butalsoincreasesthecomputingtimeandroundoerror.Ingeneral,let f 2 C n +1 [ a;b ] and x 0 ;x 1 ;:::;x n aredistinctnumbersin [ a;b ] .Wehave f x = n X k =0 f x k l k x + f n +1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n n +1! f 0 x = n X k =0 f x k l 0 k x + f n +1 d dx x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n n +1! + x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n n +1! d dx f n +1 :

PAGE 148

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 147 If x = x j for j =0 ; 1 ;:::;n; thelasttermvanishes,andusingthefollowingresult d dx [ x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x n ] x = x j = n Y k =0 ;k 6 = j x j )]TJ/F82 10.9091 Tf 10.909 0 Td [(x k weobtain f 0 x j = n X k =0 f x k l 0 k x j + f n +1 x j n +1! n Y k =0 ;k 6 = j x j )]TJ/F82 10.9091 Tf 10.909 0 Td [(x k .10 whichiscalledthe n +1 -pointformulatoapproximate f 0 x j : Themostcommonformulasuse n =2 and n =4 : Herewediscuss n =2 ,thatis,three-pointformulas.Thenodesare, x 0 ;x 1 ;x 2 . TheLagrangebasispolynomialsandtheirderivativesare: l 0 x = x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 l 0 0 x = 2 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 l 1 x = x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 l 0 1 x = 2 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 l 2 x = x )]TJ/F82 10.9091 Tf 10.91 0 Td [(x 0 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 l 0 2 x = 2 x )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 0 x 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 1 Thesederivativescanbesubstitutedin4.10toobtainthethree-pointformula.Wecansimplify theseformulasifthenodesarespacedequally,thatis, x 1 = x 0 + h;x 2 = x 1 + h = x 0 +2 h: Then, weobtain f 0 x 0 = 1 2 h [ )]TJ/F81 10.9091 Tf 8.485 0 Td [(3 f x 0 +4 f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 +2 h ]+ h 2 3 f 0 .11 f 0 x 0 + h = 1 2 h [ )]TJ/F82 10.9091 Tf 8.485 0 Td [(f x 0 + f x 0 +2 h ] )]TJ/F82 10.9091 Tf 12.105 7.381 Td [(h 2 6 f 1 .12 f 0 x 0 +2 h = 1 2 h [ f x 0 )]TJ/F81 10.9091 Tf 10.909 0 Td [(4 f x 0 + h +3 f x 0 +2 h ]+ h 2 3 f 2 : .13 Itturnsoutthattherstandthirdequations4.11and4.13areequivalent.Toseethis, rstsubstitute x 0 by x 0 )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 h inthethirdequationtogetignoringtheerrorterm f 0 x 0 = 1 2 h [ f x 0 )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 h )]TJ/F81 10.9091 Tf 10.909 0 Td [(4 f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h +3 f x 0 ] ; andthenset h to )]TJ/F82 10.9091 Tf 8.485 0 Td [(h intheright-handsidetoget 1 2 h [ )]TJ/F82 10.9091 Tf 8.485 0 Td [(f x 0 +2 h +4 f x 0 + h )]TJ/F81 10.9091 Tf 10.909 0 Td [(3 f x 0 ] ,which givesustherstequation. Thereforewehaveonlytwodistinctequations,4.11and4.12.Werewritetheseequations belowwithonemodication:in4.12,wesubstitute x 0 by x 0 )]TJ/F82 10.9091 Tf 11.165 0 Td [(h .Wethenobtaintwodierent

PAGE 149

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 148 formulasfor f 0 x 0 : f 0 x 0 = )]TJ/F81 10.9091 Tf 8.485 0 Td [(3 f x 0 +4 f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 +2 h 2 h + h 2 3 f 0 ! three-pointendpointformula f 0 x 0 = f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h 2 h )]TJ/F82 10.9091 Tf 12.104 7.38 Td [(h 2 6 f 1 ! three-pointmidpointformula Thethree-pointmidpointformulahassomeadvantages:ithashalftheerroroftheendpointformula, andithasonelessfunctionevaluation.Theendpointformulaisusefulifonedoesnotknowthe valueof f ononeside,asituationthatmayhappenif x 0 isclosetoanendpoint. Example84. Thefollowingtablegivesthevaluesof f x =sin x: Estimate f 0 : 1 ;f 0 : 3 using anappropriatethree-pointformula. xf x 0.10.09983 0.20.19867 0.30.29552 0.40.38942 Solution. Toestimate f 0 : 1 ; weset x 0 =0 : 1 ; and h =0 : 1 : Notethatwecanonlyusethe three-pointendpointformula. f 0 : 1 1 0 : 2 )]TJ/F81 10.9091 Tf 8.485 0 Td [(3 : 09983+4 : 19867 )]TJ/F81 10.9091 Tf 10.91 0 Td [(0 : 29552=0 : 99835 : Thecorrectansweris cos0 : 1=0 : 995004 : Toestimate f 0 : 3 wecanusethemidpointformula: f 0 : 3 1 0 : 2 : 38942 )]TJ/F81 10.9091 Tf 10.909 0 Td [(0 : 19867=0 : 95375 : Thecorrectansweris cos0 : 3=0 : 955336 andthustheabsoluteerroris 1 : 59 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(3 : Ifweusethe endpointformulatoestimate f 0 : 3 weset h = )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 1 andcompute f 0 : 3 1 )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 2 )]TJ/F81 10.9091 Tf 8.485 0 Td [(3 : 29552+4 : 19867 )]TJ/F81 10.9091 Tf 10.909 0 Td [(0 : 09983=0 : 95855 withanabsoluteerrorof 3 : 2 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(3 : Exercise4.6-1: Insomeapplications,wewanttoestimatethederivativeofanunknown functionfromempiricaldata.However,empiricaldatausuallycomewith"noise",thatis,error duetodatacollection,datareporting,orsomeotherreason.Inthisexercisewewillinvestigate howstablethedierenceformulasarewhenthereisnoiseinthedata.Considerthefollowing dataobtainedfrom y = e x .Thedataisexacttosixdigits.Weestimate f 0 : 01 usingthe

PAGE 150

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 149 x 1.001.011.02 f x 2.718282.745602.77319 three-pointmidpointformulaandobtain f 0 : 01= 2 : 77319 )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 : 71828 0 : 02 =2 : 7455 .Thetruevalueis f 0 : 01= e 1 : 01 =2 : 74560 ,andtherelativeerrorduetoroundingis 3 : 6 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(5 . Next,weaddsomenoisetothedata:weincrease2.77319by10%to3.050509,anddecrease 2.71828by10%to2.446452.Hereisthenoisydata: x 1.001.011.02 f x 2.446452 2.74560 3.050509 Estimate f 0 : 01 usingthenoisydata,andcomputeitsrelativeerror.Howdoestherelative errorcomparewiththerelativeerrorforthenon-noisydata? Wenextwanttoexplorehowtoestimatethesecondderivativeof f: Asimilarapproachto estimating f 0 canbetakenandthesecondderivativeoftheinterpolatingpolynomialcanbeused asanapproximation.Herewewilldiscussanotherapproach,usingTaylorexpansions.Expand f about x 0 ,andevaluateitat x 0 + h and x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h : f x 0 + h = f x 0 + hf 0 x 0 + h 2 2 f 00 x 0 + h 3 6 f x 0 + h 4 24 f + f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h = f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(hf 0 x 0 + h 2 2 f 00 x 0 )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(h 3 6 f x 0 + h 4 24 f )]TJ/F81 10.9091 Tf 7.085 1.636 Td [( where + isbetween x 0 and x 0 + h; and )]TJ/F79 10.9091 Tf 10.702 1.637 Td [(isbetween x 0 and x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h: Addtheequationstoget f x 0 + h + f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h =2 f x 0 + h 2 f 00 x 0 + h 4 24 h f + + f )]TJ/F81 10.9091 Tf 7.084 1.636 Td [( i : Solvingfor f 00 x 0 gives f 00 x 0 = f x 0 + h )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 f x 0 + f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h h 2 )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(h 2 24 h f + + f )]TJ/F81 10.9091 Tf 7.085 1.636 Td [( i : Notethat f + + f )]TJ/F46 7.9701 Tf 6.254 1.074 Td [( 2 isanumberbetween f + and f )]TJ/F81 10.9091 Tf 7.085 1.637 Td [( ,sofromtheIntermediate ValueTheorem6,wecanconcludethereexistssome between )]TJ/F79 10.9091 Tf 10.701 1.636 Td [(and + sothat f = f + + f )]TJ/F81 10.9091 Tf 7.085 1.636 Td [( 2 : Thentheaboveformulasimpliesas f 00 x 0 = f x 0 + h )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 f x 0 + f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h h 2 )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(h 2 12 f

PAGE 151

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 150 forsome between x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h and x 0 + h: Numericaldierentiationandroundoerror Aryaandthemysteriousblackbox Collegelifeisfullofmysteries,andAryafaces oneinanengineeringclass:ablackbox!Whatis ablackbox?Itisacomputerprogram,orsome device,whichproducesanoutputwhenaninput isprovided.Wedonotknowtheinnerworkings ofthesystem,andhencecomesthenameblack box.Let'sthinkoftheblackboxasafunction f ,andrepresenttheinputandoutputas x;f x . Ofcourse,wedonothaveaformulafor f . WhatArya'sengineeringclassmateswanttodoiscomputethederivativeinformationofthe blackbox,thatis, f 0 x ,when x =2 : Theinputtothisblackboxcanbeanyrealnumber. Studentswanttousethethree-pointmidpointformulatoestimate f 0 : f 0 1 2 h [ f + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f )]TJ/F82 10.9091 Tf 10.909 0 Td [(h ] : Theyarguehowtopick h inthisformula.Oneofthemsaystheyshouldmake h assmallaspossible, like 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(8 .Aryaisskeptical.Shemutterstoherself,"IknowIsleptthroughsomeofmynumerical analysislectures,butnotall!" Shetellsherclassmatesaboutthecancellationofleadingdigitsphenomenon,andtomakeher pointmoreconvincing,shemakesthefollowingexperiment:let f x = e x ,andsupposewewant

PAGE 152

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 151 tocompute f 0 whichis e 2 .Aryausesthethree-pointmidpointformulaabovetoestimate f 0 , forvariousvaluesof h ,andforeachcaseshecomputestheabsolutevalueofthedierencebetween thethree-pointmidpointestimateandtheexactsolution e 2 .Shetabulatesherresultsinthetable below.Clearly,smaller h doesnotresultinsmallererror.Inthisexperiment, h =10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 givesthe smallesterror. h 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(4 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(5 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(7 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(8 Abs.error 1 : 2 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(8 1 : 9 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(10 5 : 2 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(11 7 : 5 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(9 2 : 1 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(8 Theoreticalanalysis Numericaldierentiationisanumericallyunstableproblem.Toreducethetruncationerror,weneed todecrease h ,whichinturnincreasestheroundoerrorduetocancellationofsignicantdigitsin thefunctiondierencecalculation.Let e x denotetheroundoerrorincomputing f x sothat f x = ~ f x + e x ; where ~ f isthevaluecomputedbythecomputer.Considerthethree-point midpointformula: f 0 x 0 )]TJ/F81 10.9091 Tf 14.453 10.259 Td [(~ f x 0 + h )]TJ/F81 10.9091 Tf 13.257 2.879 Td [(~ f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h 2 h = f 0 x 0 )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(e x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 )]TJ/F82 10.9091 Tf 10.91 0 Td [(h + e x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h 2 h = f 0 x 0 )]TJ/F82 10.9091 Tf 12.105 7.38 Td [(f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h 2 h + e x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h )]TJ/F82 10.9091 Tf 10.909 0 Td [(e x 0 + h 2 h = )]TJ/F82 10.9091 Tf 9.68 7.38 Td [(h 2 6 f + e x 0 )]TJ/F82 10.9091 Tf 10.909 0 Td [(h )]TJ/F82 10.9091 Tf 10.909 0 Td [(e x 0 + h 2 h h 2 6 M + h whereweassumed j f j M and j e x j : Toreducethetruncationerror h 2 M= 6 onewould decrease h ,whichwouldthenresultinanincreaseintheroundoerror =h .Anoptimalvalue for h canbefoundwiththeseassumptionsusingCalculus:ndthevaluefor h thatminimizesthe function s h = Mh 2 6 + h : Theansweris h = 3 p 3 =M: Let'srevisitthetableAryapresentedwhere 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 wasfoundtobetheoptimalvaluefor h .The calculationsweredoneusingPython,whichreports15digitswhenaskedfor e 2 .Let'sassumeall thesedigitsarecorrect,andthuslet =10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(16 .Since f x = e x and e 2 7 : 4 ,let'stake M =7 : 4 . Then h = 3 p 3 =M = 3 p 3 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(16 = 7 : 4 3 : 4 10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(6 whichisingoodagreementwiththeoptimalvalue 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(6 ofArya'snumericalresults.

PAGE 153

CHAPTER4.NUMERICALQUADRATUREANDDIFFERENTIATION 152 Exercise4.6-2: Findtheoptimalvaluefor h thatwillminimizetheerrorfortheformula f 0 x 0 = f x 0 + h )]TJ/F82 10.9091 Tf 10.909 0 Td [(f x 0 h )]TJ/F82 10.9091 Tf 12.104 7.38 Td [(h 2 f 00 inthepresenceofroundoerror,usingtheapproachofSection4.6. aConsiderestimating f 0 where f x = x 2 usingtheaboveformula.Whatistheoptimal valuefor h forestimating f 0 ; assumingthattheroundoerrorisboundedby =10 )]TJ/F46 7.9701 Tf 6.586 0 Td [(16 whichisthemachineepsilon 2 )]TJ/F46 7.9701 Tf 6.587 0 Td [(53 inthe64-bitoatingpointrepresentation. bUsePythontocompute f 0 n = f +10 )]TJ/F49 7.9701 Tf 6.587 0 Td [(n )]TJ/F82 10.9091 Tf 10.909 0 Td [(f 10 )]TJ/F49 7.9701 Tf 6.587 0 Td [(n ; for n =1 ; 2 ;:::; 20 ; anddescribewhathappens. cDiscussyourndingsinpartsaandbandhowtheyrelatetoeachother. Exercise4.6-3: Thefunction R x 0 1 p 2 e )]TJ/F49 7.9701 Tf 6.587 0 Td [(t 2 = 2 dt isrelatedtothedistributionfunctionofthe standardnormalrandomvariable,averyimportantdistributioninprobabilityandstatistics.Often timeswewanttosolveequationslike Z x 0 1 p 2 e )]TJ/F49 7.9701 Tf 6.587 0 Td [(t 2 = 2 dt = z .14 for x ,where z issomerealnumberbetween0and1.ThiscanbedonebyusingNewton'smethod tosolvetheequation f x =0 where f x = Z x 0 1 p 2 e )]TJ/F49 7.9701 Tf 6.587 0 Td [(t 2 = 2 dt )]TJ/F82 10.9091 Tf 10.909 0 Td [(z: NotethatfromFundamentalTheoremofCalculus, f 0 x = 1 p 2 e )]TJ/F49 7.9701 Tf 6.587 0 Td [(x 2 = 2 : Newton'smethodwillrequire thecalculationof Z p k 0 1 p 2 e )]TJ/F49 7.9701 Tf 6.586 0 Td [(t 2 = 2 dt .15 where p k isaNewtoniterate.Thisintegralcanbecomputedusingnumericalquadrature.Write aPythoncodethattakes z asitsinput,andoutputs x; suchthatEquation4.14holds.Inyour code,usethePythoncodesforNewton'smethodandthecompositeSimpson'sruleyouweregiven inclass.ForNewton'smethodsettoleranceto 10 )]TJ/F46 7.9701 Tf 6.587 0 Td [(5 and p 0 =0 : 5 ; andforcompositeSimpson'srule take n =10 ,whencomputingtheintegrals4.15thatappearinNewtoniteration.Thenrunyour codefor z =0 : 4 and z =0 : 1 ,andreportyouroutput.

PAGE 154

Chapter5 ApproximationTheory 5.1Discreteleastsquares Arya'sadventuresinthephysicslab Collegelifeisexpensive,andAryaishappytolandajobworkingataphysicslabforsomeextra cash.Shedoessomeexperiments,somedataanalysis,andalittlegrading.Inoneexperimentshe conducted,wherethereisoneindependentvariable x ,andonedependentvariable y ,shewasasked toplot y against x values.Thereareatotalofsixdatapoints.Shegetsthefollowingplot: Figure5.1:Scatterplotofdata Arya'sprofessorthinkstherelationshipbetweenthevariablesshouldbelinear,butwedonotsee datafallingonaperfectlinebecauseofmeasurementerror.Theprofessorisnothappy,professors areusuallynothappywhenlabresultsactup,andasksAryatocomeupwithalinearformula, 153

PAGE 155

CHAPTER5.APPROXIMATIONTHEORY 154 somethinglike y = ax + b; toexplaintherelationship.Aryarstthinksaboutinterpolation,but quicklyrealizesitisnotagoodidea.Why?.Let'shelpAryawithherproblem. Analysisoftheproblem Let'strypassingalinethroughthedatapoints.Figure5.2plotsonesuchline, y =3 x )]TJ/F81 10.9091 Tf 11.455 0 Td [(0 : 5 , togetherwiththedatapoints. Figure5.2:Datawithanapproximatingline Therearecertainlymanyotherchoiceswehavefortheline:wecouldincreaseordecreasethe slopealittle,changetheinterceptabit,andobtainmultiplelinesthathaveavisuallygoodtto thedata.Thecrucialquestionis,howcanwedecidewhichlineisthe"best"line,amongallthe possiblelines?Ifwecanquantifyhowgoodthetofagivenlineistothedata,andcomeupwith anotionforerror,perhapsthenwecanndthelinethatminimizesthiserror. Let'sgeneralizetheproblemalittle.Wehave: Data: x 1 ;y 1 ; x 2 ;y 2 ;:::; x m ;y m andwewanttondalinethatgivesthebestapproximationtothedata: Linearapproximation: y = f x = ax + b Thequestionswewanttoanswerare: 1.Whatdoes"best"approximationmean? 2.Howdowend a;b thatgivesthelinewiththe"best"approximation?

PAGE 156

CHAPTER5.APPROXIMATIONTHEORY 155 Observethatforeach x i ,thereisthecorresponding y i ofthedatapoint,and f x i = ax i + b; whichisthepredictedvaluebythelinearapproximation.Wecanmeasureerrorbyconsideringthe deviationsbetweentheactual y coordinatesandthepredictedvalues: y 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(ax 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(b ; y 2 )]TJ/F82 10.9091 Tf 10.909 0 Td [(ax 2 )]TJ/F82 10.9091 Tf 10.91 0 Td [(b ;:::; y m )]TJ/F82 10.9091 Tf 10.91 0 Td [(ax m )]TJ/F82 10.9091 Tf 10.909 0 Td [(b Thereareseveralwayswecanformameasureoferrorusingthesedeviations,andeachapproach givesadierentlineapproximatingthedata.Thebestapproximationmeansnding a;b that minimizestheerrormeasuredinoneofthefollowingways: E =max i fj y i )]TJ/F82 10.9091 Tf 10.91 0 Td [(ax i )]TJ/F82 10.9091 Tf 10.909 0 Td [(b jg ;minimaxproblem E = P m i =1 j y i )]TJ/F82 10.9091 Tf 10.909 0 Td [(ax i )]TJ/F82 10.9091 Tf 10.909 0 Td [(b j ;absolutedeviations E = P m i =1 y i )]TJ/F82 10.9091 Tf 10.909 0 Td [(ax i )]TJ/F82 10.9091 Tf 10.909 0 Td [(b 2 ;leastsquaresproblem Inthischapterwewilldiscusstheleastsquaresproblem,thesimplestoneamongthethreeoptions. Wewanttominimize E = m X i =1 y i )]TJ/F82 10.9091 Tf 10.909 0 Td [(ax i )]TJ/F82 10.9091 Tf 10.909 0 Td [(b 2 withrespecttotheparameters a;b: Foraminimumtooccur,wemusthave @E @a =0 and @E @b =0 : Wehave: @E @a = m X i =1 @E @a y i )]TJ/F82 10.9091 Tf 10.909 0 Td [(ax i )]TJ/F82 10.9091 Tf 10.909 0 Td [(b 2 = m X i =1 )]TJ/F81 10.9091 Tf 8.485 0 Td [(2 x i y i )]TJ/F82 10.9091 Tf 10.909 0 Td [(ax i )]TJ/F82 10.9091 Tf 10.909 0 Td [(b =0 @E @b = m X i =1 @E @b y i )]TJ/F82 10.9091 Tf 10.909 0 Td [(ax i )]TJ/F82 10.9091 Tf 10.909 0 Td [(b 2 = m X i =1 )]TJ/F81 10.9091 Tf 8.485 0 Td [(2 y i )]TJ/F82 10.9091 Tf 10.909 0 Td [(ax i )]TJ/F82 10.9091 Tf 10.909 0 Td [(b =0 Usingalgebra,theseequationscanbesimpliedas b m X i =1 x i + a m X i =1 x 2 i = m X i =1 x i y i bm + a m X i =1 x i = m X i =1 y i ; whicharecalledthe normalequations .Thesolutiontothissystemofequationsis a = m P m i =1 x i y i )]TJ/F54 10.9091 Tf 10.909 8.182 Td [(P m i =1 x i P m i =1 y i m )]TJ 5 -0.654 Td [(P m i =1 x 2 i )]TJ/F81 10.9091 Tf 10.909 0 Td [( P m i =1 x i 2 ;b = P m i =1 x 2 i P m i =1 y i )]TJ/F54 10.9091 Tf 10.909 8.182 Td [(P m i =1 x i y i P m i =1 x i m )]TJ 5 -0.654 Td [(P m i =1 x 2 i )]TJ/F81 10.9091 Tf 10.909 0 Td [( P m i =1 x i 2 : Let'sconsideraslightlymoregeneralquestion.Givendata

PAGE 157

CHAPTER5.APPROXIMATIONTHEORY 156 Data: x 1 ;y 1 ; x 2 ;y 2 ;:::; x m ;y m canwendthebestpolynomialapproximation Polynomialapproximation: P n x = a n x n + a n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 x n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 + ::: + a 0 where m willbeusuallymuchlargerthan n .Similartotheabovediscussion,wewanttominimize E = m X i =1 y i )]TJ/F82 10.9091 Tf 10.909 0 Td [(P n x i 2 = m X i =1 0 @ y i )]TJ/F49 7.9701 Tf 16.219 13.636 Td [(n X j =0 a j x j i 1 A 2 withrespecttotheparameters a n ;a n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 ;:::;a 0 : Foraminimumtooccur,thenecessaryconditions are @E @a k =0 )]TJ/F49 7.9701 Tf 28.376 13.637 Td [(m X i =1 y i x k i + n X j =0 a j m X i =1 x k + j i ! =0 for k =0 ; 1 ;:::;n .weareskippingsomealgebrahere!The normalequations forpolynomial approximationare n X j =0 a j m X i =1 x k + j i ! = m X i =1 y i x k i .1 for k =0 ; 1 ;:::;n: Thisisasystemof n +1 equationsand n +1 unknowns.Wecanwritethis systemasamatrixequation A a = b .2 where a istheunknownvectorwearetryingtond,and b istheconstantvector a = 2 6 6 6 6 6 4 a 0 a 1 . . . a n 3 7 7 7 7 7 5 ; b = 2 6 6 6 6 6 4 P m i =1 y i P m i =1 y i x i . . . P m i =1 y i x n i 3 7 7 7 7 7 5 and A isan n +1 by n +1 symmetricmatrixwith k;j thentry A kj ;k =1 ;:::;n +1 ;j = 1 ; 2 ;:::;n +1 givenby A kj = m X i =1 x k + j )]TJ/F46 7.9701 Tf 6.586 0 Td [(2 i : Theequation A a = b hasauniquesolutionifthe x i aredistinct,and n m )]TJ/F81 10.9091 Tf 8.503 0 Td [(1 .Solvingthisequation bycomputingtheinversematrix A )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 isnotadvisable,sincetherecouldbesignicantroundoerror. Next,wewillwriteaPythoncodeforleastsquaresapproximation,andusethePythonfunction np:linalg:solve A;b tosolvethematrixequation A a = b for a .The np:linalg:solve functionin

PAGE 158

CHAPTER5.APPROXIMATIONTHEORY 157 PythonusesnumericallyoptimizedmatrixfactorizationsbasedontheLAPACKroutinetosolve thematrixequation.MoredetailsonthistopiccanbefoundinHeath[10]Chapter3. Pythoncodeforleastsquaresapproximation In[1]:importnumpyasnp importmatplotlib.pyplotasplt %matplotlibinline Thefunction leastsqt takesthe x -and y -coordinatesofthedata,andthedegreeofthe polynomialwewanttouse, n ,asinputs.ItsolvesthematrixEquation5.2. In[2]:defleastsqfitx,y,n: m=x.size #numberofdatapoints d=n+1 #numberofcoefficientstobedetermined A=np.zerosd,d b=np.zerosd #thelinearsystemwewanttosolveisAx=b p=np.zeros2*n+1 forkinranged: sum=0 foriinrangem: sum+=y[i]*x[i]**k b[k]=sum #p[i]belowisthesumofthei-thpowerofthexcoordinates p[0]=m foriinrange1,2*n+1: sum=0 forjinrangem: sum+=x[j]**i p[i]=sum #Wenextcomputetheuppertriangularpartofthecoefficient #matrixA,anditsdiagonal forkinranged: forjinrangek,d: A[k,j]=p[k+j] #Thelowertriangularpartofthematrixisdefinedusingthe #factthematrixissymmetric foriinrange1,d: forjinrangei:

PAGE 159

CHAPTER5.APPROXIMATIONTHEORY 158 A[i,j]=A[j,i] a=np.linalg.solveA,b returna Hereisthedatausedtoproducetherstplotofthechapter:Arya'sdata: In[3]:xd=np.array[1,2,3,4,5,6] yd=np.array[3,5,9.2,11,14.5,19] Wetaleastsquareslinetothedata: In[4]:leastsqfitxd,yd,1 Out[4]:array[-0.74666667,3.15142857] Thepolynomialis )]TJ/F81 10.9091 Tf 8.485 0 Td [(0 : 746667+3 : 15143 x .Thenextfunction polyx,a takestheoutputof a = leastsqt ,andevaluatestheleastsquarespolynomialat x . In[5]:defpolyx,a: d=a.size sum=0 foriinranged: sum+=a[i]*x**i returnsum Forexample,ifwewanttocomputetheleastsquareslineat3.5,wecallthefollowingfunctions: In[6]:a=leastsqfitxd,yd,1 poly3.5,a Out[6]:10.283333333333335 Thenextfunctioncomputestheleastsquareserror: E = P m i =1 y i )]TJ/F82 10.9091 Tf 9.33 0 Td [(p n x i 2 .Ittakestheoutput of a = leastsqt ,andthedata,asinputs. In[7]:defleastsqerrora,x,y: sum=0 m=y.size foriinrangem: sum+=y[i]-polyx[i],a**2 returnsum In[8]:a=leastsqfitxd,yd,1 leastsqerrora,xd,yd

PAGE 160

CHAPTER5.APPROXIMATIONTHEORY 159 Out[8]:2.6070476190476177 Nextweplottheleastsquareslineandthedatatogether. In[9]:a=leastsqfitxd,yd,1 xaxis=np.linspace1,6,500 yvals=polyxaxis,a plt.plotxaxis,yvals plt.plotxd,yd, ' o ' ; Wetryaseconddegreepolynomialinleastsquaresapproximationnext. In[10]:a=leastsqfitxd,yd,2 xaxis=np.linspace1,6,500 yvals=polyxaxis,a plt.plotxaxis,yvals plt.plotxd,yd, ' o ' ;

PAGE 161

CHAPTER5.APPROXIMATIONTHEORY 160 Thecorrespondingerroris: In[11]:leastsqerrora,xd,yd Out[11]:1.4869285714285714 Thenextpolynomialisofdegreethree: In[12]:a=leastsqfitxd,yd,3 xaxis=np.linspace1,6,500 yvals=polyxaxis,a plt.plotxaxis,yvals plt.plotxd,yd, ' o ' ; Thecorrespondingerroris: In[13]:leastsqerrora,xd,yd Out[13]:1.2664285714285732 Thenextpolynomialisofdegreefour: In[14]:a=leastsqfitxd,yd,4 xaxis=np.linspace1,6,500 yvals=polyxaxis,a plt.plotxaxis,yvals plt.plotxd,yd, ' o ' ;

PAGE 162

CHAPTER5.APPROXIMATIONTHEORY 161 Theleastsquareserroris: In[15]:leastsqerrora,xd,yd Out[15]:0.723214285714292 Finally,wetryafthdegreepolynomial.Recallthatthenormalequationshaveaunique solutionwhen x i aredistinct,and n m )]TJ/F81 10.9091 Tf 11.25 0 Td [(1 .Since m =6 inthisexample, n =5 isthelargest degreewithguaranteeduniquesolution. In[16]:a=leastsqfitxd,yd,5 xaxis=np.linspace1,6,500 yvals=polyxaxis,a plt.plotxaxis,yvals plt.plotxd,yd, ' o ' ;

PAGE 163

CHAPTER5.APPROXIMATIONTHEORY 162 Theapproximatingpolynomialofdegreeveistheinterpolatingpolynomial!Whatistheleast squareserror? Leastsquareswithnon-polynomials Themethodofleastsquaresisnotonlyforpolynomials.Forexample,supposewewanttondthe function f t = a + bt + c sin t= 365+ d cos t= 365 .3 thathasthebestttosomedata t 1 ;T 1 ;:::; t m ;T m intheleast-squaressense.Thisfunctionis usedinmodelingweathertemperaturedata,where t denotestime,and T denotesthetemperature. Thefollowinggureplotsthedailymaximumtemperatureduringaperiodof1,056days,from2016 untilNovember21,2018,asmeasuredbyaweatherstationatMelbourneairport,Australia 1 . Tondthebesttfunctionoftheform5.3,wewritetheleastsquareserrorterm E = m X i =1 f t i )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i 2 = m X i =1 a + bt i + c sin 2 t i 365 + d cos 2 t i 365 )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i 2 ; andsetitspartialderivativeswithrespecttotheunknowns a;b;c;d tozerotoobtainthenormal 1 http://www.bom.gov.au/climate/data/

PAGE 164

CHAPTER5.APPROXIMATIONTHEORY 163 equations: @E @a =0 m X i =1 2 a + bt i + c sin 2 t i 365 + d cos 2 t i 365 )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i =0 m X i =1 a + bt i + c sin 2 t i 365 + d cos 2 t i 365 )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i =0 ; .4 @E @b =0 m X i =1 t i a + bt i + c sin 2 t i 365 + d cos 2 t i 365 )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i =0 m X i =1 t i a + bt i + c sin 2 t i 365 + d cos 2 t i 365 )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i =0 ; .5 @E @c =0 m X i =1 2sin 2 t i 365 a + bt i + c sin 2 t i 365 + d cos 2 t i 365 )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i =0 m X i =1 sin 2 t i 365 a + bt i + c sin 2 t i 365 + d cos 2 t i 365 )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i =0 ; .6 @E @d =0 m X i =1 2cos 2 t i 365 a + bt i + c sin 2 t i 365 + d cos 2 t i 365 )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i =0 m X i =1 cos 2 t i 365 a + bt i + c sin 2 t i 365 + d cos 2 t i 365 )]TJ/F82 10.9091 Tf 10.909 0 Td [(T i =0 : .7 Rearrangingtermsinequations5.4,5.5,5.6,5.7,wegetasystemoffourequationsandfour

PAGE 165

CHAPTER5.APPROXIMATIONTHEORY 164 unknowns: am + b m X i =1 t i + c m X i =1 sin 2 t i 365 + d m X i =1 cos 2 t i 365 = m X i =1 T i a m X i =1 t i + b m X i =1 t 2 i + c m X i =1 t i sin 2 t i 365 + d m X i =1 t i cos 2 t i 365 = m X i =1 T i t i a m X i =1 sin 2 t i 365 + b m X i =1 t i sin 2 t i 365 + c m X i =1 sin 2 2 t i 365 + d m X i =1 sin 2 t i 365 cos 2 t i 365 = m X i =1 T i sin 2 t i 365 a m X i =1 cos 2 t i 365 + b m X i =1 t i cos 2 t i 365 + c m X i =1 sin 2 t i 365 cos 2 t i 365 + d m X i =1 cos 2 2 t i 365 = m X i =1 T i cos 2 t i 365 Usingashort-handnotationwherewesuppresstheargument )]TJ/F46 7.9701 Tf 6.195 -4.239 Td [(2 t i 365 inthetrigonometricfunctions,andthesummationindices,wewritetheaboveequationsasamatrixequation: 2 6 6 6 6 4 m P t i P sin P cos P t i P t 2 i P t i sin P t i cos P sin P t i sin P sin 2 P sin cos P cos P t i cos P sin cos P cos 2 3 7 7 7 7 5 | {z } A 2 6 6 6 6 4 a b c d 3 7 7 7 7 5 = 2 6 6 6 6 4 P T i P T i t i P T i sin P T i cos 3 7 7 7 7 5 | {z } r Next,wewillusePythontoloadthedataanddenethematrices A; r ,andthensolvetheequation A x = r ,where x =[ a;b;c;d ] T . Wewilluseapackagecalledpandastoimportdata.Weimportitinournotebook,alongwith NumPyandMatplotlib: In[1]:importpandasaspd importnumpyasnp importmatplotlib.pyplotasplt %matplotlibinline Weassumethatthedatawhichconsistsoftemperaturesisdownloadedasacsvleinthesame directorywherethePythonnotebookisstored.Makesurethedatahasnomissingentries.The function pd.read_csv importsthedataintoPythonasatabledataframe: In[2]:df=pd.read_csv ' WeatherData.csv ' df

PAGE 166

CHAPTER5.APPROXIMATIONTHEORY 165 Out[2]:Temp 028.7 127.5 228.2 324.5 425.6 ...... 105119.1 105227.1 105331.0 105427.0 105522.7 [1056rowsx1columns] Thenextstepistostorethepartofthedataweneedasanarray.Inourtablethereisonly onecolumnnamed Temp . In[3]:temp=df[ ' Temp ' ].to_numpy temp Out[3]:array[28.7,27.5,28.2,...,31.,27.,22.7] Let'scheckthetypeof temp ,itsrstentry,anditslength: In[4]:typetemp Out[4]:numpy.ndarray In[5]:temp[0] Out[5]:28.7 In[6]:temp.size Out[6]:1056 Thereare1,056temperaturevalues.The x -coordinatesarethedays,numbered t =1 ; 2 ;:::; 1056 . Hereisthearraythatstoresthesetimevalues: In[7]:time=np.arange1,1057 Nextwedenethematrix A ,takingadvantageofthefactthatthematrixissymmetric.The function np.sumx addstheentriesofthearray x .

PAGE 167

CHAPTER5.APPROXIMATIONTHEORY 166 In[8]:A=np.zeros4,4 A[0,0]=1056 A[0,1]=np.sumtime A[0,2]=np.sumnp.sin2*np.pi*time/365 A[0,3]=np.sumnp.cos2*np.pi*time/365 A[1,1]=np.sumtime**2 A[1,2]=np.sumtime*np.sin2*np.pi*time/365 A[1,3]=np.sumtime*np.cos2*np.pi*time/365 A[2,2]=np.sumnp.sin2*np.pi*time/365**2 A[2,3]=np.sumnp.sin2*np.pi*time/365*np.cos2*np.pi*time/365 A[3,3]=np.sumnp.cos2*np.pi*time/365**2 foriinrange1,4: forjinrangei: A[i,j]=A[j,i] In[9]:A Out[9]:array[[1.05600000e+03,5.58096000e+05,1.22956884e+01, -3.62432800e+01], [5.58096000e+05,3.93085616e+08,-5.04581481e+04, -3.84772834e+04], [1.22956884e+01,-5.04581481e+04,5.42338826e+02, 1.09944004e+01], [-3.62432800e+01,-3.84772834e+04,1.09944004e+01, 5.13661174e+02]] Nowwedenethevector r .Thefunction np.dotx,y takesthedotproductofthearrays x;y . Forexample, np.dot [1 ; 2 ; 3] ; [4 ; 5 ; 6]=1 4+2 5+3 6=32 : In[10]:r=np.zeros4,1 r[0]=np.sumtemp r[1]=np.dottemp,time r[2]=np.dottemp,np.sin2*np.pi*time/365 r[3]=np.dottemp,np.cos2*np.pi*time/365 In[11]:r Out[11]:array[[2.18615000e+04], [1.13803102e+07], [1.74207707e+03], [2.78776127e+03]]

PAGE 168

CHAPTER5.APPROXIMATIONTHEORY 167 Wecansolvethematrixequationnow. In[12]:np.linalg.solveA,r Out[12]:array[[2.02897563e+01], [1.16773021e-03], [2.72116176e+00], [6.88808561e+00]] Recallthattheseconstantsarethevaluesof a;b;c;d inthedenitionof f t .Hereisthebest ttingfunctiontothedata: In[13]:f=lambdat:20.1393+0.00144928*t+2.72174*np.sin2*np.pi*t/365+ 6.88460*np.cos2*np.pi*t/365 Wenextplotthedatatogetherwith f t : In[14]:xaxis=np.arange1,1057 yvals=fxaxis plt.plotxaxis,yvals,label= ' Leastsquaresapproximation ' plt.xlabel ' timet ' plt.ylabel ' TemperatureT ' plt.plottemp,linestyle= ' ' ,alpha=0.5,label= ' Data ' plt.legendloc= ' uppercenter ' ;

PAGE 169

CHAPTER5.APPROXIMATIONTHEORY 168 Linearizingdata Foranotherexampleofnon-polynomialleastsquares,considerndingthefunction f x = be ax withthebestleastsquaresttosomedata x 1 ;y 1 ; x 2 ;y 2 ;:::; x m ;y m .Weneedtond a;b that minimize E = m X i =1 y i )]TJ/F82 10.9091 Tf 10.909 0 Td [(be ax i 2 : Thenormalequationsare @E @a =0 and @E @b =0 ; however,unlikethepreviousexample,thisisnotasystemoflinearequationsintheunknowns a;b . Ingeneral,arootndingtypemethodisneededtosolvetheseequations. Thereisasimplerapproachwecanusewhenwesuspectthedataisexponentiallyrelated. Consideragainthefunctionwewanttot: y = be ax : .8 Takethelogarithmofbothsides: log y =log b + ax andrenamethevariablesas Y =log y;B =log b .Thenweobtaintheexpression Y = ax + B .9 whichisalinearequationinthetransformedvariable.Inotherwords,iftheoriginalvariable y is relatedto x viaEquation5.8,then Y =log y isrelatedto x viaalinearrelationshipgivenby Equation5.9.So,thenewapproachistottheleastsquaresline Y = ax + B tothedata x 1 ; log y 1 ; x 2 ; log y 2 ;:::; x m ; log y m : However,itisimportanttorealizethattheleastsquaresttothetransformeddataisnot necessarilythesameastheleastsquaresttotheoriginaldata.Thereasonisthedeviationswhich leastsquaresminimizearedistortedinanon-linearwaybythetransformation. Example85. Considerthefollowingdata x 012345 y 358122337 towhichwewillt y = be ax intheleast-squaressense.Thefollowingtabledisplaysthedata x i ; log y i ,usingtwo-digits: x 012345 Y =log y 1.11.62.12.53.13.6

PAGE 170

CHAPTER5.APPROXIMATIONTHEORY 169 WeusethePythoncode leastsqt totalinetothisdata: In[1]:x=np.array[0,1,2,3,4,5] y=np.array[1.1,1.6,2.1,2.5,3.1,3.6] In[2]:leastsqfitx,y,1 Out[2]:array[1.09047619,0.49714286] Thereforetheleastsquaresline,usingtwo-digits,is Y =0 : 5 x +1 : 1 : ThisequationcorrespondstoEquation5.9,with a =0 : 5 and B =1 : 1 .Wewanttoobtainthe correspondingexponentialEquation5.8,where b = e B .Since e 1 : 1 =3 ,thebestttingexponential functiontothedatais y =3 e x= 2 .Thefollowinggraphplots y =3 e x= 2 togetherwiththedata. Exercise5.1-1: Findthefunctionoftheform y = ae x + b sin x thatbesttsthedata belowintheleastsquaressense. x 12345 y -46-1520 Plotthefunctionandthedatatogether.

PAGE 171

CHAPTER5.APPROXIMATIONTHEORY 170 Exercise5.1-2: Power-lawtyperelationshipsareobservedinmanyempiricaldata.Two variables y , x aresaidtoberelatedviaapower-lawif y = kx ,where k; aresomeconstants.The followingdata 2 liststhetop10familynamesintheorderofoccurrenceaccordingtoCensus2000. Investigatewhetherrelativefrequencyofoccurrencesandtherankofthenamearerelatedviaa power-law,by aLet y betherelativefrequenciesnumberofoccurrencesdividedbythetotalnumberof occurrences,and x betherank,thatis,1through10. bUseleastsquarestondafunctionoftheform y = kx .Uselinearization. cPlotthedatatogetherwiththebestttingfunctionfoundinpartb. NameNumberofOccurrences Smith2,376,206 Johnson1,857,160 Williams1,534,042 Brown1,380,145 Jones1,362,755 Miller1,127,803 Davis1,072,335 Garcia858,289 Rodriguez804,240 Wilson783,051 5.2Continuousleastsquares Indiscreteleastsquares,ourstartingpointwasasetofdatapoints.Herewewillstartwitha continuousfunction f on [ a;b ] andanswerthefollowingquestion:howcanwendthe"best" polynomial P n x = P n j =0 a j x j ofdegreeatmost n thatapproximates f on [ a;b ]? Asbefore,"best" polynomialwillmeanthepolynomialthatminimizestheleastsquareserror: E = Z b a 0 @ f x )]TJ/F49 7.9701 Tf 16.219 13.636 Td [(n X j =0 a j x j 1 A 2 dx: .10 Comparethisexpressionwiththatofthe discreteleastsquares : E = m X i =1 0 @ y i )]TJ/F49 7.9701 Tf 16.219 13.636 Td [(n X j =0 a j x j i 1 A 2 : 2 https://www.census.gov/topics/population/genealogy/data/2000_surnames.html

PAGE 172

CHAPTER5.APPROXIMATIONTHEORY 171 Tominimize E in5.10weset @E @a k =0 ,for k =0 ; 1 ;:::;n; andobserve @E @a k = @ @a k 0 @ Z b a f 2 x dx )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 Z b a f x 0 @ n X j =0 a j x j 1 A dx + Z b a 0 @ n X j =0 a j x j 1 A 2 dx 1 A = )]TJ/F81 10.9091 Tf 8.485 0 Td [(2 Z b a f x x k dx +2 n X j =0 a j Z b a x j + k dx =0 ; whichgivesthe n +1 normalequationsforthe continuousleastsquaresproblem : n X j =0 a j Z b a x j + k dx = Z b a f x x k dx .11 for k =0 ; 1 ;:::;n: Notethattheonlyunknownsintheseequationsarethe a j 's;hencethisisalinear systemofequations.Itisinstructivetocomparethesenormalequationswiththoseofthe discrete leastsquaresproblem : n X j =0 a j m X i =1 x k + j i ! = m X i =1 y i x k i : Example86. Findtheleastsquarespolynomialapproximationofdegree2to f x = e x on ; 2 : Solution. Thenormalequationsare: 2 X j =0 a j Z 2 0 x j + k dx = Z 2 0 e x x k dx k =0 ; 1 ; 2 : Herearethethreeequations: a 0 Z 2 0 dx + a 1 Z 2 0 xdx + a 2 Z 2 0 x 2 dx = Z 2 0 e x dx a 0 Z 2 0 xdx + a 1 Z 2 0 x 2 dx + a 2 Z 2 0 x 3 dx = Z 2 0 e x xdx a 0 Z 2 0 x 2 dx + a 1 Z 2 0 x 3 dx + a 2 Z 2 0 x 4 dx = Z 2 0 e x x 2 dx Computingtheintegralsweget 2 a 0 +2 a 1 + 8 3 a 2 = e 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 2 a 0 + 8 3 a 1 +4 a 2 = e 2 +1 8 3 a 0 +4 a 1 + 32 5 a 2 =2 e 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(2

PAGE 173

CHAPTER5.APPROXIMATIONTHEORY 172 whosesolutionis a 0 =3 )]TJ/F81 10.9091 Tf 8.485 0 Td [(7+ e 2 , a 1 = )]TJ/F46 7.9701 Tf 9.68 4.295 Td [(3 2 )]TJ/F81 10.9091 Tf 8.485 0 Td [(37+5 e 2 ;a 2 = 15 4 )]TJ/F81 10.9091 Tf 8.484 0 Td [(7+ e 2 : Then P 2 x =1 : 17+0 : 08 x +1 : 46 x 2 : Thesolutionmethodwehavediscussedfortheleastsquaresproblembysolvingthenormal equationsasamatrixequationhascertaindrawbacks: Theintegrals R b a x i + j dx = )]TJ/F82 10.9091 Tf 5 -8.837 Td [(b i + j +1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(a i + j +1 = i + j +1 inthecoecientmatrixgiveriseto matrixequationthatispronetoroundoerror. Thereisnoeasywaytogofrom P n x to P n +1 x whichwemightwanttodoifwewerenot satisedbytheapproximationprovidedby P n . Thereisabetterapproachtosolvethediscreteandcontinuousleastsquaresproblemusingthe orthogonalpolynomialsweencounteredinGaussianquadrature.Bothdiscreteandcontinuous leastsquaresproblemtriestondapolynomial P n x = P n j =0 a j x j thatsatisessomeproperties. Noticehowthepolynomialiswrittenintermsofthemonomialbasisfunctions x j andrecallhow thesebasisfunctionscausednumericaldicultiesininterpolation.Thatwasthereasonwediscussed dierentbasisfunctionslikeLagrangeandNewtonfortheinterpolationproblem.Sotheideaisto write P n x intermsofsomeotherbasisfunctions P n x = n X j =0 a j j x whichwouldthenupdatethenormalequationsforcontinuousleastsquares5.11as n X j =0 a j Z b a j x k x dx = Z b a f x k x dx for k =0 ; 1 ;:::;n: Thenormalequationsforthediscreteleastsquares5.1getsasimilarupdate: n X j =0 a j m X i =1 j x i k x i ! = m X i =1 y i k x i : Goingforward,thecrucialobservationisthattheintegraloftheproductoftwofunctions R j x k x dx ,orthesummationoftheproductoftwofunctionsevaluatedatsomediscrete points, P j x i k x i ,canbeviewedasaninnerproduct h j ; k i oftwovectorsinasuitablydenedvectorspace.Andwhenthefunctionsvectors j are orthogonal ,theinnerproduct h j ; k i is0if j 6 = k ,whichmakesthenormalequationstrivialtosolve.Wewilldiscussdetailsinthenext section.

PAGE 174

CHAPTER5.APPROXIMATIONTHEORY 173 5.3Orthogonalpolynomialsandleastsquares Ourdiscussioninthissectionwillmostlycenteraroundthecontinuousleastsquaresproblem; however,thediscreteproblemcanbeapproachedsimilarly.Considertheset C 0 [ a;b ] ,thesetofall continuousfunctionsdenedon [ a;b ] ; and P n ; thesetofallpolynomialsofdegreeatmost n on [ a;b ] : Thesetwosetsarevectorspaces,thelatterasubspaceoftheformer,undertheusualoperationsof functionadditionandmultiplyingbyascalar.Aninnerproductonthisspaceisdenedasfollows: given f;g 2 C 0 [ a;b ] h f;g i = Z b a w x f x g x dx .12 andthenormofavectorunderthisinnerproductis k f k = h f;f i 1 = 2 = Z b a w x f 2 x dx 1 = 2 : Let'srecallthedenitionofaninnerproduct:itisarealvaluedfunctionwiththefollowingproperties: 1. h f;g i = h g;f i 2. h f;f i 0 ; withtheequalityonlywhen f 0 3. h f;g i = h f;g i forallrealnumbers 4. h f 1 + f 2 ;g i = h f 1 ;g i + h f 2 ;g i Themysteriousfunction w x in5.12iscalleda weightfunction .Itsjobistoassigndierent importancetodierentregionsoftheinterval [ a;b ] : Theweightfunctionisnotarbitrary;ithasto satisfysomeproperties. Denition87. Anonnegativefunction w x on [ a;b ] iscalledaweightfunctionif 1. R b a j x j n w x dx isintegrableandniteforall n 0 2.If R b a w x g x dx =0 forsome g x 0 ; then g x isidenticallyzeroon a;b : Withournewterminologyandset-up,wecanwritetheleastsquaresproblemasfollows: Problem ContinuousleastsquaresGiven f 2 C 0 [ a;b ] ; ndapolynomial P n x 2 P n that minimizes Z b a w x f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(P n x 2 dx = h f x )]TJ/F82 10.9091 Tf 10.91 0 Td [(P n x ;f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(P n x i : Wewillseethisinnerproductcanbecalculatedeasilyif P n x iswrittenasalinearcombinationoforthogonalbasispolynomials: P n x = P n j =0 a j j x .

PAGE 175

CHAPTER5.APPROXIMATIONTHEORY 174 Weneedsomedenitionsandtheoremstocontinuewithourquest.Let'sstartwithaformal denitionoforthogonalfunctions. Denition88. Functions f 0 ; 1 ;:::; n g areorthogonalfortheinterval [ a;b ] andwithrespectto theweightfunction w x if h j ; k i = Z b a w x j x k x dx = 8 < : 0 if j 6 = k j > 0 if j = k where j issomeconstant.If,inaddition, j =1 forall j ,thenthefunctionsarecalledorthonormal. Howcanwendanorthogonalororthonormalbasisforourvectorspace?Gram-Schmidt processfromlinearalgebraprovidestheanswer. Theorem89 Gram-Schmidtprocess . Givenaweightfunction w x ; theGram-Schmidtprocess constructsauniquesetofpolynomials 0 x ; 1 x ;:::; n x wherethedegreeof i x is i; such that h j ; k i = 8 < : 0 if j 6 = k 1 if j = k andthecoecientof x n in n x ispositive. Let'sdiscusstwoorthogonalpolynomialsthatcanbeobtainedfromtheGram-Schmidtprocess usingdierentweightfunctions. Example90 LegendrePolynomials . If w x 1 and [ a;b ]=[ )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1] ,therstfourpolynomialsobtainedfromtheGram-Schmidtprocess,whentheprocessisappliedtothemonomials 1 ;x;x 2 ;x 3 ;::: , are: 0 x = r 1 2 ; 1 x = r 3 2 x; 2 x = 1 2 r 5 2 x 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 ; 3 x = 1 2 r 7 2 x 3 )]TJ/F81 10.9091 Tf 10.909 0 Td [(3 x : Oftenthesepolynomialsarewritteninitsorthogonalform;thatis,wedroptherequirement h j ; j i =1 intheGram-Schmidtprocess,andwescalethepolynomialssothatthevalueof eachpolynomialat1equals1.Therstfourpolynomialsinthatformare L 0 x =1 ;L 1 x = x;L 2 x = 3 2 x 2 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(1 2 ;L 3 x = 5 2 x 3 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(3 2 x: ThesearetheLegendrepolynomials,polynomialswerstdiscussedinGaussianquadrature,Section 4.3 3 .Theycanbeobtainedfromthefollowingrecursion L n +1 x = 2 n +1 n +1 xL n x )]TJ/F82 10.9091 Tf 21.498 7.38 Td [(n n +1 L n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x ; 3 TheLegendrepolynomialsinSection4.3dierfromthesebyaconstantfactor.Forexample,inSection 4.3thethirdpolynomialwas L 2 x = x 2 )]TJ/F7 6.9738 Tf 9.598 3.923 Td [(1 3 ,buthereitis L 2 x = 3 2 x 2 )]TJ/F7 6.9738 Tf 9.598 3.923 Td [(1 3 .Observethatmultiplyingthese polynomialsbyaconstantdoesnotchangetheirrootswhatwewereinterestedinGaussianquadrature, ortheirorthogonality.

PAGE 176

CHAPTER5.APPROXIMATIONTHEORY 175 n =1 ; 2 ;::: ,andtheysatisfy h L n ;L n i = 2 2 n +1 : Exercise5.3-1: Show,bydirectintegration,thattheLegendrepolynomials L 1 x and L 2 x areorthogonal. Example91 Chebyshevpolynomials . Ifwetake w x = )]TJ/F82 10.9091 Tf 11.316 0 Td [(x 2 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 = 2 and [ a;b ]=[ )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1] ,and againdroptheorthonormalrequirementinGram-Schmidt,weobtainthefollowingorthogonal polynomials: T 0 x =1 ;T 1 x = x;T 2 x =2 x 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1 ;T 3 x =4 x 3 )]TJ/F81 10.9091 Tf 10.909 0 Td [(3 x;::: ThesepolynomialsarecalledChebyshevpolynomialsandsatisfyacuriousidentity: T n x =cos n cos )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x ;n 0 : Chebyshevpolynomialsalsosatisfythefollowingrecursion: T n +1 x =2 xT n x )]TJ/F82 10.9091 Tf 10.91 0 Td [(T n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 x for n =1 ; 2 ;::: ,and h T j ;T k i = 8 > > > < > > > : 0 if j 6 = k if j = k =0 = 2 if j = k> 0 : Ifwetaketherst n +1 LegendreorChebyshevpolynomials,callthem 0 ;:::; n ; thenthese polynomialsformabasisforthevectorspace P n : Inotherwords,theyformalinearlyindependent setoffunctions,andanypolynomialfrom P n canbewrittenasauniquelinearcombinationof them.Thesestatementsfollowfromthefollowingtheorem,whichwewillleaveunproved. Theorem92. 1.If j x isapolynomialofdegree j for j =0 ; 1 ;:::;n ,then 0 ;:::; n are linearlyindependent. 2.If 0 ;:::; n arelinearlyindependentin P n ,thenforany q x 2 P n ; thereexistuniqueconstants c 0 ;:::;c n suchthat q x = P n j =0 c j j x : Exercise5.3-2: Provethatif f 0 ; 1 ;:::; n g isasetoforthogonalfunctions,thentheymust belinearlyindependent.

PAGE 177

CHAPTER5.APPROXIMATIONTHEORY 176 Wehavedevelopedwhatweneedtosolvetheleastsquaresproblemusingorthogonalpolynomials.Let'sgobacktotheproblemstatement: Given f 2 C 0 [ a;b ] ; ndapolynomial P n x 2 P n thatminimizes E = Z b a w x f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(P n x 2 dx = h f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(P n x ;f x )]TJ/F82 10.9091 Tf 10.909 0 Td [(P n x i with P n x writtenasalinearcombinationoforthogonalbasispolynomials: P n x = P n j =0 a j j x . Intheprevioussection,wesolvedthisproblemusingcalculusbytakingthepartialderivativesof E withrespectto a j andsettingthemequaltozero.Nowwewilluselinearalgebra: E = * f )]TJ/F49 7.9701 Tf 16.219 13.637 Td [(n X j =0 a j j ;f )]TJ/F49 7.9701 Tf 16.218 13.637 Td [(n X j =0 a j j + = h f;f i)]TJ/F81 10.9091 Tf 17.575 0 Td [(2 n X j =0 a j h f; j i + X i X j a i a j h i ; j i = k f k 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 n X j =0 a j h f; j i + n X j =0 a 2 j h j ; j i = k f k 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(2 n X j =0 a j h f; j i + n X j =0 a 2 j j = k f k 2 )]TJ/F49 7.9701 Tf 16.219 13.636 Td [(n X j =0 h f; j i 2 j + n X j =0 h f; j i p j )]TJ/F82 10.9091 Tf 10.909 0 Td [(a j p j 2 : Minimizingthisexpressionwithrespectto a j isnowobvious:simplychoose a j = h f; j i j sothatthe lastsummationintheaboveequation, P n j =0 h h f; j i p j )]TJ/F82 10.9091 Tf 10.909 0 Td [(a j p j i 2 ,vanishes.Thenwehavesolvedthe leastsquaresproblem!Thepolynomialthatminimizestheerror E is P n x = n X j =0 h f; j i j j x .13 where j = h j ; j i : Andthecorrespondingerroris E = k f k 2 )]TJ/F49 7.9701 Tf 16.219 13.637 Td [(n X j =0 h f; j i 2 j : Ifthepolynomials 0 ;:::; n areorthonormal,thentheseequationssimplifybysetting j =1 : Wewillappreciatetheeaseatwhich P n x canbecomputedusingthisapproach,viaformula 5.13,asopposedtosolvingthenormalequationsof5.11whenwediscusssomeexamples.But rstlet'sseetheotheradvantageofthisapproach:how P n +1 x canbecomputedfrom P n x : In

PAGE 178

CHAPTER5.APPROXIMATIONTHEORY 177 Equation5.13,replace n by n +1 toget P n +1 x = n +1 X j =0 h f; j i j j x = n X j =0 h f; j i j j x | {z } P n x + h f; n +1 i n +1 n +1 x = P n x + h f; n +1 i n +1 n +1 x ; whichisasimplerecursionthatlinks P n +1 to P n . Example93. Findtheleastsquarespolynomialapproximationofdegreethreeto f x = e x on )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1 usingLegendrepolynomials. Solution. Put n =3 inEquation5.13andlet j be L j toget P 3 x = h f;L 0 i 0 L 0 x + h f;L 1 i 1 L 1 x + h f;L 2 i 2 L 2 x + h f;L 3 i 3 L 3 x = h e x ; 1 i 2 + h e x ;x i 2 = 3 x + e x ; 3 2 x 2 )]TJ/F46 7.9701 Tf 12.105 4.295 Td [(1 2 2 = 5 3 2 x 2 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(1 2 + e x ; 5 2 x 3 )]TJ/F46 7.9701 Tf 12.105 4.295 Td [(3 2 x 2 = 7 5 2 x 3 )]TJ/F81 10.9091 Tf 12.104 7.38 Td [(3 2 x ; whereweusedthefactthat j = h L j ;L j i = 2 2 n +1 seeExample90.Wewillcomputetheinner products,whicharedeniteintegralson )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1 ,usingtheve-nodeGauss-Legendrequadraturewe discussedinthepreviouschapter.Theresultsroundedtofourdigitsare: h e x ; 1 i = Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 e x dx =2 : 350 h e x ;x i = Z 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 e x xdx =0 : 7358 e x ; 3 2 x 2 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(1 2 = Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 e x 3 2 x 2 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(1 2 dx =0 : 1431 e x ; 5 2 x 3 )]TJ/F81 10.9091 Tf 12.104 7.38 Td [(3 2 x = Z 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 e x 5 2 x 3 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(3 2 x dx =0 : 02013 : Therefore P 3 x = 2 : 35 2 + 3 : 7358 2 x + 5 : 1431 2 3 2 x 2 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(1 2 + 7 : 02013 2 5 2 x 3 )]TJ/F81 10.9091 Tf 12.105 7.38 Td [(3 2 x =0 : 1761 x 3 +0 : 5366 x 2 +0 : 9980 x +0 : 9961 : Example94. Findtheleastsquarespolynomialapproximationofdegreethreeto f x = e x on )]TJ/F81 10.9091 Tf 8.485 0 Td [(1 ; 1 usingChebyshevpolynomials.

PAGE 179

CHAPTER5.APPROXIMATIONTHEORY 178 Solution. Asinthepreviousexamplesolution,wetake n =3 inEquation5.13 P 3 x = 3 X j =0 h f; j i j j x ; butnow j and j willbereplacedby T j ,theChebyshevpolynomials,anditscorresponding constant;seeExample91.Wehave P 3 x = h e x ;T 0 i T 0 x + h e x ;T 1 i = 2 T 1 x + h e x ;T 2 i = 2 T 2 x + h e x ;T 3 i = 2 T 3 x : Consideroneoftheinnerproducts, h e x ;T j i = Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 e x T j x p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 dx whichisanimproperintegralduetodiscontinuityattheendpoints.However,wecanusethe substitution =cos )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x torewritetheintegralasseeSection4.5 h e x ;T j i = Z 1 )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 e x T j x p 1 )]TJ/F82 10.9091 Tf 10.909 0 Td [(x 2 dx = Z 0 e cos cos j d: Thetransformedintegrandissmooth,anditisnotimproper,andhencewecanusecomposite Simpson'sruletoestimateit.Thefollowingestimatesareobtainedbytaking n =20 inthe compositeSimpson'srule: h e x ;T 0 i = Z 0 e cos d =3 : 977 h e x ;T 1 i = Z 0 e cos cos d =1 : 775 h e x ;T 2 i = Z 0 e cos cos2 d =0 : 4265 h e x ;T 3 i = Z 0 e cos cos3 d =0 : 06964 Therefore P 3 x = 3 : 977 + 3 : 55 x + 0 : 853 x 2 )]TJ/F81 10.9091 Tf 10.909 0 Td [(1+ 0 : 1393 x 3 )]TJ/F81 10.9091 Tf 10.909 0 Td [(3 x =0 : 1774 x 3 +0 : 5430 x 2 +0 : 9970 x +0 : 9944 :

PAGE 180

CHAPTER5.APPROXIMATIONTHEORY 179 Pythoncodefororthogonalpolynomials ComputingLegendrepolynomials Legendrepolynomialssatisfythefollowingrecursion: L n +1 x = 2 n +1 n +1 xL n x )]TJ/F82 10.9091 Tf 21.499 7.38 Td [(n n +1 L n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 x for n =1 ; 2 ;::: ,with L 0 x =1 ,and L 1 x = x . ThePythoncodeimplementsthisrecursion,withalittlemodication:theindex n +1 isshifted downto n ,sothemodiedrecursionis: L n x = 2 n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 n xL n )]TJ/F46 7.9701 Tf 6.587 0 Td [(1 x )]TJ/F49 7.9701 Tf 12.104 4.295 Td [(n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 n L n )]TJ/F46 7.9701 Tf 6.587 0 Td [(2 x ,for n =2 ; 3 ;::: . In[1]:importnumpyasnp importmatplotlib.pyplotasplt %matplotlibinline In[2]:deflegx,n: ifn==0: return1 elifn==1: returnx else: return2*n-1/n*x*legx,n-1-n-1/n*legx,n-2 HereisaplotoftherstveLegendrepolynomials: In[3]:xaxis=np.linspace-1,1,200 legzero=np.arraylistmaplambdax:legx,0,xaxis legone=legxaxis,1 legtwo=legxaxis,2 legthree=legxaxis,3 legfour=legxaxis,4 plt.plotxaxis,legzero,label= ' $L_0x$ ' plt.plotxaxis,legone,label= ' $L_1x$ ' plt.plotxaxis,legtwo,label= ' $L_2x$ ' plt.plotxaxis,legthree,label= ' $L_3x$ ' plt.plotxaxis,legfour,label= ' $L_4x$ ' plt.legendloc= ' lowerright ' ;

PAGE 181

CHAPTER5.APPROXIMATIONTHEORY 180 Least-squaresusingLegendrepolynomials InExample93,wecomputedtheleastsquaresapproximationto e x usingLegendrepolynomials. Theinnerproductswerecomputedusingtheve-nodeGauss-Legendrerulebelow. In[4]:defgaussf: return0.2369268851*f-0.9061798459+0.2369268851*f0.9061798459+ 0.5688888889*f0+0.4786286705*f0.5384693101+ 0.4786286705*f-0.5384693101 Theinnerproduct, e x ; 3 2 x 2 )]TJ/F46 7.9701 Tf 12.105 4.295 Td [(1 2 = R 1 )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 e x 3 2 x 2 )]TJ/F46 7.9701 Tf 12.105 4.295 Td [(1 2 dx iscomputedas In[5]:gausslambdax:3/2*x**2-1/2*np.expx Out[5]:0.1431256282441218 Nowthatwehaveacode legx,n thatgeneratestheLegendrepolynomials,wecandotheabove computationwithoutexplicitlyspecifyingtheLegendrepolynomial.Forexample,since 3 2 x 2 )]TJ/F46 7.9701 Tf 12.142 4.296 Td [(1 2 = L 2 ,wecanapplythe gauss functiondirectlyto L 2 x e x : In[6]:gausslambdax:legx,2*np.expx Out[6]:0.14312562824412176 Thefollowingfunction polyLegCoef,n computesthecoecients h f;L j i j oftheleastsquares polynomial P n x = P n j =0 h f;L j i j L j x ;j =0 ; 1 ;:::;n ,forany f and n ,where L j aretheLegendre polynomials.Thecoecientsaretheoutputsthatarereturnedwhenthefunctionisnished. In[7]:defpolyLegCoefff,n: A=np.zerosn+1 forjinrangen+1: A[j]=gausslambdax:legx,j*fx*2*j+1/2 returnA

PAGE 182

CHAPTER5.APPROXIMATIONTHEORY 181 Oncethecoecientsarecomputed,evaluatingthepolynomialcanbedoneecientlybyusing thecoecients.Thenextfunction polyLegx,n,A evaluatestheleastsquarespolynomial P n x = P n j =0 h f;L j i j L j x ;j =0 ; 1 ;:::;n ,wherethecoecients h f;L j i j ,storedin A ,areobtainedfromcalling thefunction polyLegCoef,n . In[8]:defpolyLegx,n,A: sum=0. forjinrangen+1: sum+=A[j]*legx,j returnsum Hereweplot y = e x togetherwithitsleastsquarespolynomialapproximationsofdegreetwoand three,usingLegendrepolynomials.Notethateverytimethefunction polyLegCoe isevaluated, anewcoecientarray A isobtained. In[9]:xaxis=np.linspace-1,1,200 A=polyLegCoefflambdax:np.expx,2 deg2=polyLegxaxis,2,A A=polyLegCoefflambdax:np.expx,3 deg3=polyLegxaxis,3,A plt.plotxaxis,np.expxaxis,label= ' $e^x$ ' plt.plotxaxis,deg2,label= ' Legendreleastsquarespolyofdegree2 ' plt.plotxaxis,deg3,label= ' Legendreleastsquarespolyofdegree3 ' plt.legendloc= ' upperleft ' ; ComputingChebyshevpolynomials ThefollowingfunctionimplementstherecursionChebyshevpolynomialssatisfy: T n +1 x = 2 xT n x )]TJ/F82 10.9091 Tf 11.621 0 Td [(T n )]TJ/F46 7.9701 Tf 6.586 0 Td [(1 x ,for n =1 ; 2 ;:::; with T 0 x =1 and T 1 x = x .Notethatinthecodethe indexisshiftedfrom n +1 to n .

PAGE 183

CHAPTER5.APPROXIMATIONTHEORY 182 In[10]:defchebx,n: ifn==0: return1 elifn==1: returnx else: return2*x*chebx,n-1-chebx,n-2 HereisaplotoftherstveChebyshevpolynomials: In[11]:xaxis=np.linspace-1,1,200 chebzero=np.arraylistmaplambdax:chebx,0,xaxis chebone=chebxaxis,1 chebtwo=chebxaxis,2 chebthree=chebxaxis,3 chebfour=chebxaxis,4 plt.plotxaxis,chebzero,label= ' $T_0x$ ' plt.plotxaxis,chebone,label= ' $T_1x$ ' plt.plotxaxis,chebtwo,label= ' $T_2x$ ' plt.plotxaxis,chebthree,label= ' $T_3x$ ' plt.plotxaxis,chebfour,label= ' $T_4x$ ' plt.legendloc= ' lowerright ' ; LeastsquaresusingChebyshevpolynomials InExample 94 ,wecomputedtheleastsquaresapproximationto e x usingChebyshevpolynomials. Theinnerproducts,afteratransformation,werecomputedusingthecompositeSimpsonrule.Below isthePythoncodeforthecompositeSimpsonrulewediscussedinthepreviouschapter.

PAGE 184

CHAPTER5.APPROXIMATIONTHEORY 183 In[12]:defcompsimpsonf,a,b,n: h=b-a/n nodes=np.zerosn+1 foriinrangen+1: nodes[i]=a+i*h sum=fa+fb foriinrange2,n-1,2: sum+=2*fnodes[i] foriinrange1,n,2: sum+=4*fnodes[i] returnsum*h/3 TheintegralsinExample 94 werecomputedusingthecompositeSimpsonrulewith n =20 .For example,thesecondintegral h e x ;T 1 i = R 0 e cos cos d iscomputedas: In[13]:compsimpsonlambdax:np.expnp.cosx*np.cosx,0,np.pi,20 Out[13]:1.7754996892121808 Nextwewritetwofunctions, polyChebCoef,n and polyChebx,n,A .Therstfunction computesthecoecients h f;T j i j oftheleastsquarespolynomial P n x = P n j =0 h f;T j i j T j x ;j = 0 ; 1 ;:::;n ,forany f and n ,where T j aretheChebyshevpolynomials.Thecoecientsarereturned astheoutputoftherstfunction. Theintegral h f;T j i istransformedtotheintegral R 0 f cos cos j d ,similartothederivation inExample 94 ,andthenthetransformedintegraliscomputedusingthecompositeSimpson'srule by polyChebCoe . In[14]:defpolyChebCoefff,n: A=np.zerosn+1 A[0]=compsimpsonlambdax:fnp.cosx,0,np.pi,20/np.pi forjinrange1,n+1: A[j]=compsimpsonlambdax:fnp.cosx*np.cosj*x,0,np.pi,20*2/np.pi returnA In[15]:defpolyChebx,n,A: sum=0. forjinrangen+1: sum+=A[j]*chebx,j returnsum Nextweplot y = e x togetherwithpolynomialapproximationsofdegreetwoandthreeusing Chebyshevbasispolynomials.

PAGE 185

CHAPTER5.APPROXIMATIONTHEORY 184 In[16]:xaxis=np.linspace-1,1,200 A=polyChebCoefflambdax:np.expx,2 deg2=polyChebxaxis,2,A A=polyChebCoefflambdax:np.expx,3 deg3=polyChebxaxis,3,A plt.plotxaxis,np.expxaxis,label= ' $e^x$ ' plt.plotxaxis,deg2,label= ' Chebyshevleastsquarespolyofdegree2 ' plt.plotxaxis,deg3,label= ' Chebyshevleastsquarespolyofdegree3 ' plt.legendloc= ' upperleft ' ; ThecubicLegendreandChebyshevapproximationsarediculttodistinguishfromthefunction itself.Let'scomparethequadraticapproximationsobtainedbyLegendreandChebyshevpolynomials.Below,youcanseevisuallythatChebyshevdoesabetterapproximationattheendpoints oftheinterval.Isthisexpected? In[17]:xaxis=np.linspace-1,1,200 A=polyChebCoefflambdax:np.expx,2 cheb2=polyChebxaxis,2,A A=polyLegCoefflambdax:np.expx,2 leg2=polyLegxaxis,2,A plt.plotxaxis,np.expxaxis,label= ' $e^x$ ' plt.plotxaxis,cheb2,label= ' Chebyshevleastsquarespolyofdegree2 ' plt.plotxaxis,leg2,label= ' Legendreleastsquarespolyofdegree2 ' plt.legendloc= ' upperleft ' ;

PAGE 186

CHAPTER5.APPROXIMATIONTHEORY 185 Inthefollowing,wecompareseconddegreeleastsquarespolynomialapproximationsfor f x = e x 2 .ComparehowgoodtheLegendreandChebyshevpolynomialapproximationsare inthemidintervalandtowardtheendpoints. In[18]:f=lambdax:np.expx**2 xaxis=np.linspace-1,1,200 A=polyChebCoefflambdax:fx,2 cheb2=polyChebxaxis,2,A A=polyLegCoefflambdax:fx,2 leg2=polyLegxaxis,2,A plt.plotxaxis,fxaxis,label= ' $e^{x^2}}$ ' plt.plotxaxis,cheb2,label= ' Chebyshevleastsquarespolyofdegree2 ' plt.plotxaxis,leg2,label= ' Legendreleastsquarespolyofdegree2 ' plt.legendloc= ' uppercenter ' ;

PAGE 187

CHAPTER5.APPROXIMATIONTHEORY 186 Exercise5.3-3: UsePythontocomputetheleastsquarespolynomialapproximations P 2 x ;P 4 x ;P 6 x to sin4 x usingChebyshevbasispolynomials.Plotthepolynomialstogether with sin4 x .

PAGE 188

References [1]Abramowitz,M.,andStegun,I.A.,1965.Handbookofmathematicalfunctions:withformulas, graphs,andmathematicaltablesVol.55.CourierCorporation. [2]Chace,A.B.,andManning,H.P.,1927.TheRhindMathematicalPapyrus:BritishMuseum 10057and10058.Vol1.MathematicalAssociationofAmerica. [3]Atkinson,K.E.,1989.AnIntroductiontoNumericalAnalysis,SecondEdition,JohnWiley& Sons. [4]Burden,R.L,Faires,D.,andBurden,A.M.,2016.NumericalAnalysis,10thEdition,Cengage. [5]Capstick,S.,andKeister,B.D.,1996.Multidimensionalquadraturealgorithmsathigherdegree and/ordimension.JournalofComputationalPhysics,123,pp.267-273. [6]Chan,T.F.,Golub,G.H.,andLeVeque,R.J.,1983.Algorithmsforcomputingthesample variance:Analysisandrecommendations.TheAmericanStatistician,37,pp.242-247. [7]Cheney,E.W.,andKincaid,D.R.,2012.Numericalmathematicsandcomputing.Cengage Learning. [8]Glasserman,P.,2013.MonteCarlomethodsinFinancialEngineering.Springer. [9]Goldberg,D.,1991.Whateverycomputerscientistshouldknowaboutoating-pointarithmetic.ACMComputingSurveysCSUR,231,pp.5-48. [10]Heath,M.T.,1997.ScienticComputing:Anintroductorysurvey.McGraw-Hill. [11]Higham,N.J.,1993.Theaccuracyofoatingpointsummation.SIAMJournalonScientic Computing,14,pp.783-799. [12]Isaacson,E.,andKeller,H.B.,1966.AnalysisofNumericalMethods.JohnWiley&Sons. 187

PAGE 189

Index Absoluteerror,30 Beasley-Springer-Moro,142 Biasedexponent,22 BigOnotation,78 Bisectionmethod,46 errortheorem,48 Juliacode,47 linearconvergence,49 Black-Scholes-Mertonformula,55 Chebyshevnodes,96 Chebyshevpolynomials,175 Juliacode,181 Chopping,29 CompositeNewton-Cotes,127 midpoint,127 roundo,130 Simpson,127 trapezoidal,127 Degreeofaccuracy,124 Divideddierences,85 derivativeformula,96 Dr.Seuss,117 Extremevaluetheorem,6 Fixed-pointiteration,64,65 errortheorem,68 geometricinterpretation,66 high-order,72 high-ordererrortheorem,73 Juliacode,68 relationtoNewton'smethod,73 Floating-point,21 decimal,29 IEEE64-bit,22 innity,24 NAN,24 normalized,22 toymodel,25 zero,24 Gammafunction,87 Gaussianquadrature,131 errortheorem,136 Juliacode,135 Legendrepolynomials,132 Gram-Schmidtprocess,174 Hermiteinterpolation,96 computation,99 Juliacode,101 Impliedvolatility,56 Improperintegrals,144 Intermediatevaluetheorem,6 Interpolation,75 Inverseinterpolation,92 Iterativemethod,44 stoppingcriteria,44 Julia abs,63 188

PAGE 190

INDEX 189 Complex,63 Distributions,58 dot,166 JuliaDB,164 LatexStrings,92 LinearAlgebra,164 reverse,90 standardnormaldistribution,58 Lagrangeinterpolation,78 Leastsquares,153 continuous,170 discrete,153 Juliacode Chebyshev,182 discrete,157 Legendre,180 linearizing,167 non-polynomials,162 normalequations,continuous,171 normalequations,discrete,155 orthogonalpolynomials,173 Legendrepolynomials,174 Juliacode,179 Linearconvergence,45 Machineepsilon,31 alternativedenition,32 Meanvaluetheorem,6 Midpointrule,125 MonteCarlointegration,140 Muller'smethod,61 convergencerate,62 Juliacode,63 Multipleintegrals,136 Newtoninterpolation Juliacode,89 Newton'smethod,50 errortheorem,54 Juliacode,52 quadraticconvergence,55 Newton-Cotes,121 closed,124 Juliacode,128 open,124 Normalequations continuous,171 discrete,155 Numericaldierentiation,145 three-pointendpoint,148 three-pointmidpoint,148 backward-dierence,146 forward-dierence,146 noisydata,148 roundo,150 secondderivative,149 Numericalquadrature,121 midpointrule,125 MonteCarlo,140 multipleintegrals,136 Newton-Cotes,121 Simpson'srule,123 trapezoidalrule,122 Orthogonalfunctions,172 Orthogonalpolynomials,173 Chebyshev,175 Juliacode,179 Legendre,174 Overow,25 Polynomialinterpolation,76 errortheorem,83 Existenceanduniqueness,83 highdegree,92 Lagrangebasisfunctions,78 monomialbasisfunctions,77 Newtonbasisfunctions,81

PAGE 191

INDEX 190 Polynomials nestedform,38 standardform,39 Power-law,169 Propagationoferror,33 addingnumbers,36 alternatingsum,37 cancellationofleadingdigits,33 divisionbyasmallnumber,34 quadraticformula,35 samplevariance,36 Quadraticconvergence,45 Relativeerror,30 Representationofintegers,25 Rhindpapyrus,41 Rolle'stheorem,96 generalized,96 Root-nding,41 Rounding,29 Runge'sfunction,92 Secantmethod,58 errortheorem,59 Juliacode,60 Signicantdigits,30 Simpson'srule,123 Splineinterpolation,103 clampedcubic,106 cubic,105 Juliacode,109 linear,104 naturalcubic,106 quadratic,105 Runge'sfunction,112 Stirling'sformula,136 Subnormalnumbers,24 Superlinearconvergence,45 Taylor'stheorem,6 Trapezoidalrule,122 Two'scomplement,25 Underow,25 vanderMondematrix,77 Weightfunction,173 Weightedmeanvaluetheoremforintegrals,122