"LI B RAFLY 
 
 OF THE 
 UNIVLRSITY 
 Of ILLINOIS 
 
 SVO.SA- 
 
 COD- 3 
 
NOTICE: Return or renew all Library Materialsl The Minimum Fee for 
 each Lost Book Is $50.00. 
 
 The person charging this material is responsible for 
 its return to the library from which it was withdrawn 
 on or before the Latest Date stamped below. 
 
 Theft, mutilation, and underlining of books are reasons for discipli- 
 nary action and may result in dismissal from the University. 
 To renew call Telephone Center, 333-8400 
 
 UNIVERSITY OF ILLINOIS LIBRARY AT URBANA-CHAMPAIGN 
 
 f^B20 
 
 L161— O-1096 
 
Digitized by the Internet Archive 
 
 in 2013 
 
 http://archive.org/details/useofchebyshevma85golu 
 
UNIVERSITY OF ILUNOIS 
 
 GRADUATE COLLEGE 
 
 DIGITAL COMPUTER LABORATORY 
 
 REPORT NO. 85 
 
 THE USE OF CHEBYSHEV MATRIX POLYNOMIAIS 
 IN THE ITERATIVE SOLUTION OF LINEAR 
 EQUATIONS COMPARED TO THE METHOD 
 OP SUCCESSIVE OVERREIAXATION 
 
 ^7 
 Gene Howard Golub 
 
 March I6, 1959 
 
 (This is Toeing submitted in partial 
 fulfillment of the requirements for 
 the degree of Doctor of Philosophy 
 in Mathematics, June 1959.) 
 
 This vork was supported in part by the 
 Office of Naval Research under Contract Nori-07130. 
 
UNIVERSITY OF ILLINOIS 
 
 GRADUATE COLLEGE 
 
 DIGITAL COMPUTER LABORATORY 
 
 REPORT NO. 85 
 
 THE USE OF CHEBYSHEV MATRIX POLYNOMIALS 
 IN THE ITERATIVE SOLUTION OF LINEAR 
 EQUATIONS COMPARED TO THE METHOD 
 OF SUCCESSIVE OVERREIAXATION 
 
 I'J 
 Gene Howard Golub 
 
 March l6^ 1959 
 
 (This Is being submitted in partial 
 fulfillment of the requirements for 
 the degree of Doctor of Philosophy 
 in Mathematics^ June 1959=) 
 
 This work was supported in part by the 
 Office of Naval Research under Contract Nori-07130. 
 
ACKWOWLEDGEMERT 
 
 The author is greatly indebted to Professor 
 A. H. Taub for suggesting this invest igatio-n and for 
 his invaluable aid in obtaining the results. 
 
IV 
 
 TABLE OF COITEWTS 
 
 I INTRODUCTION 
 
 II ON AVERAGING PROCESSES 
 
 2.1 Linear iterative processes 
 
 2.2 The averaging process 
 
 2.3 The Chebyshev iterative process 
 2.k Estimating the error after k iterations 
 
 1 
 7 
 7 
 9 
 12 
 
 15 
 
 2.5 The behavior of the b 's 
 
 k 17 
 
 ,2.6 A one parameter iteration process ,q 
 
 2.7 A starting procedure for the one parameter iteration process 21 
 
 2.8 The choice of b when > = g^ + Hy 
 
 2.9 A combination iterative process 
 
 2k 
 
 27 
 
 2.10 The vectors r, and z 
 
 k k 33 
 
 2 . 11 The operator G 
 
 2.12 A comparison of the methods 
 III A MODIFICATION OF THE AVERAGING METHODS ]^2 
 
 3 . 1 Int roduct ion 
 
 3.2 Proper values and proper vectors 
 
 3.3 The modification 
 3''^ Bounds for the error vector 
 3.5 Comparison between various iteration schemes 55 
 
 3.51 Comparison of various modified methods 55 
 
 3.52 Comparison between the modified and unmodified 
 methods 
 
 3.6 Bounds for the error vector for successive overrelaxation 
 
 3.7 Bounds for the difference vector 
 
 34 
 36 
 
 k2 
 
 ^3 
 k6 
 50 
 
 63 
 
 69 
 
IV THE NUMERICAL BEHAVIOR OF THE GENERATED SEQUENCES 
 ^.1 The numerical process 
 
 ' h.2 The numerical behavior of tbe one parameter iteration 
 scheme 
 
 ^•3 Terminating the one parameter iteration scheme 
 
 h.k The numerical behavior of the Chebyshev iteration 
 scheme 
 
 4.5 Numerical behavior of the generated sequences when 
 matrices have property (A) 
 
 4.6 Bounds for the error vector 
 ^ A NUMERICAL EXAMPLE 
 
 5.1 Introduction 
 
 5.2 The numerical procedure 
 
 5.3 The numerical results 
 
 5.4 Discussion of the numerical results 
 
 5.5 Conclusions 
 
 APPENDIX 
 
 A.l Basic matrix relationships 
 
 A. 2 The solution of a finite difference equation 
 
 A. 3 A useful inequality 
 
 BIBLIOGRAPHY 
 
 73 
 73 
 
 79 
 83 
 
 86 
 
 88 
 102 
 104 
 104 
 107 
 
 113 
 123 
 127 
 
 130 
 130 
 131 
 132 
 
 133 
 
vi 
 
 The work reported herein was supported 
 in part by the Office of Naval Research under 
 Contract Nori-07130, G.'i- 001. 
 
I IlffiRODUCTION* 
 
 It is frequently necessary for many problems In applied mathematics 
 to solve large systems of linear algebraic equations. These equations arise 
 Often from statistical data and finite difference approximations to continuous 
 problems. For problems of the latter kind the matrix of coefficients contain 
 very few non-zero elements. It is not unkno™ for the number of equations to 
 be as large as 10,000. For such problems iterative methods are often used for 
 solving the system of equations. 
 
 In recent years, a number of iterative methods have been proposed' for 
 solving the system of equations 
 
 Ax = y 
 
 where y is a known N ^ order vector, A is a known matrix of order N, and ^ is 
 the unknown N^^ order vector. For a particular finite difference approxi:nation 
 for partial differential equations of the elliptical t^e, it has been shown by 
 Young [93 that the matrix of coefficients under a suitable permutation appears 
 thusly: 
 
 h 
 
 (1.1) 
 
 Where D^ and D^ are N^xN^ and N^x N^ diagonal matrices, respectively. If there 
 exists a permutation matrix P such that 
 
 P A P = A' 
 
 And A- has the form of (l.l) then A is said to have property (A). If a matrix 
 with property (A) has the form of (l.l) then it is said to have property (A) 
 * III ^peS^l'"' "°''''°" "^"^ '" ^^^^ ^^^ subsequent chapters is defined in 
 Numbers in brackets refer to the references cited in the bibliography. 
 
with ordering o- ^. The purpose of this investigation is to show the relation, 
 between two iterative methods, the modified Chebyshev method (see below) and 
 the successive overrelaxation method, when the raatrix A is symmetric, positive 
 definite and possesses property (A) with ordering cT . 
 
 In chapter II we give the derivation of von Neumann [3] of the 
 Chebyshev iteration scheme which is as follows: 
 
 /k.l = ^\,l (f, - Sy"-^k.l) -^1 (k> 1) (l.2.a) 
 
 'l - io-^y (l.2.b) 
 
 b 1 
 
 k+1 ~ ^ -2 
 2 - \^b, 
 k 
 
 h = 1 
 
 I 
 
 where \ is the largest proper value of G. G and H satisfy the relation 
 
 G + H A = I. 
 It is shown that if H is positive definite, then 
 
 1 
 
 cosh a; = ■-— . 
 
 X 
 
 In addition, we consider ( 1, 2 .a )when h^^^ = b for all k; this itera- 
 tion scheme is called the one parameter iteration process . It is shown that 
 
 ( j = 1, ...,N 
 .|2b-i| 0^-1) ^Ma.^^ |^,/-2 1 13-^^1 1 j (1.3) 
 
 j = 1, ..,n ^ 
 
 where 0^^ (i = 1^2; j = 1, ...,n) are the roots of the polynomial 
 
0^ - 2bA..0. + (2b - 1) = 
 
 and X. (J = 1, ..., N) are the proper values of ,G. For < b < 1, |0. .| < 1, 
 and hence | |x - ^^| | ^ as k -^ - By a theorem of Frankel[l], 
 
 Max |0i,(b)| 
 1=1,2 ^J 
 
 j=l, ...,N 
 Is minimized with respect to b when 
 
 1 + Jl-X'^ 
 and it is also shown that b^ (k > l) decreases with k monotonically to ^. 
 
 The bound given by (l. 3) is actually attainable and consequently for 
 
 b = b, max 
 
 .-> ^1 t _. I 1^ ~ ^kl I °^^y increase for small k. When ^ = g/ + Hy, 
 
 ii5-;ji=ai 
 
 however, for the one parameter iteration scheme 
 
 f II /|h| y/2 
 
 where 
 
 /^ = / 2S - 1 = ^, 
 
 ^ 1. vC^ 
 
 and, hence, max I 1^ " 5,1 I ^^^* decrease as k increases. 
 
 In chapter III we use a device of Riley [4] to modify the Chebyshev it- 
 eration scheme and the one parameter iteration process. If G = j - d"^ A where 
 D is a diagonal matrix whose non-zero elements are the diagonal elements of A 
 then for matrices which are symmetric, positive definite possess property A and 
 ordering o-^, it is possible to modify the Chebyshev iteration scheme and one 
 parameter iteration scheme so that after k iterations, we have computed 
 
;. 
 
 ^2k has N^ components and ^^^^^ ^^^ ^^ components (n^ + n^ = N), 
 
 For the modified iteration scheme, 
 .-1 /^ 
 
 J.= -.=-^(?-^.-U^.,-B^.,).)^^., 
 
 (l.'i- ■) 
 
 where 
 
 L = 
 
 
 
 D = 
 
 r _ 
 
 ^2:-" 
 
 C T 
 
 and If ^ - G^Q + Hy;^ then 
 
 If c^ = b^ with b^ defined as for (l.2.c) then (l.k) descr-^bes the mo dified 
 Chebyshev method and if c^ . %, then (1.4.0 describes the successive over .. 
 relaxation method. Whereas the Chebyshev iteration scheme and the one para- 
 meter iteration scheme require the two previous approximants to compute the 
 new approximant, the modified Chebyshev method and successive overrr-laxation 
 method only reqirire the previous appro-imant. 
 
 For matrices which have property (a) and ordering c- V^e v-ov-r 
 vectors and proper values satisfy certain simple relationships. U.injt th- S' 
 
 th 
 
 relationships, we are able to show that if j?- is the k*^ iterate of the 
 
 1/2 
 
 modified Chebyshev method 
 
 ll^-f^ll /ID 
 
 |5^^'-r'll 
 
 :(i) _ d(i)'i , ^ ( iDl 
 
 I'u 
 
 / 
 
 vhere cosh ^ = ^ :-nd ^ is the largest proper value of G = I - d"^ A. If 
 y^ is the k iterate of the successive overrelaxation method when 
 
then 
 
 f ^ = B- (.-(^) - S/,X)) 
 
 I |3 -h\ I / |D I W2 y 
 
 and 
 
 /^= /^l - tf 
 
 < 1 + /^ • 
 
 Then using a simple argument, we are able to show that for k fixed 
 
 The results of chapters II and III are ^11 arrived at under the 
 assumptions that the numerical operations have been performed exactly. When 
 the calculations are performed on a high speed digital computer In fixed point 
 arithmetic, the operations of multiplication and division are replaced by 
 pseudonnultlpllcatlon and pseudo division. In chapter IV we consider the 
 effect of the roundoff errors. 
 
 For the Chebyshev iteration scheme one computes the sequence [^ ). 
 Under some simple assumptions about the numerical operations, it is shov^^ 
 
 that 
 
 ■^ > b^ >....> bj; 
 
 and that for k > M-, 
 
 \ = Vi= • • • • = ^'• 
 
 Thus the.Chehyshev iteration scheme degenerates into the one parameter itera- 
 tion scheme and similarly the modified Chebyshev method degenerates into the 
 successive overrelaxation method. 
 
 We assume that the digital numbers are represented as a sign vith 
 
^ digits in a number system of base_ p. Then within the computer for the 
 Chebyshev iteration scheme and one paraiaeter iteration scheme, one has 
 
 /^k "^ (k = 0, 1, ...) 
 
 where f 3 indicates a right shift of p^ places and 3^ is a scaling para- 
 meter chosen so that the elements of the scaled approximants are less than 
 one in magnitude. It is supposed that f or k > K 
 
 ^K = Vl ^ • " • = P • 
 A bomxd is given for mln | \1 - (p.^ ^ ^P) . pP, | ,^^^^ increases with X' 
 vhdl-e X- is the largest proper value of G, the digital representation of G. 
 
 Likewise for the modified Chebyshev method and successive over- 
 relaxation method, one computes 
 
 ^ . ' ^k 
 
 Jk7^ (k = 0, 1, ...) . 
 
 A bound is given for min Mx - (7 - rP^ ftP| I ttv,-; v, • - 
 
 B ^x laxii I I A Y^ , p ; f3 I I which increases as jj.' 
 
 approaches one where m' is the largest proper value of G, the digital 
 iNepre^entation of G = I - d"''"A. 
 
 In chapter V, a numerical example is given in which the modified 
 Chebyshev method and two variants of the successive overrelaxation method 
 are used. For variant on., f^^^ . d'^J^^) _ 3^(1)^. ^^^ ^^^.^^^ ^^^^ 
 
 there is no special relationship between J^^^ and f^^\ it is seen 
 that the analytical results of chapter III are vitiated by roundoff error. 
 Nevertheless we conclude that the modified Chebyshev method should be used 
 to solve linear equations for which the matrix A is symmetric positive 
 definite and possesses property (A) with ordering a-. 
 
II ON AVERAGING PROCESSES 
 
 2.1 Linear Iterative processes 
 
 We consider a system of linear algebraic equations in N unknowns 
 
 A5 = ^ (2.1) 
 
 where A is a known square matrix of rank and order N; and if and f are vectors 
 of order N. If A is not symmetric, then we can multiply (2.1) on the left so 
 that A Ax = A 3^, and now A A is symmetric and positive definite. Hence ^ we 
 assume here and in the following that A i£ symmetric and positive definite . 
 
 As has been pointed out by von Ne\miann in [ 31, a large number of 
 iteration schemes for solving (2.1) are described by the equations 
 
 If ? = X, then we requiic f.-, = x, and, hence, 
 
 t = G^5^ + \y 
 
 = G, X + H, Ax . 
 k k 
 
 (2.3) 
 
 A necessary and sufficient condition that (2.3) hold true for all 5f is the- L 
 
 k iSL 
 We shall assume that this conditioii Lolds true in the following. Furthermore, 
 
 -' fk = fk+i' ^'^^'^ h - °kfk * V ^>- 
 
8 
 
 However, from (2,k-) (l j. a ) - tj & ., ^ ^ , x ^ 
 
 ^ ^ ^ '^^ U Gj^; - Hj^A and, hence, (2.5) becomes H^^A^ = 
 
 ^^^""fk = /k+1 implies /k = ^ i^ 
 
 V- 
 
 ^^^ K\^^ . (2.6) 
 
 We: shall assume that this condition is satisfied. 
 Nov subtracting (2.2) from (2.3) ve have 
 
 The solution of this matrix difference equation i 
 
 If Gj^ - G for all k, then (2.7) becomes 
 
 in ^rder for the error vector, 1 -J^, to tend to .ero 1„ (2.8), the initial, 
 errar vector, {t - ^ ^) , „,,t te In the .anifold generated by the proper and 
 invariant vectors of G associated with the proper values which are less than 
 one ln^„agnltude. However, since we do not have any a priori information 
 about 3 -f^, in general, we shall assume that all the proper values of G are 
 
 less than one In magnitude, and, hence t - 9 wi 1 1 +»„/. * 
 
 , ..V.I, iiei.ce, ^ - y^ will tend toward zero inde- 
 pendently of 1 - ^^. Then since H = (I - GIa'^ by (£.3), 
 
 K > rank H > mln[rank (l - G), rank A"^] = H 
 
 and, thus, rani E - S and H ^ exists. Note that for u = ,., (k > o), the con- 
 dition that an the proper values of G be less than one ln'"magnltude implies 
 (£.6). 
 
2,2 The averaging process 
 
 Let us now ccr.sider the ixeratire process ^ - g7 + ^^ ^ 
 f^ OS /k+1 - ^/k + Hy- From 
 
 (2.aj, we see that if the largest pr-op^r va^r*- of r •>, 
 
 e P-oper va^ue of C- in magnitude is near one, 
 
 the iterative process may be slow to coiig<=rge Tr f ^1 
 
 t-Liigergfe.. in [ 3J, von Neumann <3onsider3 
 
 replacing the original sequence / ? >.v . * 
 
 ^ So'S'l' ••• ^y ^ Sequence of averages 
 
 /'O^A' ••• where 
 By (2,8) then . ^ ^ 
 
 Tha^ is, afts''" k it>--^a + Tn->c -r-^ t-v^ 
 
 ^ J. o>-^ A LO'^_aT:ions m the new T)roce'-iq ^'^^.i ^•r^•5^■-,•^^ 
 
 i^jLceob, ^_.e initial error is multi- 
 plied by a matrix polynomial so that ' 
 
 ■'■- V = ^' *''^" "e i^-'e t!is original process and P,(g) = a''. 
 
 Then for a given matrix G, ve wish t-^ dot — i-i^ +^r. 
 
 , c WXK..I ov. ae-csij^lne the necessary and 
 
 s-,;.fficient conditions necessary fo- 11m P fal - n mv 
 
 " tTo/^ ~ answer is well knowji 
 
 and stated in [3]. Let X. (i .. i 2 ^"^ m< -^ i.« ^v 
 
 i V - ±, ^, . . . ,m < nj be .he proper values of G, and 
 
 e. (i = 1, 2, ...) be the order of the elementary divisor of G that 
 corresponds to X., Let the r^^ derivative of P^^(z) be denoted as p(-)(.). 
 Then the ncefesary and sufficient condition that lim P (g) = is that 
 
 ■I j\ 
 
 k^- ^ 
 
10 
 
 m P ^ (x^) . for all those coablnatloas 1(= i, 2, ... „), ,(, ^^ ,^ __ ^ 
 
 for which e. > r. 
 
 Then the choice of polynomial P^(g) „iii depend on the distribution 
 
 Of the proper values and the order of the el«neata.y divisors. It has been 
 
 sho™ by Varga t6] that if the only tao.ledge about the roots Is that they 
 
 lie in a circle around the origin in the complex plane (i.e. \K^\ < ^ 
 
 1 = 1., 2, ... n), the best choice of P (g) Is P (r) r^ tt ,. . . 
 
 ^j^V<^; IS ■f^l.Cr; = G . We shall assume 
 
 that the proper values of G are real and satisfy |\ | < x. 
 
 Let us assume now that H is real, symmetric, and positive definite 
 Since this covers certain important applications and allows the use of some 
 rather effective methods. Therefore, there exists a matrix F such that 
 f/ =^H. Since H and A are symmetric, E'^ - A is symmetric a^d likewise 
 F (H-1 - A)F. Then there exists an orthogonal transforation such that 
 
 fV^-a)f = oao , (,^^^) 
 
 Where A is the diagonal matrix of proper values of F^H'^ - A)F. Pre- 
 multiplying (2.10) by F and postmultiplying by F'^, we have 
 
 FF*(H"^ - A) = FOAO*F"^ 
 so that 
 
 H(H"^ - A) = F07lO*F"^ . 
 But S{r^ - A) = I - HA = G by (2.4) and thus 
 
 G = FQA0*F"-^ . , 
 
 (2.11) 
 
 Then (2.11) expresses G in the Jordan canonical form with A the matrix of 
 proper values of G. Since vl is diagonal, the elementary divisors of G are 
 simple and real. 
 
Substituting (2.11) into (2.9), we see that 
 
 X 
 
 Since (2.11) yields G in the Jordan 
 
 canonical form. G = FO-A''b*F~''". 
 
 Thus 
 
 P.(G) = Z 
 
 V FO^'^0*F"^ = F0| E a aWo*F"^ 
 
 = FOR (7l)0*F~-'- 
 k 
 
 Because TV is a diagonal matrix. 
 
 P,(-A) 
 
 p^(\^) 
 
 ^ ^Ic^^n) 
 
 Now (5r -|^) . ?^(G) (5e - ^q) . FOP^(yl)0*F-l(? -^^) and hence by (A.2), 
 (A. 6), (A.8), (A.9), (A, 10), (A. 11) 
 
 ii^-r.iiii 
 
 H 
 
 1/2 
 
 u 
 
 ihT/ 
 
 Max 
 i=l,2. 
 
 ,N 
 
 I P.O.,) 
 
 X - 
 
 /o 
 
 (2.12) 
 
 Naturally, we wish to choose P^(x) so that Max |P (?^.)|-^0 as k -^ ^ . 
 
 i=l 2 . ,N 
 Since the elementary divisors of G are simple/ the 'choice of P (\) will depend 
 
 on the distribution of the \. 's. ¥e shall assume that we do not have any 
 
 exact knowledge of ,\^, ... K^ i,ut simply the information 
 
 11 
 
 -X < X. < X 
 
 — 1 — 
 
 (1 = 1, 2, ... i^) 
 
 (2.13) 
 
 If a < \^ < \^ . , , < \^ < b, then by a suitable linear transformation, (2.I3) 
 
 will hold true. Then for | |? - J^l | fixed, we wish to choose P^(x) so 
 
 that MaxJPj^(\)| is minimized subject to the condition P, (l) = l since 
 |/c |<X k 
 
k 
 
 =0 
 
 It is veil known, ef. [3], that the desired polynomial Is 
 
 
 (2,14-) 
 
 where 
 
 1 r._ . ,r2 
 
 \i-) = i[(z Wz2 .1 )^^ (, ./;2-77 ^k J 
 
 th 
 
 thek Chebyshev polynomial. If | z| < 1, thenl^(z) = cos k (arc cos z), 
 and for |z| > i, r^(z) = eosh z(arc cosh z). Thus 
 
 cos k ;^ 
 
 V^i^ ^ cosh k <^ ""^^^^ cos Q.- = ~ and cosh 6<^ = -^ 
 
 Then (2.12) becomes 
 
 \l/2 
 
 "^-^^"^(Sjj 
 
 l-"-fol 
 
 cosh k tu 
 
 2.3 The Chebyshev Iterative process 
 ^ k 
 Since p^ =^Z^ a^^^, it would seem that it is necessary to store 
 
 all the vectors ^Q, ' ' - /k i« order to compute the vector f^. However, 
 the Chebyshev polynomials satisfy simple recursive relationships which we 
 now proceed to take advantage of. The equations developed here were 
 originally obtained in [3]. 
 Since 
 
13 
 
 \-^l^^) -^ Vl^2^ = 2zR^(z) 
 and, hence, 
 
 •'m(-) = 2^V-) - \.l(^) . (2.15) 
 
 The Initial conditions are R^(z) = 1 and Rj^(z) = z. 
 
 Mow we use (s.lt) to pass to P^(x.). Then (2.15) becomes 
 
 and dividing through, by E^^^i-^)^ we have 
 
 1 \ / X, 
 
 Mr 3x H,/-^ H^A 
 
 R , /^ R, . ^ 
 
 V. (tj ^ »„(tj ■. (tJ " Mr? ^j 
 
 Using (2.14), the above relationship becomes 
 and therefore, 
 
 ^k.i(\) = ^iVA(^i) - ^\VA.i(\) (2.1,) 
 
 where 
 
 .. . ""'(t 
 
 ■^ XR^/4-\ ■ (2-18) 
 
 From (2.15), we have 
 
Ik 
 
 \(^] ^.. (^ 
 
 _2 "KXJ T'- 1 1 ^ ; 
 
 9Q tliat using (2.17) 
 
 k+1 . \"k+l -^ (2.19) 
 
 or 
 
 1 
 
 'k+1 2 _ ^2^ ^1 = 1 • (2.20) 
 
 Also we are able to rewrite (2.I7) using (2.I9) so that 
 We now pass back to the original process which yields from the preceding 
 
 equation 
 
 \.X^^) = 2\.l[°Pk(°) - P,-l(°)] - P,.i(G) . (2.21) 
 
 Postmultlplytng (2.21) by {1 -^^), „e have 
 ^k.l(°) (? -/o) = 2\,i[GP,(G) (? -^g) - P^_^(G) (Sr .^;)] 
 
 ^Vl(0)(5-fo) • (£.22) 
 
 Slr.ce ? -^ = Pj^(G) (J -j* ), (2.22) now becomes 
 
 ^^ -^l' = '\*l''^^ -%^ - (J - %.^l ^(-^,.,) . (2.23) 
 rToB (S^>, X- - Gx = Hy so that- the above becomes 
 
 fk.l - 2\.l(sf, ^ Hy* - ?,.,) . f^_^ (k = 1, 2, . . . ) (2.2U.a) 
 
15 
 
 and since 
 
 
 = X. 
 
 ''^(t^ ^ 
 
 =^lled the Chebychev Iteration _scheme . 
 
 2.4 MlE«££ the errOT after k Ueratlons 
 
 For the Iteration process ? = o ? ^ u ■* 
 
 /k+1 \Jk + \y' we wish to determine 
 an upper bound for 1 1? - <? I I *• 
 
 , . - 11- /,l I froM the numbers which have been computed during 
 
 the Iteration process. Since 
 
 we have by (A.ll.b) 
 
 In addition, since 
 
 /k+i-/k = °k/k- V-fk 
 
16 
 then by (A.ll.b) 
 
 
 vould slMply need to Iterate until /kM < ^^ one 
 
 
 We note that I I -^ I I = I !^ 1 j n • ^ 
 
 IKW - lt\f| ve W When ve are close to a solution. 
 
 On the other hand, for the averaging process It .. . 
 + ^, M-^ -* process It is not necessarily 
 
 true that ^ ->|| n --p . ^ _^ 
 
 ' 7k+l ?'kl I = If and only if I^ . ^[ I _ u 
 
 ^ ' Y k ^fl -0- We may, however, 
 
 compute 7 vex. 
 
 and thus by (A.ll.b) 
 
 
 \t 
 
 
 The residual vector r QH--?n v ^-i .^ 
 
 , ,. , , -^ ^'' '^"^ ^•'^ 5"^^-*^' i l?,l I - If and only If 
 
 I \r^\ I = 0. Then for exact calculations, to insure 1 I? - 3 I I < e 
 
 „ . -, ' ' /]rl\ S^? one would 
 
 simply need to iterate until ^ 
 
 mm 
 
 < £ 
 
17 
 
 2.5 The behavior of the b 's 
 
 ^For the Chebyshev it^rati6n sch#te, it is hecessary to compute b^ 
 
 for each ^ by the relationship b , ^ i-— v . t, ,. , ,„, 
 
 f ^ ^ 2 5C^ "" 1- ^ equation (^.18), 
 
 k 
 
 vf 
 
 
 ''^^'''^ '\( -r- ) - cosh k ^, cpsh ^ = -4c 
 
 ^ 5: 
 
 Since cosh oo = -■ '*' ?, 1- 
 
 6*^= J- + Ji'->? 
 
 (2.25) 
 
 Theorgn (s.l): The b^'s fo™ a strictly monotonlcally decreasing se^ence 
 Whose limit is '^ = - 
 
 1 + /l - \^ 
 
 Proof : We have b, > b if 
 
 k k+1 ^^ 
 
 "•-''ii . -^(t 
 
 XR, /-J^l XR, /J^ 
 
 or 
 
 coshU - l)^.cosh(k + 1)1^ - cosh^ k o; > . ^ (2.26) 
 
 But cosh(k - l).t/cosh(k + 1)^"= cosh^k C^ cosh^ u/ - sinh^ k o^^sl^h^o; 
 ao that (2.26) becomes 
 
 CQsh^ k c^ cosh^Lo - sinh^ k uj Blni? u; - cosh.- k cO > 
 
18 
 
 or since cosh^ to _ sinh^ i^ = \ 
 
 1 fjj( n^c\n U //; _• •■ 2 
 
 sinh ^(cosh k ^ - ^inh^ k uj) = sinh^ ^^ > o 
 
 1 + e-2^ 
 
 Since b = e + p> ^__ e 
 
 k+1 ^j^(k+l)a.;^-(k+i)u,J = -^ 
 
 \ 
 
 1 + e' 
 
 ■2(k+l)^ 
 
 andO<e-^<l, l^^^l = ^- However, t^y. (2.25) 
 
 1 + /l - }? 
 
 and thus 11m b = L _ ^ 
 
 k->- ^-^1 i + /rr^ • 
 
 2.6 A one parameter iteration process 
 
 As was Shown in the last section, the b^'s form a monotonically 
 decreasing sequence and converges to a limit ^. Let us denote the machine 
 generated b^'s as b^. Then it will be shown in Chapter IV that under suit- 
 able assumptions for some k>M,b=:b - ^.i 
 
 - •" ""m M+1 ~ = ° • Hence the 
 
 Chebychev Iteration process when carried out on a computer ultimately de- 
 generates to a process in v^ich b^ = b. In this section we discuss such a 
 process ab initio . 
 
 From (2.11), we have the elementary divisors of G are simple, and 
 thus we have 
 
 G = m-^ (2.27) 
 
 Where JL is the diagonal matrix of proper values. We wish to consider the 
 iteration formula 
 
19 
 
 Where ^^ and f^ are given. The computation scheme described by (s.28) will 
 be called the one paj^meter Iteration scheme . Then from (2.23), we have 
 
 (^^-^.l)=2^[°(^-;,)-(2-^,.,)l.(?.f^.^) (,>i) (2.,5j 
 
 and thus substituting (2.27) into (2.29), 
 
 Premultiplying by Q'^ and setting u^ = Q"^(^ . ^ ) 
 
 \+l = 2bA^ + (1 - 2b) \_^ 
 or since A is a diagonal matrix 
 
 •^J^k.l = 2b"j,k + (1 - St) u.^^_^ (J = l, 2, ...N) (k>l) . 
 
 The solution of this difference equation Is given by lemma (A.l). Returning 
 to the vector equation (2.29), we have 
 
 - ^^ - P ' ^K«"'(? - %) - ^^^ ■ ^) «S,-l«-'(? - %) (2.30) 
 
 IJ " hi " 'IJ '■ ''2J 
 
 ""'"' [ 'i^iio = jfr^. " ^ - J »^ ^1, / 0, 
 
 = k?!^-'- if 1 = J and ^^_j . 0g^ 
 =0 If i / J 
 
 and ^^j and 0^^ are the roots of the equation 
 
 0f - 2bXj0j ^ (2b - 1) = (J=l, 2, ...H) . (2.31) 
 
20 
 
 Thus since Q = FO by (2.11) anri' TTir* it ^ . , . , 
 
 ^J K^.±±) and FF = H, we h^ve by (A.l), (A, 2), 
 
 (A,3), (A.6), (A.8), (A,9), (A.IO) and (A.ll) 
 
 '""^^"^(w7)"f,.,^-,JK)n 
 
 
 + jSb - ll Max Ifs i I !i;r _ 2 
 
 ,.,^..JK}..lil^-^Jlj. (..3.) 
 
 f -^ ^1 i " ^On k-1 
 
 Since j S, { . . = -i^ isL _ v ^m ^k-m-l ^ , 
 
 Max |(SJ..| < K M.x(|0^j|>^-1, |^^.|H-1). ,^^^ ^^^^ ^^ ^^^^^^^^ ^^^^^^^^^ 
 when 0^^ = 0g^. Thus (2.32) becomes 
 
 1/2 
 
 e 
 
 j=l,2, ..N 
 
 + |2b - l| (k - 1) Max 10. 1^-2 | |^ -^ | | 
 
 j=l.'2, ..n , - ^ 
 
 (2.33) 
 
 Since I 1^ - ^0 1 I and I 1^ - ^J I are fixed, ve wish to detemine b so that 
 
 Max 10 (b)| 
 i=l,2 ij 
 j=l,2, .,N 
 
 is minimized with respect to b. The problem has been considered by Frankel 
 [1] and he has shown: 
 
 aeorem (2.2): If x 1. the largest proper value of G and -X is the smallest, 
 
 then for b = t = —1_ , the Max |0 (b)| Is minimized with 
 
 ± + vi - A. i=l, 2 
 
 j=l,2,.. 
 
 respect to b and 
 
21 
 
 Furthermore, for 1 > b >^, |0 . .(b) | = (2b - l)V2_ 
 
 Let 0^ = b?: + ^^^ - 2b + 1 and = bX - /^ 
 
 2 , u /v - 2b + 1 
 
 <> . 2r2 
 
 - yb + 1 = and (^^ = (a , 
 
 then for b = b, b X - 2b . 1 = o and ^^ = 0^. i^n (2.33) beco.es 
 
 \l/2 
 
 X - -- 
 
 r ^ , r- (2.3t) 
 
 where p =i'Sa - I. a i^. 
 
 2,7 A starting procedure for the one parameter iterat 
 
 ion process 
 
 If t IS the proper veetox associated with -\, and t -% = x - ^ 
 ^ ( o /I 
 
 = ae, then from (2.30) 
 
 II" -fi^ii -(t%) f^'"'^" " '" ■ 'V') 11^ - foil 
 
 we see that although ^'^\x + (k - l)/)l-^ a^ > -^ ^ +>,^ 
 
 -o ^ LX5. -r v-cl x/^j-? u as js -^ 00, the maximimi error 
 
 may actually grow during the early iterations. From (2.30;, we note that 
 
 by choosing ^^ carefully, it may be possible to obtain a cancellation of 
 
 terms. In the following theorem, we give a means of accomplishing this. 
 
 Theory (2.3): If ^^ = G^^ + ^, and if 1 > b >^, 
 
 il^'-ill^lSjj'^^f-^^.jllx-foll 
 wh&re r> = i/2b - 1 . 
 
Proof: Since ? - 7, = G(i? - ^ . ^,-l(^ . y^ (,,3^) ^^_^^ 
 
 ?-f, = Q[s^A.(£,.i)3^.^j«-V-p. (2.35) 
 
 Since all the roots of (2.31) are equal 
 
 m modulus for 1 > b >'S, we can 
 
 represent them in polar form, i.e. 
 
 Plj =V2b - 1 e J = (Oe J 
 
 02j =\/2h - 1 e "^ = ^e ^ 
 
 Hence 0^. . 0^ . = 2^ cos 9. . But by (2.31) 0^. . 0^. = 2bX. so that 
 
 cos 9 . 
 \. = i 
 
 Now if 0^^ ^ 0^^ (i ^ 1^ 2, ..n) 
 ^ ^ k-ljjj r sin 9. b 
 
 cos 9 . - c 
 
 sin(k - 1)9. 
 
 sin 9. 
 J 
 
 L 
 
 ^1 sin k9 cos 9. sin(k - 1)9 
 
 -si li_ j 
 
 sin 9. b 
 
 J 
 
 sin 9 
 
 sln(k - 1)9. 
 But since i 
 
 sin k9 
 
 sin 9 . 
 J 
 
 cos 9 - cos k9. , 
 
 sin 9^ j J 
 
 the above becomes 
 
 -L \ sin(k - 1)9, 
 t» J sin 9. 
 
 ~ + -r— cos k9 . 
 
 < 
 
 '^-.)i...).^ 
 
 since 
 
 sin k9. 
 sin 9 . 
 
 < k 
 
Because Sb - 1 . f , ^^ ,ee the above becomes 
 
 f 
 
 .i^ 
 
 1 + — ^ k 
 1 V 
 
 If the roots are equal In magnitude, then the above Inequality be- 
 comes an equality. Now taking bounds of (2.35), we have 
 
 If b = ^ = 
 
 J. 
 1 + h -1? ' *^^^ ^^^ ^^°'^^ inequality becomes 
 
 '^-?^"^fSi7rV(-'^^"'-?°" 
 
 By the following calculation, ve show that i; 
 
 = G^ + Hy then 
 
 Max 
 
 X " 
 
 foll^o 
 
 1"^ -^ 
 X - - 
 
 fkil 
 
 \T^^ 
 
 F 
 
 miist decrease with k for b = '^. 
 
 Now 
 
 |Hi,A^/2 
 
 H 
 
 'U 
 
 u 
 
 if 
 
 
 2 
 
 or 
 
 \X{^-\) +(^] 
 
 1 -e; 
 
 1 + f ^ 
 
 < 1 - 
 
 r 
 
 But since < /^ < i^ 
 
2h 
 
 since f^, = G^g + ^, 
 
 where 
 
 and 
 
 \^j_iG) = 2bGPj^(G) + (1 _ 2b) PjG) 
 
 Po(G) = I and F^(G) = G . 
 2.8 The choice of h when f = Qp + jif 
 
 Although lb minimizes Max 10. (b) I i^ith rP^T.Pr.+ +^ >.- •+ • 
 
 i=l 2 ij ' respect to b, it is not true 
 
 J=1,2,.M 
 
 that Max f ! x - ^ I I -? o • • • , 
 
 I Ix-^ 1 Uo ' ^k" ^^ mnDjmzed with respect to b when 
 
 7i = Gfo - =^ -nd k is fixed. For J. = 2, the Chebyshev Iteration sch^e 
 
 yields the polynomial 
 
 Sb^G^^ (1 . 2b2)l . 
 Thus the best choice of b when two itez^tions are perfox^ed is b = b^. m 
 ge4eral, k is not predete^ined but rather one iterates until either^] |?J | 
 or I |?J i is sufficiently small. From the following we see that the opti- 
 mum b is arbitrarily close to ^ for k large. 
 Theorem (2.4): Let 
 
 Pj^^^(\,b) = 2bXP^(x,b) + (1 . 2b) P^ rx,b) 
 
■with 
 
 PqCX,!)) = 1 and P^(\^b) = ^ . 
 Then for ^^ = G^Q + Hy, 
 
 , Max. l^iK'^)] 
 Proof ? From theorem (2.3)_, 
 
 wjjare ^ = /^ - 1 . and 
 
 Where and are the roots of 
 
 1.) ^2j 
 
 0^ - 2t)\ + (2b - 1) = 
 
 1. Let < b < ^, then 
 
 0^ -0^ 
 
 ^^^^^ 01 = t)x + /t^^Tim 
 
 02 = t^ - /b^'x^- 2b + 1 . 
 0-L and 02 are distinct for < b <'^ sinceb^ - 2^ + 1 = 0. 
 
 Then since 
 
26 
 
 03_ + $2^2 = 3bX, 
 
 V^,b) 
 
 
 0f 1 . 0^^-i 
 
 '1 =^2 
 
 1 /^r-^r 
 
 2b 
 
 2 ^r-^r 
 
 ?r^^--(i^A)' 1-^ 
 
 Now 
 
 Max_ IP (\',^)| ^ 
 
 - Max_ |P^(X,t)I ^ TP^(X7b)T 
 
 I A, |*^Ai 
 
 21) 
 
 ^^1 + k /l - X^ ) 
 
 k+1 , ,. ,, .k_i 
 
 1 - i^^/^^r'^ ^2 1 " (y^i) 
 
 1 - ^ v^i) " ^2 rrry^ 
 
 Now 0^ > ^ Since ^ minimizes the Max |0, , (b) |, and, hence, 
 
 1=1,2 ^ 
 
 j=l,2, ..N 
 
 lim 2b 
 k-^oo 
 
 /-^j (1 + k yr - ^2) 
 
 = . 
 
 ftirtherrtore, < 0^ < so that 
 
 ^ - (^g/i^l)'^"' . 1 - (02/0,)''-^ 
 
 1 - ^fz'h^ -hT-7JJJ^ 
 
 1 -0; 
 
 k->^ 1 
 
 ■^?7^ 
 
 / 
 
 Max \V^{\^)\ 
 Thus for < b < ^, lim \\\<\ 
 
 k-^oo Max |P (\,b)| 
 |X|<\ ^ 
 
 2. Let b < b < 1, then 
 
 Pk(^j.t) = (V2b - 1")^ 
 
 (^-) 
 
 sin(k - 1)0. cos k9 . 
 
 sin 9 . 
 
 ^ + 
 
27 
 
 cos = ■ J _■ . Hence, 
 "^ /2b - 1 
 
 Maxjp U,-^)! 
 
 o< 1^1^^ 
 
 ^^1 + k JT^W) 
 
 (2b - 1)^/2 Max 3- I (^^ - 1) sln(k - l)Q cos k9 
 
 72b -1 
 
 Again since 
 
 (^ < y(2b - 1) for % < b<T ehL±±Jl~[Sl ^ n . ^ 
 
 (2b - 1)^ ^° ask-^o.. 
 
 Nov 
 
 lim 
 k-^oo 
 
 /-i- _ 1^ sin (k-1) Q cos kQ 
 V b y sin + -b 
 
 does not exist, and hence 
 
 Max _ 
 .cos9|<=^ 
 
 /_!_ _ ;l^ sin(k-l) Q CO 
 \^ b- y sin 9 
 
 s k9 
 
 -> as k 
 
 Therefore Tor's < b < 1, 
 
 lim |\|sa" 
 
 Max_ |P^(\,'^)| 
 
 k-*"" Max_ |P (X^b) 
 
 I.I j£ 
 
 "^0 as k-> "^^ 
 
 |x|<\. 
 
 2.9 A combination iterative process 
 
 As was noted In section {s.6), the computed ^^' s, b^'s, become 
 constant for k > M. For this reason we wish to consider two additional pro. 
 cesses for solving linear equations In this section. 
 
 Process 1 '>? *M ~* 
 
 ■"• ^0' 7^""'7m ^^^ generated by the Chebyshev iteration scheme 
 
 described by (2.24) and (2.20). For k > M ^ ^ 
 
 y rui K ^ m, >7j^^^, 7m+2*°- ^^® generated by 
 
 the iteration formula (2.28) with b ='^. 
 
^2=ess 2. The computation scheme Is as follows: 
 
 Mp+q 
 
 ^^d l3»4„ -, = 1 
 
 Mp+1 • 
 
 Then for process 1, after (M + q) iterations. 
 
 X - 
 
 ?M+q = ^q+iQ" (x - fl) - (2b - l) <^q'\^ . ^. ) 
 
 q (0 
 
 Where S^ and Q are defined as for (2. 30). Then 
 
 since 
 
 ^ ■ ?1 = ^ - ?M = ^^ihl^W^V^x - f^) 
 
 and 
 
 ? - ^. - ;? _^ 
 
 <^0 = ^ Ym-1 = cosh(M - 1)6^ '^^u.i^'h^ - ^q) 
 
 where 
 
 / ^IW 1 • ■ ~ r>nc IWQ rv ~1 .1 
 
 
 = cos M9., 9. = cos" -«i 
 J J 
 
 if i ?^ J , 
 
 X - 
 
 Hence, 
 
 ^to(q . 1)9^ cos M9^^ ^^^ Sin 49 oos(M - 1)9. 
 f^ sln9. coshM«-p "ilTg: cosh(M - lja> (2-36) 
 
29 
 
 M -M 
 where cosTi Mc*> = ^ — ^ f . 
 
 We vish to determine the maximum of (2.36). 
 
 Now 
 
 Max 
 O<0<7T 
 
 q sin(q+l.)Q cos MQ d+1 sin q@ cos(M-1)0 
 ^ sin cosh M<^ " r 
 
 Max 
 O<0<F 
 
 Max 
 0<9<F 
 
 Max 
 O<0<7r 
 
 sin cosh(M-l)6o' 
 
 q /sin q0 COS ^ ^^^ ^ COSM0 _ 0.1 sin^ cos(M-1)0 
 ^ V ® ^°s^ M6^ r sin cosh(M-l)^ 
 
 h(M-l)6<.>, 
 
 .cosh M 
 
 cos 
 
 cosh M^ cos 
 
 :q+i 
 
 cos M0 - P^"^^ cos(M-l)0 \ sin q0 q cos q0 cos M0 
 ' cosh(M-l)^i sin "" r cosh M^ 
 
 f;" , V 1 sin q0 cos(M-l)0 q cosq0 cosM0 - sinQ sinM0 
 h(M-lM sin ■" e coshMco 
 
 - I cosh Mo; ■ cosh {U-l)c4j ^ "*" 
 
 cosh U(aJ 
 
 since 
 
 Thiis 
 
 sin q0 
 sin 
 
 £ q. by lemma (A.- 2) 
 
 /C- 
 
 Theory (2.5): If after M iterations with the Chehyshev process, 'b is used 
 then 
 
 
 .q+1 
 
 cosh May cosh(M-l)6i> 
 
 q + 
 
 cosh UuJ 
 
 ^^ere ^ = ^ and cosh M^ 
 
 1 + 4 - X"^ 
 
 - ^^ ^ ^-" 
 
 We now wish to compare the maximijm error of process 1 with the 
 maximum error of process 2 when \\^ - x| | is fixed. 
 
 ^eorem (2.6): If after every M iterations with the Chebyshev process we 
 begin the Chebyshev iteration scheme over again, i.e. process 2, then after 
 after Mp+q iterations the maximum error of process 2 is greater than or equal 
 to the maximum error of process. 1 for | |^ - x| | fixed. 
 
30 
 
 Pro2£-" If the Chebyshev sequence is rebegun after 
 
 every M iterations^ then 
 
 ig 
 
 )+q 
 
 -^1 
 
 t^^ 
 
 < 
 
 |Hf 
 
 \V2 
 
 u 
 
 H 
 
 MJ (coshM^)P cosh qu> • 
 
 After Mp+q iterations with process 1, then 
 
 \l 
 
 X 
 
 H 
 
 M+q ■■ ^ / • -u 
 
 \l/2 
 
 A 
 
 f 
 
 ,M(p-l)+q ^M(p-l)+q+l 
 
 .-l)+q) 
 
 J 
 
 ^2^_^>^^ II y M(p-l)+q 
 
 coshM^ - cosh(M-lW /r(p-l)+q^. ^^^^^^ 
 
 We wish to show 
 
 ''^M(p-l)+q J4(p-l)+q+l 
 
 cosh Mou " cosh(M-l)^ )(M(p-l)+q) + 
 or multiplying through by cosh Mu) 
 
 pM(p-l)+q 
 cosh Kui — 
 
 (coshM^)P cosh q^ 
 
 M -M 
 Since cosh Mm. = ^ g ^ , the above inequality becomes 
 
 )-l 
 
 1 
 
 Mco cosh qco' 
 
 (.M(p-l)+q M(p-l)+q+l r'^1 + (r^f 
 
 M(p-l)+ql +^M(p-l)+q 
 
 < .^ihsh^-^ 
 
 2p- 
 
 ^(pFp 1 . (f 2) 
 
 or 
 
 i.i^inif 
 
 1 . ((>^f-\ 
 
 M(p-l)+q) + 1 < 
 
 (^^WP^T^ 
 
 • C2.37) 
 
 (1) 
 
 We consider two cases, 
 q = 0. 
 
31 
 
 2 
 Let jo = t, then we wish to show 
 
 1 + t^ 1 ,.. . oP-1 
 
 -— ^iM(p.)..,_^ 
 
 1 . t^-y ^^^^ ^^ ^ - A . .MW-l • (2.38) 
 
 Now for i = 0, ._,... M-1, 
 
 t^-1 < t^ 
 
 M-1 . ^ M 
 
 and, hence, Mt*^""^ < T. t^ = ^ - '^ 
 
 i=0 1 - t 
 
 so that Mt^-l(l - t) + t^ < 1 , 
 
 Then (1 + t^) ^ Mt^-l(l - t) < 
 
 or 
 
 2 
 
 .. . [(1 + t^) +Mt^-^1 - t)]P-l <2P-1 . 
 But by the binomial theorem, 
 
 (1 - t")5-l . „(p.l)t"-l(l . t)(l . t")P-2 < [(1 , t«) . Mt«-^l - t)]P-l < sP-l 
 IH- M(p-l)t" (J- - *) < g"'^ 
 
 and since < t < 1 
 
 1 . M(p-l)t^ -11^ < 2^-^ 
 
 1 + t^-^ - d + tV^' • • 
 
 and thus we see that (2,38) holds true, 
 
 (2) < q < M - 1 . 
 
 Again since < t < 1, 
 
32 
 
 +M-1 . , 1 
 
 ^ < t for i = 0, ., ...M-1 
 
 and t^*^"*"^ ^ , I+M-I ^ 
 
 ^ ^ * for 1 = 0, 1, ...q . 
 
 Thus adding the two inequalities, 
 
 t^-V . t^) < (1 . .M-1^ ,i (i = 0,l,...,<M.l) 
 
 or qt^-^d + t^) < "^Z (1 + t^-1) t^ = (1 + t^-h 1 - t'^ 
 
 i^O ^ 1 - t 
 
 which implies qt^-1 (l - t) 1 - t^ 
 
 1 . t"-i - rrr^ 
 
 Since 1+ t -^(1 - t)M(T)-l) ^ 2^"^ 
 
 1 + t^-l - (i^tV^ 
 
 q 
 
 But 
 
 1 , t -V - t)M(p-i) t;^:!^^^ ,p-i , . , 
 
 ~ (1 - t")P-i (1 + t^) 
 
 2^ 
 
 (1 - t")P-^ (1 + t*) 
 
 and, hence. 
 
 ^M-l[l - t][M(p.i) + q] 
 
 1 + 1 .- . < ^ 2 
 
 P 
 
 1 - t^"' - (1 - t^)P-^l . t^) 
 
 and thus (2.3?) holds true. 
 
33 
 
 2.10 The vectors r, and "z 
 k k 
 
 In section (2.2), our choice of polynomial P^(g) was detemlned by our 
 de.ire to minimize | |^^ - 1| | for | |,^. -'5^1 ( fixed. Let 
 
 (0 ^ff -Lxxea. hez us now consider the 
 
 effect of the choice of polynomial on the residual vector ?^ where ?^ = J - A;? 
 and on t^ = G^^ = H^ - ^^. Again we shall assume H is positive definite and 
 
 symmetric. 
 
 since (x - ^j^) , Pj^(G)(J .^^), „, ^^^^ ^^ premultlplyi,^ the 
 atove by A that 
 
 But since \ = ? - A^^ = A(3 - ^^), (J . ^^) . ^'^ and h.nce, (£.39) be- 
 
 comes 
 
 since 
 From. (2. 11), 
 
 AGA-1 = A(I - M)A-^ = (I - AH) = H-^d - HA)H = H'^GH . 
 
 H"^GH = (F*)-lojloV . 
 
 and therefore 
 
 ti^-^th^; i=i,2,..i*'^ -(i^j i^i<^ 
 
 As was shown in section (2.4), \ = G^ + H^ - ^ = (i . g)(x -^.^) 
 
 (I - G)(:r - J^) = (I . g)Pj^(g)(i - g)-1(i - g)(1 - t) 
 
 Max, |p^(\)| . 
 
 Then 
 
 and hence, f^ = Pk^G)^^ . 
 
3h 
 
 Consequently^ 
 
 ll\IISlV°'iuli^0ll</]5~i=^..N"'^^'^^' "'oil 
 Thus we see that the Chebyshev polynomial simultaneously mini 
 
 minimizes 
 
 X - - ' ' ' ' 
 
 ,5^,, ^'^ Max IK 
 
 l='foll 11^ -foil ' ll^oll TW^ 
 
 and Max ''\| 
 
 ^oll ll^ol 
 
 Max N^k' 
 
 In [5], Stiefel chooses the Chebyshev polynomials to minimize 
 
 l^kll^^ 11^0" ' ^"^ ^^^"^ °^'^^^ ^^""^^ analysis, the iteration scheme of 
 Stiefel is equivalent to the Chebyshev iterative sequence. His choice of G 
 is G = I - oA. 
 
 2.11 The operator G 
 
 We have so far imposed the restrictions that H be positive 
 definite and symmetric and that the proper values of G lie between -\ and 
 X. Two operators which are simply related to the matrix of coefficients of 
 the original equations to be solved, the matrix A, and which will be shown 
 to satisfy these properties are 
 
 (1) G = I - OA (a > 0) 
 
 (2) G = I - aD"-^A (a > 0) 
 
r 
 
 11 
 
 where 
 
 22 
 
 35 
 
 nn 
 
 In the next chapter, it will be shown that for certain matrices the second 
 operator affords great advantages. : 
 
 If A? = vl>% then y = ^^ , and since A is positive definite 
 
 z Dz 
 
 ^^ > 0. Thus the proper values of D'^A and A are positive. 
 
 Let C be a matrix whose proper values are positive and G = I - aC, 
 Then if the proper values of C are < a < v, < . . . < ^^ < ^^ i, i3 ^^^^ 
 in [2] that ^ = -^ minimizes the largest proper value in magnitude of 
 
 G = I - aC. If a = 
 
 ^ = 
 
 a + b ' "^^^^ ^^^ proper values of G are distri- 
 
 buted between -/^-~^] and J^ ' ^ 
 
 b + a 
 
 b + a/* 
 
 If G = I - 
 
 polynomials become 
 
 a + b ^' then the iteration formulas with Chebyshev 
 
 'k+1 
 
 2b, , (I - - 
 k+1 V a 
 
 ^b^^^k^rTb^-^k.i^-^k.i (k>.i) 
 
 ?i 
 
 = (I 
 
 a + b ^^fo + TT"b 
 
 (2. 40. a) 
 (2.40.b) 
 
 Since the residual vector is ?^ = ? - a|, the above becomes 
 
 ?k+l 
 
 = 2b 
 
 k+1 
 
 a + b 
 
 ^k ^ f k - A-i] ' %., 
 
 (k> 1) 
 
 % = i-Tb ^0 ^ f c 
 
 (2.41) 
 
 If we wish to solve the system of equat 
 
 ions 
 
36 
 
 where A^ is asynmetric, then as was pointed out in section (2.;l)^- it is 
 possible to solve the equivalent system 
 
 where 
 
 A = A% and ? = A*y, . 
 
 Although a large number of elements of A^ may have been zeros, they may be 
 destroyed in computing A^A^. This can be avoided for the operator G = I - oA. 
 Instead of performing the matrix multiplication, one may compute ^ in 
 equation (2.4l) by first computing 
 
 and then computing ?^ in equation (2.4l) from 
 
 This corresponds to letting G = I - aA*A , and H = oA*. 
 
 2.12 A comparison of the methods 
 
 Let us now compare the number of iterations that are necessary so 
 
 that 
 
 
 If we use the iteration scheme described by (2.2) with G = G then 
 
 k' 
 
 
 u 1 -k 
 
I 
 
 We wish to determine k so that 
 
 r /\-\.Y'' 
 
 or 
 
 where 
 
 ^1 -"' - e 
 
 \ 
 
 
 ■■m 
 
 6 
 
 Then, 
 
 k = ^n 6 ' 
 1 T—T- 
 
 Xn \ 
 
 If, however, the Chebyshev iteration scheme is used, then 
 
 Since cosh ko/ = ^ , we wish to determine k^ so that 
 
 ® + e = -— and thus e '^ = ^ 
 
 ^- i + ^rrTT^ • 
 
 Consequently, 
 
 A 
 
 f 
 
 k. ., 1 - ^1 ^ -'' 
 
 37 
 
 2 1 = (2.42) 
 
 1 + (l-X^ 
 
5Q 
 
 Then 
 
 ... /n f 
 
 1 + v'l - 6'^ 
 
 For the iteration process described by (2.28), if b = ^ and ^ = Gf + nf 
 then by theorem (2.4) 
 
 \\f -t\\ /|h| W^ 
 
 llfo-3?||<(^THl7J ^^i^ktTrF) . 
 
 Thus we wish to determine k- so that 
 
 (> ^(1 + k3 /l - X^) =^' . 
 
 An explicit splution for this equation does not seem possible. Table I 
 .(page 41) gives some sample values of k^ for \ near 1. From Table I, we 
 note that as £ decreases k^/k^ increases. We now show 
 Theorem (2.7): Let 
 
 ^2 
 
 ir = e(6',\) 
 
 then 
 
 for 
 
 lim 9(6 ',\) = 1 
 
 < X < 1 . 
 
39 
 
 ^^^22£- Nov ^^^j-^^ is the maximum of the Chebyshev polynomial, and by 
 theorem (2.3) ^^1 . k JTT^) is the maximum of the polynomial associated 
 with the constant b iteration scheme when b = 1) and ^^ = g/ + Hf. Among 
 all k degree polynomials which satisfy the condition P (l) = 1 the 
 Chebyshev polynomials have the property that their maximum is minimal on 
 the interval (-\,\). Then for k fixed, 
 
 < ^^(1 + /l -X^i 
 
 cosh k^ - f U + 1^1 - \ k) . 
 Since kg and k increase monotonically as £' decreases 
 
 and, hence. 
 
 Since 
 
 ^2<^3 
 
 < 9(6, \) < 1 . 
 
 ^ ^(1 + l^X^k ) = 6' 
 
 3' 
 
 /n 
 
 £' 
 
 1 +/l - \^k. 
 
 k- = ^ 3 ^ 
 
 1 ^ \ll - x'^ 
 
 and, hence. 
 
 
 ^n ^' 
 
 
 ^2 
 
 .rn ^_ 
 1 + 1^1 - 
 
 ■^■^ 
 
 ^3 
 
 ^n «' 
 
 
 1 + /l - \\ 
 
 Now 
 
 e 3(1 , k3 yrr^) < (.''3(1 ^ yrTF)''3 . ^^3 ^ 
 
 and since k^ and k^ increase monotonically as £- dec 
 
 jcreases 
 
k3<k^ 
 
 Then 
 
 Jn ^ 
 
 ^ > 1 + A 
 
 l^Z. 
 
 '3~ in 
 
 /n \ 
 
 1 . A 1 + 1^1 - ^'^ 
 
 . in 
 
 1 - 
 
 /n >C 
 
 /n 6' 
 
 40 
 
kl 
 
 TABLE I 
 
 \ = .95 
 6 
 
 k^/k^ 
 
 y^3 
 
 10-^ 
 U„9 
 
 9.3 
 
 11.9 
 
 ,21 
 
 .78 
 
 10"^ 
 
 13^.6 
 
 23.5 
 
 28.5 
 
 .17 
 
 .82 
 
 10-5 
 
 224.5 
 
 37»8 
 kk.o 
 
 .17 
 
 .86 
 
 \ = .99 
 
 kg/k. 
 
 ^2/^3 
 
 229.1 687.3 1,1115,5 
 
 .092 .078 ,075 
 
 21-1 53.5 85.9 
 
 '^^ .82 .86 
 
 27. 3 65,0 100.2 
 
 X. = .999 
 k. 
 
 ^2/\ 
 
 ^2/^3 
 
 230i.if 6,904.3 11,507.2 
 
 •029 ,025 .024 
 
 66.9 169.9 272.8 
 
 .77 ,82 .86 
 
 86.9 206.4 318.2 
 
k2 
 
 III A MODIFICATION OF THE AVERAGING METHODS 
 
 3«1 Introduction 
 
 In the previous chapter, ve considered several iteration pro^ 
 cesses for solving linear equations of the form 
 
 Where the c^'s vere determined hy some rule. In this chapter, ve shall con- 
 sider the above iteration process for matrices which are symmetric, positive 
 definite and possess property (A) (defined below). It will be shown that by 
 modifying (3.I), it is possible to substantially reduce the number of iter- 
 ations necessary to reduce the error vector to a given value. Furthemore, 
 if the calculation is performed on a high-speed computer, then with the 
 modification, the memory requirements will be half those needed for (3.1). 
 
 Definition (3.I): A matrix is said to possess property (a) if there exist 
 two disjoint subsets S and T of W, the first N integers, such that 
 SvT = W and if a. . ^ then either i = j or i.S and >T or i^T and jeS. 
 
 If a matrix has property (a), then it is possible to reorder the 
 equations so that S = ^1, 2, . . . N^^ and T = ^^N^^ 1, „.., nJ. m this 
 section, we shall assume that the equations have been reordered in this 
 fashion so that A appears as the hypermatrix 
 
 * 
 
 where 
 
 I?j_ is an N^ X N^ diagonal matrix, 
 D^ is an Ng - N^ diagonal matrix. 
 
and S is N X N, . 
 
 Such matrices are said to have ordering a- 
 
 43. 
 
 3o2 Proper values and proper vectors 
 
 We wish to investigate the proper values and proper vectors of 
 D-^A, Since D'^A is similar to ^^1^^-^)^-^!^ = ^-^l^^'^l^ ^ ^^^ ^^^^^^ 
 values of D'^A and jf^l^Klf^l^ are the same. Furthermore, if 
 
 vz, L> AD z = VD / z, and, hence, the proper vectors of 
 D-^A and D'^^^-Vs ^^^ related. The vectors which will be considered 
 will consist of two subvectors so that 
 
 where v 
 
 '1 
 
 and ^(2) 
 
 Theorem (3,l): If D"^/^^"^^ ^^^ 
 
 D-V2ad-V2/| 
 
 (1) 
 (27 
 
 property (A) and ordering <r and if 
 -(1) 
 
 (1 + n) 
 
 W 
 
 then 
 
 D-V2;^-V2 {-A 
 
 (1) 
 
 7(27i = ^1 - ^^) 
 
 for [1^0, 
 
 Proof: Now 
 
 d-V£ad-V£ . 
 
 D-V£sD-l/2 
 
 D, 
 
 D-VVd-1/2 
 
hk 
 
 Where I^ and I^ are N^ x N^ and N^ x N^ identity matrices, respectively. 
 
 Let d:^/Vd:^/2 ^ ^* ^^ ^^^^ ^ .^^^ 
 
 (1 + n) 
 
 T*|(2) ^ ^|(1) ^^ ^(1) ^ ^^(2) 
 
 Then 
 
 
 -d) . .M) 
 
 
 = (1 - n) 
 
 -1/2 -1/2 
 Thus D / AD / has (say) 2r proper values 1 + u 1 + n i . „ 
 
 and N- - 2r proper values equal to one. 
 
 The proper vectors of (D-^/^AD'^^^ - j) are the same as 
 (D-1/2aD-1/2), Since D'^^^AD-^/^ has proper values 1 + ^^, 1 + ^, , 
 -'A t^-^ i^^l^ 0) a.ndl repeated N - 2r times, (d'-^/^AD"^/^ - I) has 
 proper values i^^, -^^^, .-», i^^ and (n - 2r) proper values equal to 
 zero. Let us assume at least one element of T is non-zero. Then there 
 exist no non-zero linear combination of the row vectors of (o|T*) equal 
 to a row vector of (t|o) and, hence, 
 
 rank(D-l/2^-l/2 . j) ^ rank(o|T*) + rank (t|o) 
 
 = 2 rank T 
 since rank T = rank T*, If all the elements of T are zero, then. 
 
h3 
 
 rank (d' ' AD""*"' ^ - I) _ P ^.^.^y, t _ n ^« 4.^. a. * 
 
 ^ "^ ^^"^ ^ =^ so that r = ran]? T. Now T is 
 
 \ X N^ and T is N^ X N^; thus r < min(N^,N ), 
 
 Therefore T*J<2) ^ - ^^^ ^^^ _ ^^ ^^^^^^^^ independent solutions, 
 and T? p -3 has (n^ - r) linearly independent solutions. By the Gram- 
 SchmKit orthogonalization process the vectors t[^} ..., ^^2)^ ^^^ ^^ ^^^^ 
 orthogpnal to one another. Since the new vectors are linear'^combinations 
 Of the original set, they too satisfy the homogeneous equations T*!^^) ^ ^^ 
 
 Ifaturally, this same process can be applied to 3^^\ ..., ^d) . ^hua 
 
 N^-r 
 
 IM. -r 
 we assume in our future discussion, the vectors f 5^2) ^^2) ^ 
 
 / ^1 ' "^"^ % -r > 
 are orthogonal to one another, and likewise il)^^^ ^(l) ) ^ ^ 
 
 Then the spt. nf Tro/^+,^v,<- •'^' -^ 
 
 ^ • '^ 
 
 are If - Sr proper vectors of D'VSad-I/^ ^^ „^ ^^^^^^^ orthogonal. 
 
 Therefore, 
 
 i = 1,.=,, r 
 i = 1. ..., r 
 
 D-V2^-V2J^ = 5^ 
 
 i = 1, .,., Nj^-r 
 
 i = 1, ...; Ng-r 
 
 V 
 
 (3.2) 
 
 The proper vectors will also be considered orthononnal. 
 
 V 
 
kS 
 
 Theorem (3-2): For the set of vectors described Tby (3.2), 
 
 = . 
 
 anci lor every proper vector g 7^ f.^ S ^ e. 
 Proof; By theorem (3,l), 
 
 . [IF 
 
 S^nce e^ and f . are orthonormal vectors, 
 Therefore, [^i[^\ S^^^l = f^^', 4^^^ = 1/2 . 
 
 3f3 The modification 
 
 In chapter II, it was noted that for G = I ~ aD~^A the magnitude 
 
 (Jf the largest proper value of G is minimized when a = a = -^-~ wh^^^ = 
 
 a + b *""^-^« d. 
 
^7 
 
 is the smallest proper value of D"^A a^.d b is the largest proper value of 
 D A. By theorem (3.I), if 1 f ^ is the largest proper value of D'^A, 
 then 1 - ^i is also a proper value and hence must be the smallest prope:e 
 value of D" A. Then for 
 
 cc = 'a = 1, 
 the magnitude of the largest proper value of I - aD'^A is minimized irlth 
 respect to a. 
 Thus 
 
 (3.3) 
 
 and the largest proper value of G is ]!. Let us now consider the iteration 
 formula (3,1). Substituting (3.3) into (3.I), we have 
 
 
 = 2c 
 
 k+1 
 
 D 
 
 -1 
 
 1 / fk-l 
 
 ^2] t(2) 
 
 
 Expanding the hypermatrices, we have 
 
 .(1) 
 
 k+1 ^k+1 1 
 
 
 = 2c^^,D, r ' - S 
 
 - hill"^ ^ '^'^' 
 
 .fk-i)* K-i 
 
 If in addition, t = Gf + Ef 
 
 (3.i^.a) 
 (3A.b) 
 
 (1) _ T,-l/t,(l) c*>?(2) 
 
 J 
 
 h'^^hTr-'-s^ 
 
 (3. 5. a) 
 
k8 
 
 ?f ^ = ^;'( 
 
 i 
 
 <t(2) 
 
 - S: 
 
 :i) 
 
 (3.5.-b) 
 
 If the iteration process described by (3.I) is used on a high-speed computer, 
 
 then to compute: ^^_^^ it is necessary to have ^^ and ^ . Thus 2N memory 
 
 locations are needed for storage. For matrices under study in this chapter, 
 
 however, we are able to take advantage of the large block of zeros. 
 
 With ^Q and ^ % we are able to compute ^g by {3,k.a.), 
 
 Furthermore, as we compute each component of ^^ \ we may replace the 
 
 corresponding components of ^^^^ by it„ With ^^^ and ^^^^, we are able 
 
 to compute f^^^ by (S.i^-.b). And in general, ^ith^^^^^ ^^^/gk-l^ '^^ ^^^ 
 
 able to compute ^^^^ by (3.^. a); ^"^^ ?2k-l ^^^ fik^ ' ^^ ^^^ ^^^^ ^° 
 
 (2) 
 compute ^2k+l "^^ (3.^.b). If f^ = G^^ + Ef for the original process, with 
 
 ^Q we are able to compute p^'^ by (3.5.b) in the new process. Th\is after 
 
 k iterations we have computed the vector 
 
 ^^.^ at no time has more than N memory locations been necessary . We do not 
 have the vector ^^, and we are unable to compute it. Note that at each 
 stage the latest iterate is used. 
 
 Our iteration scheme now becomes 
 
 f; 
 
 2k 
 
 ^2k+l/ 
 
 %a' 
 
 
 
 
 
 "2k+l°2 
 
aaid 
 
 Let 
 
 r = <CV^^ - 3^(^)) . 
 
 OS / I 
 
 = li, f-i 1 = L, I %— I =D 
 
 10/ I S I y \ I D 
 
 and /^2k-'-l 
 
 
 
 
 
 °2k+1^2 
 
 — = C^f then L + D + U = A 
 
 and (3-6) becomes 
 
 ■ % - 2V-l(y' - U/^.^ . Ij; . Dj|_^) . f^_^. (3.,) 
 
 If Cj^ = bl^ b a scalar, then (3.?) describes the successive over- 
 relaxation method of D. Young [9]. If^ however, 
 
 ■^ ^ ° I ^2k*1^2 
 
 where 
 
 and 
 
 2 - ^t^ 
 
 b^ = l 
 
 A'^ =B-^(y(2).s;(^) , 
 
 then 
 
 /2k+l 
 is the k"" iterate of the modified Chebyshev iteration scheme. 
 
 k9 
 
where 
 
 and 
 
 50 
 Equation (3.?)^ then may he rewritten as 
 
 Vl = ^° -^ ^V^'^ f^ - 2C^(U + D)] 
 
 ^^'^ ?k = Jk-1 ^Pli^^ jL = ^ i^ ^^* iHk.il ?^ 0. We shall now show that 
 this condition is satisfied for successive overrelaxation when b > and 
 for the modified Chebyshev method. Since A is positive definite, the 
 elements of D are positive. Then since 2Cj^L is a lower triangular matrix, 
 det JD + 2Cj^l| = det|D| > 0. For successive overrelaxation with b > 
 det|bl| > and, hence ^^^ > 0. For the modified Chebyshev since 
 \>^ > 1/2 by theorem (2.1), 
 
 In addition. 
 
 det f ^-^4- — 1 > 
 
 2k+l 2 
 
 ^k-1 ^ \-l^ = I 
 
 so that ] |J^_^ -J^\ I = if and only if J^ = 5. Consequently, we may 
 terminate the successive overrelaxation method and the modified Chebyshev 
 iteration scheme when | |J^ - J^_^| | is less than some preassigned number 
 and be assured that the error vector is related to this number. 
 
 3.^ Bounds for the error vector 
 
 In chapter II, we were able to describe an upper bound for the 
 length of the error vector 1 - ^^. We desire now to describe an upper 
 bound for the length of the vector x - ? . 
 
51 
 
 By (2.9). (^^ - $) . P^(a)(|^ - ^) = p^(j . :)-i^)(^^ . .). 
 
 We now expand (/^ - 1) ±n the proper vectors of I - d'^A which are 
 
 given in section (3.2). This is possible since the elementary 
 
 , , -1 
 
 divisors of I - D A are simple. Thus 
 
 N, -r N-r 
 
 
 Now 
 
 (;, - t) = P^d - ^-hKf^ .-) . D-V2 fta,P^(,^)e; . i^ 
 
 a. P (-U. )f 
 i+r k^ ^i'^ 1 
 
 -^ P^(0) 
 
 N, -r N -r ~l 
 
 Zj a^ . P . + Z a 1? 
 .^^ 2r+i^i .^^ ^N^fr+i\ 
 
 so that from theorem (3.1) 
 
 It will be shown that the polynomials we wish to consider have the property 
 that 
 
 P^(-..) = (-l)'^Pj^(^) (3.8) 
 
 and, hence, 
 
 P, (0) =0 for k odd. . , 
 
 Then by (3.8) and (3-9), we have 
 
 N. -r 
 
 -..oPoJ0)l?l) 
 
 ;^^^^^^=^^/^^|(^i-^i.^^ 
 
 j_ i+2r 2k' ^-^i 
 
52 
 
 Let 
 ;(i^' - S<^) . 4^) ana ^<^i, - J(^) = 4^) , 
 then by theorem (3.2) 
 
 (4^'' V?) - \ i (a, - Vr)^ 4(^.) ^ '^r 4.r4(0) 
 
 r N^-r 
 
 E (a. - a )^ P^ fLL ) + y 
 i=l ^ i+r^ 2k^^i^ -^ .^^ -i+2r'2k- 
 
 and 
 
 l2 ^2 
 
 (,^k ^ °2\y =-^ Z (a. - a )^ P^ (n ) 
 Hence ^ 
 
 42r4(°) 
 
 From this we have 
 
 (3-10) 
 Nov 
 
 ^i=l ^ 1+^ 1 1^1 i+2r^i 
 
 ^0 '= ;6 '- ^ 
 
 so that by theorem (3.2) 
 
 ^ Vo J = 2-,>.(^ ~ Vr^ + ^, ^+2r 
 
 Th-.JG (3.10) becomes , 
 
 0,|i^..M^ 
 
53 
 
 or 
 
 \\:>'''%\f< ns. /4^;;774j7iiDV|Wir.' (3.11) 
 
 Using relation (A.IO), we have now 
 
 Consequently, we have 
 
 Theorem (3.3) ^ If T is computed by (3.7) and if 
 
 (1) G = I - D'^A , 
 
 (2) Pj^(-fi) = (-l)\(n) 
 
 then 
 
 l^.il ,/lVu^'/' 
 
 lljrr^-I^Ti^^ 
 
 Max ./p^, (u.) + p2 (n ) 
 
 1» O f-*, 
 r 
 
 Now the unmodified methods converge if and only if Max |P (w )!-♦ o 
 
 Tn "j 
 as m-*<=o , Then since ,°'%'**^r 
 
 Max /P^^(,^).P^^^^(,^)< Max iP2,(.,)|. Max \P^^^^i^^\ 
 
 we have 
 
 Corollary (3-31): Under the hypothesis of theorem (3.3); if 
 
 Max |P (ia.)| -» as 
 ' mi ' 
 
 m — > e«3 
 
 then 
 
 0,n^..^i^ 
 
 ,->(l),, -^ as k 
 
 :^o 
 
■ if in (3.1)/ Cj^^j = 1/2 then 
 
 /lc+1 "^ %-'- ^^ • 
 
 Thus P, (G) = G , and. hence, condition (-2) iq qnt-' q-p-,- ^^ mv,^^ 
 
 K ' f ) ^^iiu.j. uxuii v^; IS sax.vfariedo This process corresponds 
 
 to the Gauss-Seidel method. Then 
 and, hence. 
 
 0^ /4(-) - 4.i(^) - i^' ^^ . 
 
 Corollary (3.32): 
 
 
 IfPj^(G) 
 
 = G^, then 
 
 
 ?J 
 
 For the Chehyshev iterative process 
 
 p (n) = COS__k©_ 
 
 V^^ cosh k«^ ^ 
 
 Where © = cos"^ -H_ and -^ = cosh'^c^. This again satisfies condition (2). 
 Purthemore, the maximum occurs for P^^i^^) and l?^^^^i^^) when ^^ = ^. Thus 
 we have 
 
 Corollary (3.33): If 
 
 k^^'' cosh k«*> ' 
 then 
 
 11^^" < f h!u ) '^' /ZTiTziiz: 
 
 ^0 " V'^U/ / cosh^ 2k*^ cosh^(2k+l)60 
 
55 
 
 If we modify (3.I), vhen c^^^ =^ and with f^ ^ G^^ + H^ then we have 
 a specialization of successive over relaxation, For this process, 
 
 P (u) = P^ (-^ - 1^ sin(k - 1)Q cos k Q 
 
 cos 9 = 
 
 and 
 
 ^ i./TT^ • 
 
 This again satisfies the relation that P^(-^,) = (-l)^^(^) . Furthermore, the 
 maximum of P^^{^) and Pgk+l^^^^ °==^^ ^^^^ M- = H- Thus ve have,, 
 Corollary (3^3^): If the successive overrelaxation method is used with 
 
 then 
 
 Let us also consider the processes which have been termed process 1 end 
 process 2 in section (2.9). The polynomial associated with process 1 is 
 
 O^) 
 
 q sin(q + 1)9 cos M9_ _ q+1 sin q9 cos(M - l)9 
 ' sin 9 cosh M^ " ^ sin 9 cosh(M - l)^ 
 
 where cos 9 = 
 
 -^ . By theorem (2o5) 
 
 Max |Qi-^^(u)| = / ^ 
 
 0<|j.<il ^+^ ' ( cosh Mij cosh 
 
 U- l)a,)^ -^ 
 
 cosh M«; 
 
 ill^^y 
 
56 
 
 Again <<-.) = (-l)"^^4l)(.) 
 
 Corollary (3^35): If 
 
 and, hence,, 
 
 then 
 
 P (it) - ^^ sln(q + 1) q cos M9 
 
 'M+q^ 
 
 f 
 
 sin 9 
 
 cosh Muf 
 
 q+1 sin qQ cos(M - 1)Q 
 ^ sin 9 cosh(M - l^j ' 
 
 
 
 D, 
 
 < 
 
 I'u 
 
 1/2 
 
 D 
 
 u 
 
 4i^(^) 
 
 1 2 
 
 4kii(^) 
 
 Associated with the process 2 is the polynomial 
 
 0(2) / \ ^ /cos M9_ \^ cos (^9 
 ^+q^' I cosh Ui^ J cosh qc^ 
 
 where cos 9 = -^ . By theorem (2.6) 
 
 ^^ '€.(^)l = 
 
 (cosh M^)-^ 
 
 (2) 
 
 cosh q_u> T4p+q^^' 
 
 Also 
 
 so that 
 Corollary (3.36) 
 
 then 
 
 If 
 
 4p],(?) 
 
 
 Mp+q^^^ 
 
 cos M9 
 
 cos q9 
 
 cosh Uu)j cosh q6^ 
 
 \ll JI^Lu 
 
 1/2 
 
 ll^ 
 
 u 
 
 [ 
 
 il\^) 
 
 [illr.)] 
 
 3.^ Comparison between various iteration schemes 
 
 3.51 Comparison of various modified methods 
 
 In chapter II, we saw that after k(> 2) iterations, for \\^ - t\\ fixed, 
 the maximum error of the Chebyshev iteration scheme was less than the maximum error 
 
57 
 
 of the constant parameter iteration scheme when b = "& and ;^ = G/ + H^. The 
 question arises whether after k iterations for [ |^^^^ - x^^^\\ fixed, the maxi- 
 mum error of the modified Chebyshev method is less than the maximum error of 
 the .successive overrelaxation method when b = 'b and |^^^ = D""^/y^^^ - S?^^-^). 
 Theory (3.4): For | H^^^ | | and k fixed, the maximum error for the modified 
 Chebyshev iteration scheme is less than the maximum error for the successive 
 overrelaxation method when ^[^"^ = 'D'J' f }^^^ - si^^A and b =^. 
 
 ^^2^' ^^"^ cosh mcj ^^ ^^-^ maximum of the Chebyshev polynomial and 
 p (1 + m 4 - jl ) is the maximum of the polynomial associated with the one para- 
 meter iteration scheme when b = ^ and f ^ = G^^ + By'. Among all k*^ degree poly- 
 nomials which satisfy the condition F^{l) = 1, the Chebyshev polynomials have 
 the property that their maximum is minimal on the interval (-jT, jl) , Furthermore, 
 the Chebyshev polynomials are the unique set of polynomials which have this 
 property. Thus 
 
 ^^ < f^\l + 2k /iT^) . (k > 0) 
 
 cosh 
 
 and — , 
 cosh 
 
 T2k\ 1)^ < e^^^^^l + (2k + 1) /l - ^) (k > 0) 
 
 and hence. 
 
 1 , 1 
 
 2 '^ p < 
 
 cosh 2kw cosh (2k + 1)(aJ 
 
 l[^^\l . 2k ^^\[f'^-'\l. (2k.l) ^ 
 
 Then from corollaries (3.33) and (3.3k), we see the maximum error for the modified 
 
 Chebyshev iteration scheme is less than the successive overrelaxation method when 
 h=S, and |(2).D-1(?(2). 3^(1)) _ 
 
 We now wish to consider the number of iterations necessary so that 
 
58 
 
 lljl-2|| 
 
 ■^*(i) Tuxr ~ ^ 
 
 If we use the Gauss -Seidel method^ then by corollary (3.32) 
 
 
 We wish to determine k| so that 
 
 I'u — 1 L —2 
 id] — I ^ VI + la^ = <^ . 
 
 Then /|dL \/^ 
 
 
 Vu/ TTTW 
 
 2 /n n 
 Next;, we wish to deterxuine the number of iterrLtlons recessar - so tho/ 
 
 
 
 IJ. -X 
 
 for the modified Chebyshev method. By corollary (3.32), 
 
 1 1^(1) -?(i),, - Idt; / J 2 — "^ — 2, T" 
 
 I IJo -^11 \ ' '/ / V cosh^ 2ka; cosh^(2.k + 1)^ 
 We wish to determine k' so that 
 
 ' I'u 1 
 
 D 
 
 1 1, 
 
 , / 2 ? 
 
 -^ / \' cosh 2k^6^ cosh (2k' + l)uj 
 
59 
 
 Since 
 
 K nh^\' j\\K 
 
 ^'^ /IDJ \V2 
 
 =oBh(2k..i).l^ T51J7 ^ [TBip / / ;;;;t 
 
 cosh'- 2k^w cosh^(2k;+l)^ 
 
 cosh 2k' ct; I l°|/ 
 
 we my solve the right-hand side and left-hand side of the above Inequality to 
 get upper and lower bounds for k'. The calculations are the san,e as for deriving 
 
 (2.42). 
 
 let 
 
 , . >]^f" . 
 
 t I 
 
 D. 
 
 i'u/ /i" 
 
 then 
 
 where 
 
 (3.12) 
 
 -2 
 - 1^ 
 
 ^'-TTt 
 
 From the above equation, we get an estimate of k' which Is in error by at most 1/2. 
 
 We shall later compare k with k ' , 
 
 For the successive overrelaxation method with b = "B and 
 ^(2)_ -1 A(2) ^^{1)\ 
 (1 " 2 (^ ~ fo ) > "^^ ^^^® V corollary (3,34) 
 
60 
 
 Thus ve wish to detenjiine k' so that 
 
 |B I V^^ 2k' / -^ — ■ 
 
 ^ ^ (3.13) 
 
 An explicit solution of this equ.ation does not seem possible. However, we have 
 the following. 
 
 Theorem (3.5): Let 
 
 ^2 
 ^3 
 
 Then lim 9 '(6,|i) = 1 
 
 for < ^' < 1. 
 
 P3-oof : By theorem (3-^); for | |Iq ^| | and k fixed, the maximum error of the 
 modified Chebyshev iteration scheme is less than the maximum error for the 
 successive overrelaxation method when f\ = D"^ (y -S^^^^) and b ="%, 
 Then since k^ and k' increase monotonically as £ decreases 
 
 k,' < k 
 
 - ^^3 ' 
 
 and, hence, 
 
 9 < 9'(6,^) < 1 
 
 From (3,13), we see 
 
 /n 
 
 S— ) ' ^.2k.yi-^f .(.2^l.(2k..l)/rT=^ 
 
 An c 
 
61 
 
 and, hence, 'pf (3.12) 
 
 2 
 
 ^3 
 
 /n I £. 
 
 1 1 
 
 
 ^/n 
 
 f 
 
 /n 
 
 I'u 
 
 W 
 
 y 
 
 ("i + 2k' JTT^f ■" f ^ f^ -^ (2k' + % /TTpJ 
 
 Nov 
 
 SJc 
 
 f 3 /^i , 2k' K^y . p Y^ . (2k^.i) iT^Y 
 
 . ^^3 iY r-~^VS T, / 5A^ / ^^3 -^^3 
 
 < f y (i + /i - ;r2 j 3 -. p^^i + yi - 1) ^ =/ ii -^ ^ 
 
 where 
 
 f ' 
 
 1 + yi - U 
 
 and since kj^ and k' increase monotonically as £ decreases 
 
 kJj < kj_ . 
 
 Then 
 
 ^^-kT^ 
 
 /n 
 
 1 + 
 
 /TTTT^ g 
 
 1,2 
 
 f 
 
 3 A 
 
 /-|D|- \l/'2 
 
 fe;4' '/f' ^ ^''i ^' ^ P'6 ^ '^-^i^^) ^ 
 
 
 /n^S^ 
 
 'm 
 
 
 1 - 
 
 m 
 
 
62 
 
 Since 
 
 'n 
 
 jDj 
 
 ID" 
 
 1/2 
 
 l'u> 
 
 ki = 
 
 /T 
 
 + [1 
 
 2/n ji 
 
 it is not diffic-ult to verify that 
 
 lim 
 ^0 
 
 / 
 
 = 
 
 n <f 
 
 and hence 
 
 lim "^2 
 
 £-^►0 k^ - ^ - 
 
 Theoren (3.6): For k and | ||^^^ | | fixed, the maximum error for the modified 
 process 1 is less than or equal to the maximum error of the modified process 2 
 described in section (2.9). 
 
 Proof: By corollary (3-3^), the maximum error for the modified process 1 is 
 after k iterations when | |^i | | is fixed 
 
 D 
 
 I'u 
 
 1/2 
 
 m 
 
 A 
 
 i^\^) \' . 
 
 ^^IL (^)' 
 
 and the maximum error for the modified process 2 is after k iterations when 
 
 1^(1)11 . ^ 
 \6q 'II is fixed 
 
 (ptj '/S^ 
 
 ill (^)' 
 
 However, by theorem (2.5) 
 
 -ilh-^}y<l^il\-.)J and [^JJlj 
 
 <v 
 
 Qi!:', G)J 
 
 ^2k+l 
 
63 
 
 Hence 
 
 'E^^^n^isw 
 
 < 
 
 QAr/ (n)J' + 
 
 (2) ,^^ |2 
 2k 
 
 
 3« 52 Comparison between the modified and unmodified methods 
 
 From equation (2A2) for the unmodified Chehyshev iteration schei 
 
 me 
 
 ■^2 = 
 
 Jxy p 
 
 where 
 
 ^ ' TdT 
 
 ' 'u 
 
 1 1/2 
 
 For the modified Chebyshev iteration scheme, we have by (3.12) 
 
 A 
 
 1 + \/l - £"^ 1 
 
 A 
 
 Jn -2 
 
 2 <k'< 
 
 1 + \/l - £"2 
 
 /n^2 
 
 where 
 
 D' 
 
 / 
 
 1 1/2 
 
 Then 
 
 and 
 
 (i^i'uy /? 
 
 £' < 6" if 2|dJ < IdI 
 - '- ' I'u - ' 'u 
 
 £' > 6" if 2|dJ > JDl 
 
 'I'u ' 'u 
 
 Hence if 2|D^|^< |d|^ and ^' < 1 , 
 
 kj 
 
 -2_ < _1. 
 k^ 2 
 
We will now show 
 
 6k 
 
 lim 
 6-*0 
 
 •=2 
 
 (3.11*) 
 
 Now 
 
 _1_ 
 2 
 
 L^ 
 
 Jn 
 
 ^n 
 
 1 + \ll - 6 -'.'^ 
 6' 
 
 1 + /l - ^'^ 
 
 >! 
 
 r 
 
 /n 
 
 1 + /I 
 
 ^i^ 
 
 2 ^ _^ 1 + /i - e""^ 
 2 xfn 
 
 ^f: 
 
 1 + /I 
 
 a = 
 
 
 b = 
 
 u 
 
 D 
 
 u 
 
 Then 
 
 A 
 
 ae 
 
 lim 
 
 and thus 
 
 i + /rT^2 2 
 
 a ^ 
 
 1 + /l - h'k 
 
 lim 
 
 1 + 
 
 /n 
 
 a 
 
 ^n e 
 
 /n b 
 
 1 + Tp — 
 
 Yn 1 -Hl/T 
 
 2 2 
 ~ a e 
 
 n £ 
 
 jni^ /r~- 
 
 ^'e' 
 
 Jn 
 
 - 1 
 
 ^2 
 
 _1_ 
 2 
 
 6^-? 
 
 We now wish to compare the one parameter iteration scheme when 
 ^1 " % "^ ^^ ^^^ b = b with the successive overrelaxation method when 
 
 f[^^ : .;X^/^^ - s;(^)J 
 
 We will now show 
 
 lim 3_ 
 
 _1_ 
 2 
 
 Since 
 
 ■i 
 
 ■^2 
 
65 
 
 K i.-m K ^.^ K .,._ ^2 
 
 lim ^3 ^ llm 3 lim _f2_ lim 
 6-^0 k^ ^-^0 k^ * «f-^0 k^ " ' 6-^0 ^3 
 
 and by theorems (2.7) and (3-5), and (3.1U) 
 
 ^-^0 ^3 2 2 • 
 
 3.6 Bo-unds for the error vector for successive overrelaxation 
 
 In section (3.^), we developed bounds for the successive overrelaxation 
 
 error vector under the assumption p[^^ = D'Yy^^^ -^fo^^) ' ^^ ^^is section, 
 
 we shall arrive at a bound independently of this assumption. 
 
 As in section (3A), let us expand (^^ - t} and (^^ - t) in tenns of 
 
 the proper vectors of G = I - D~^A. Then 
 
 ^° - '= °"'' [k v^"^ ' A v.A ^ 1\,.3A ^ X %.-.-^^) 
 
 and 
 
 Now by (2.30), 
 
 ,-l/::> 
 
 N^-r Ng-r 
 
 ■'^^- /i=l ^^ i=l l^^^^i i^l 1. i+2rPi -^ ^^^ ^i+N^+r^iJ- 
 
 ^ - ^ = QS^Q- (f^ - t) - (2^ - 1) QS^_,Q-1(^Q - 5) 
 
 so that 
 
 
 N^-r 
 
66 
 
 and 
 
 Ng-r 
 
 where 
 
 Then 
 
 ^ T sin m9. 
 m-1 
 
 m^ i' r sin 9. ii w^ jt u. 
 
 m-1 
 = mp if 9^ = 
 
 / , \m-l m-l 
 = (-1) mp if 9. = TT 
 
 and = /2^ - 1 = £= 
 
 1 + /i - ;i^ 
 
 and hence S^(0) = for m even. 
 
 Then in view of the above, we have 
 
 N -r 
 
 -(^^-^)^%i.2Al^l(0)pi- 
 
67 
 
 and 
 
 :>(2) -»'(g) 1/2 )'^ 
 
 +1 a, . „ s (0)0^^^ 
 
 llBf 4^'||^ = -^ Jj(a^ . ^l.r'S^.f'^i) - (^ - 1)(V - a,^,^^)S^^,^(,^)] 
 
 2 
 
 N -r 
 k2 
 
 H- (^ - ir E a2 ^ S2 (0) 
 
 and 
 
 ,.1/2^(2)1 |2 1 ^. (., 
 
 Ng-r 
 
 Now 
 
 and 
 
 s6 that by theorem (3.2) "^ 
 
 K[ -r 
 
 and 
 
 N -r 
 Mb^/2^(2),.i2 1 V , x2 I 2 
 
 '20 ^ 1=1 ^ l,x+3r' .^^ 0,i+N^+r 
 
68 
 
 Then 
 
 ||DV2/j|2n|Dp4^^||^.||DV2,f)|i2 
 
 + (2b - 1)^ Z af S^ (0) + Z a^ q^ Cn^ 
 i=l 0,i+2r''2k-l^"^ + ^t*j_ ^l,l+N^+Ak+l^°^ 
 
 and, hence by (A.l) 
 
 H-(2S-1) Max /s^^^,(^) + S2^(^x) |iDV2;(l)|| , 
 
 0<ii<n 
 The maximum of S (|j,) Is attained at (j. = jlj, and hence 
 
 m 
 
 1^.11 i/4(?)-Li(i^)H^rikrii 
 
 y 
 
 and thus by the Cauchy-Schwarz inequality 
 
 i /(4(S) - 4.i(^))({^ - (2^ - i)X4-i(^' - 4(^) 
 
 • /T^Fii^^Mit^^ 
 
 \ 
 
 '^i( 
 
 
 
^ 
 
 "^eorem (3-7): For the successive overrelaxatlGn method with b = 'J; 
 
 l?JI /. _ „ .In 
 
 where •' 
 
 Then hy comparing the results of theorem (3.6) with corollary (3.34) 
 
 that a cancellation of terms takes place when 4^^ = D'Vy^^-^ - S$^-^^') in 
 
 the same fashion as for the one parameter iteration scheme when ^ = ^t. + Hy 
 
 we see 
 
 and b = b. 
 
 3.7 Bounds for the difference vector 
 
 As was pointed out in section (3.3). a possible termination procedure 
 for the successive overrelaxation method and modified Chebyshev iteration scheme 
 is to stop iterating when | \J^ - j?^_J | is less than a pre-assigned number. We 
 shall now develop for i \f^ - f^J\ bounds which will be used subsequently. 
 
 Let us assume as we did for theorem (3-3) that G = I - D"\ and that 
 \(-i^ = (-1) Pl5.(^^)• Then we have from the proof of theorem (3.3) that 
 
 / 1=1 -L_3_ "^^ 
 
 t(2)_^(2) _^(2) ^(2) T^-l/2y?, 
 
 . )P (n )t^^^ 
 i+r^ 2k+l^^i^ 1 
 
 Hence, We have 
 
 f' -P ^P -p.. - Bl^/^f J^(^ - ^„)tW^l)-...3(^l))f ' 
 
 W^-r 
 
 * ,^, ^.2r[f2k<°) - P2k-2(°»P^^'* 
 
70 
 
 p{2)_ M2) ^ ^{2) _^(2) _ ^-1/2)' 
 
 Then by theorem (3.2), 
 
 N -r 
 
 + (P2k.l(^l) - ^Slc-l^^l?)" 
 
 ^ ,^, ^-Srt^Sk^O) - ^2^-2(0)]^ 
 
 (3.15) 
 
 From this, we ijave 
 
 ■''^'(JK-Zk-l^llS 
 
 Max 
 
 (^2k(^l) - ^2k-2(^lf ^ (P2K.l(^..) - ^2I..l^^i)) 
 
 ; 
 
 2 
 
 r 
 
 Z (a, 
 1=1 ' 
 
 a. )' 
 
 N^-r 
 
 Z 
 
 1=1 
 
 2 
 a 
 l+2r 
 
 Max 
 
 
 1 "^0 
 
 Then using (A.IO), we have 
 
 !ljk - jl-lll ^ .'^'^ /tP2k(^l' - P2k-2(^)J' - [P2k.l(^l) - ^2,-1(^1)1' 
 
 
 (3.16) 
 
 For the modified Chebyshev method, 
 
 cos mO 
 
 m cosh m co> 
 
 , 9 = cos 
 
 
 '^= cosh 
 
 -1 1 
 
and hence (3.l6) becomes 
 
 '.a 
 
 I Ifk -/k-il I </ jwifj Jhdr^ ^ 
 
 cosh(2k-2)^y ""f cosh(2k+l)w ^ cosh(2k-lV) 
 
 A 111^^^ 
 
 (3.17) 
 
 For the successive overrelaxation method with ^^^^ = D'V^^^^ - s|^^^) and b ='S 
 
 we have 
 
 V^> =^ 
 
 m 
 
 I^IL. . 1 ) sin(m-l)Q cos mQ 
 
 9 = cos' -tL . h - 
 
 
 1 + A - ^ 
 
 = /^&TT 
 
 so that 
 
 .^ . _\l/2 
 
 C2[p2^1 . (2k^i) /TT^j , ^, , (2^.,) ,^T^2)Jj I 1^(1) I 
 
 (3.18) 
 
 Let the proper vector associated vith u, the largest proper value of G, 
 be D-^/^i. Pr„„ (3.15) „e see that if ,<^K ^^/^Z^^) , a a scalar, then 
 
 for the modified Chebyshev method and successive overrelaxation method when 
 
 (3.19) 
 
72 
 
 We wish to consider the successive overrelaxation method without the 
 
 assumption that ?$^^ = D'-^ft^^^ - s%^^^] t?o>. +>,<. ^ 4. . .^ 
 
 /^l 2 V -^ % / • -^^^ ^^® °n® parametric iteration 
 
 scheme, vre have 
 
 •k-i 1^ > 1 
 
 and, hence, 
 
 Then since the difference equation for ^^ - J^_^ is identical as the difference 
 equation for ^^ - 1 given by (2.29), the development of the bound for | |J^ ^ f \\ 
 is identical to the development of the bound for I |/j^ - ^| | . Hence as in section 
 (3.6), we have 
 
 Mfl^ ^iC^ffl • 4.A|l§i' • <* - .>Y4. 
 
 
 2k'^y |D 
 
 where 
 
 m 
 
 
 
IV THE MBERICAL "BEmVItK t^ "THE ^EHERATED SEQUENCES 
 
 4ol ^e numarlcal process 
 
 m the previous chapters, ve have discussed the behavior of the 
 se^ence of approxin^tions to the solution of a system of linear equations 
 under assumptions that the calculations ^re performed exactly and that there 
 vas no error in. the given data. In any calculation, however, the numerical 
 operations are .replaced by pseudo-operationsc Furthermore, it may be 
 neceasaiy to replace A, 5^, G and H by their digital approximations. In this 
 chapter, we shall consider the effect of the above inaccuracies. 
 
 The basic iteration process described previously is 
 
 ^+1 = 2c^+l^^^ + ^y - ^k_i) ^-/k-l (k > 1) (^.l.a) 
 
 with 
 
 nd^ givene We wish to consider the case when t^_^^ = b^_^_^, the coefficients 
 associated with the Chebyshev iteration scheme, and c^ = b for all k. 
 
 In order to proceed in our analysis, we shall make some assumptions 
 about the coo^utationo 
 
 lo The computation is c^fried out on a digital computer in fixed 
 point c 
 
 2. All digital numbers, S, lie in the range - 1 < x < 1 and are 
 represented by a ^- place, P - base digital aggregate. 
 
3^ H, the digital T^^pr.^Maxr.on of H, is positlVB definite and 
 
 symnetric <. 
 
 ^. The proper va.i.ues of G, the digital representation of S, are 
 let.b than one in magnitude. 
 
 E-^en though all the elements of p^..may be less than one in 
 HHgnitude, there is no assurance tnat all the elements of the generated 
 vectors uill remin less t^an one in magnitude. Frequently, vhen a problem 
 is solY^. -it is possible to scale it .before it is placed on the computer. 
 We sb^l aot assume that this b^s been done here. One can either scale each 
 toment'of the generated vector individually by some power of p or scale the 
 entire vector by a power of p . If the elements are scaled individually, then 
 for the nmnerical process {k.l), it is necessary to store 2N exponents. 
 They may be stored separately or stored with the numbers themselves. If they 
 are stored with the numbers, then some of the digits of the numbers are lost. 
 If they are stored separately, then the storage required is increased. We 
 •shall assume, then, that all the elements of the vector are scaled by some 
 power of ^c 
 
 The initial vector, ^^^ is scaled by (say) p^ so that within the 
 
 machine ve have f^^^ f /^ (i = i, 2, ,.., .w) ^ere f 3^^ indicates a right 
 
 shi-ft,^f p^ pilaces. If during the calculation, a number exceeds the capacity 
 
 ■:of a xe^stfir or is about to do so (such as for a division), then the 
 
 exponent ^p lisjcreasedo Thus there ±s asstsciated with each machine generated 
 
 _ p., 
 •vector, ^^^ 1 T P "" (i =1, 2, ..o, n)^ a power of p, p^, and, hence, 
 
 % - % ^ •" ° - Pjc' ^^ ^^^ ^j> ■theaa either the computation should be 
 stopped since tbe range of the answers ^exceeds the capacity of one register^ 
 or the iterates must be stor^ in more than one register o Since the p '- 
 
form an Increasing sequence and are boimded by c, we shall assiane that for 
 k > K 
 
 75 
 
 % = P K+l 
 
 The actual computed vector will be denoted 
 
 P, 
 
 as ^, — 3 , and thus 
 
 (4.1) becomes during the computation 
 
 ?k^ 
 
 ?] 
 
 k+1 
 
 . „^k+l - 
 
 ■^ ^ = 2=k.l"l 
 
 ks . .^k+l-Pk-. ,^ ^. . .Pk+1 
 
 Hf^^'k T P ^) f P "^" "] + (H^^y) r P 
 
 (? ^ B^k-1. ^ Pk+1-Pk-]J 
 7k-l • P ^ " P 
 
 + (f ^ A-l^ • A+l'^k-l 
 
 (^k-l^^^'^)TP 
 (k>l) 
 
 With 
 
 0^ . A-Pq 
 
 7i 7 P = G^^f^^G ^ P ) f P ] + (H^3y) f P 
 
 (if. 2. a) 
 
 (if.2.b) 
 
 where ic^^, ^, ^ represent pseudo operations which may be dependent on G and H 
 
 . A+l"Pk-i , . , . 
 
 U = 0,1; indicates a right shift of p - p . (i = 0, l) 
 
 places. Eqmtions (4.2) may be rewritten as 
 
 and f p 
 
 k+1 
 
 _. .z? 
 
 ^k+l-Pk- 
 
 A+1 7 P ^^'^ = 2c^^^ JG[(^^ f P -) ^ P-- -] , (Hy-) f p^k+1 
 
 Pv 1 Pi,,n-P 
 
 (^k.i r P ) - 3 
 
 k-ls . -^k+l'-^k-l 7 ^ P 
 
 Pn. , n -Pi 
 
 k-lv - ^^k+1 ^k-1 
 
 ^l. 
 
 (k>l) (4. 3. a) 
 
 .:? . Pi _ J* Pn Pi-Pn r» Pt ::> 
 
 7i 7 ^ - <5E(^^ 7 P ) T P ^ °] + (H?) r p 1 + /q (4.3.b) 
 
 where 
 
76 
 
 1 = 2c, 
 
 ^\.i-iY-2^^k r ^'') ^ ^'""'■'"] . (H.3?) r ^''"' 
 
 _ ^^ ^ A ---^ ^ «^k+l"Pk-l 
 
 k-1- 
 
 (7k-i-p )"p 
 
 -2Vi[^f(;k ^ ^'') r P'^^^"'^]. (H^)- p'^-^ - i$^_^ , 
 
 - ^^^-^) 
 
 Pk+i-Pk-ii 
 
 (k>l) 
 
 and 
 
 We shall again rewrite (4.3) as 
 
 p'^ - g4 , p^ . g'l-^ . (h|, . 3^1. 
 
 fk.l = 2=k+l[% * ^ - ?k-l j * /k-l ^ A 
 
 ?i = 7o + =y + ?o 
 
 (4.1t.b) 
 
 where 
 
 r^.. 
 
 k+1 
 
 / 
 
 -P 
 
 k+1 ? 
 
 (fk.iP '^^ -? 
 
 k+1 
 
 i A*^) 
 
 ± Pjj, p,,_, , -p, 
 
 ^w.^WK ^ p ") - p"''^' '' - f, p""'^^'] 
 
 + m)T^^*^ -tfi"^*^ 
 
 * [(1 , - p'-^-i) ^ e'^^-i'^'^-i - }^_^ p-'^'^-i] 
 
 f.-i r ^''^-^) - ^ 
 
 ^ C-x r 3 '^-) ^ /-^"- - f,., e"'^-- 
 
 (k>l) 
 
 and 
 
 P 
 
 «^ + f^- 
 
 ■ (fi3 -?i 
 
 Pi -Pn ^ -Pn 
 
 ^^^).5[(?o^^^")^^"^ °-/o ^^'] 
 
 ,-5, . Pi ..:* -Pn 
 
 + (Hy) f P "- - By ^ ^ 
 
I/e vn.sli to determine 
 
 Now by (A.l), (A.3.S.), (A.ll.a) 
 
 an ^pper iound for j ]/^| | (i > i) ^„^ j |^ 1 1 
 
 k+1, 
 
 f 9 ''"■■'1 
 
 -^l=k.i#IJlf,TP^fg'^-l-"'^.^^ 
 
 k+li 
 
 ^k+1 .- -Pk+i, 
 
 + II(By) r 3 "-^^ - Hy p^+l| 
 
 77 
 
 Pv^T -Pn 
 
 ^ii(?.-i^3''-^)- ^"^-ii p'^^^^ii 
 
 - ft^'^-l^ • =Vl"^k-l i -P 
 
 ^11(^-1 7 P'^-^)-g 
 
 k+1 1 
 
 where | |/ | | < | |/| | 
 
 (k >1) 
 
 Now if 
 
 W,ir& 
 
 k+1 
 
 ,^k,lp"'^"'l<f^3'''^"^ 
 
 "k+1 r? -P 
 
 I l?k.i ^ P '' - fn+i P '''I I < /i i :^ p'>^-i 
 
 Aiso^ 
 
 Pv^i -P 
 
 I(fk,irP')r3''^^''=-;,^^ p"'''-i| 
 
 
 <f ^p'^k+i-Pk -K+i-Vi' 
 
 ^fe 
 
Hence for |c7| < 1, 
 
 p 
 
 78 
 
 I K\ I <3 ^'-^ h l/l I . /ief p'^-1 . 2[iG|^/N (^6 - 3~Vl-P. ^ ^-(Pk.l-P.) ^ ^ ^P,j 
 
 /. . Pv^i -P. 
 
 /^6 7 P ^'^ + /i" (f ^ p^k+i-Pk-i ^ 
 
 -(p 
 
 k+1 
 
 -\J , ^ \.i 
 
 i - 
 
 ./f (^ - p^k.i-Pk-i ^ ;(Px.i-p,.i) ^ ^ ^p^,i-i>k-i^ 
 
 and similarly 
 
 llfoll^/ff^- /^noi^yf^f 3"^"'° . p-^^i-V, . /^1 
 
 + /N ^73^ 
 
 where 
 
 We define x to satisfy the equation 
 
 (I - G) X = Ey 
 
 S.=h a. ? exlats since all the proper values of G are less tl,an one In magnitude 
 by 'assumption k. Hence, 
 
 Gx + Hy - X = 
 
 -^-^ 
 
 + X . 
 
 and .thus 
 
 ± _ 
 
 ^ = 2c^^^ |Gx + Hi^ - X 
 
 Subtracting (1..4.a) from (1^.6) and (4.4.b) from (4.5), we have 
 
 ^.1 = ^\.i^K - \-i) - \_i - ?. 
 
 (^.5) 
 
 ih.6) 
 
 where 
 
 1 = Gd^ - ;^- 
 
 
 (k>l) (1^.7, a) 
 (^•T.b) 
 
4.2 The numerical behav^^ 
 
 We now proceed to develop an upper bound for | | J p'^^ - ? f /^| | 
 ''^^'' Vl = ^' ^^ \+i = ^ for all k, then (4,7) becomes 
 
 ^k+1 = 2bG\ ^ (1 - 2b) \_^ -\ (k > 1) (4.8.a) 
 
 ^ = ^^ - ^0 • (4.8.b) 
 
 The solution of (4,8) is obtained in the same fashion as that of (2.30); only 
 now ve have an inHomogeneous difference equation. Also as in equation (2. 11), 
 
 we have 
 
 G = FO AC *?"■'■ 
 
 where since H is positive definite and symmetric by assumption 3, there exists 
 a matrix F such that 
 
 FF = H 
 
 (4.10) 
 
 A. is the diagonal matrix of proper values of G, 
 is an orthogonal transformation. 
 
 Q = FO 
 Then the solution to (4.8) is 
 
 (4.11) 
 
 ^'\ "Y^k-^ - (2^ - 'K-l] ^~\ - '^' \., ^'' ?, (4.12) 
 
 where 
 
 j=o ^-J J 
 
 = -^:' if ?^ . = 02J --^ i = J 
 
 = ±^ j 
 
 19 
 
-a ^^j and ^^ . are the roots of the polynomial ^ 
 
 J J^J ^^V ^>' ~ • (j = i,2,...if) (4.13) 
 
 7ZT.X'"^'' -'-''^'^■'^' '-'' ^-), u.), u.a), u.,, 
 
 (A. 10), tA.ll) that 
 
 \l/2 
 
 (4.14) 
 Wow for G, 
 
 ^ = — i__ 
 
 1 + A - \'2 (4.15) 
 
 "here^V Is the largest proper value of 0. „e shall first consider the case 
 w.en . < h < 1. P.. S ,, ,,,, ,,^^ ^^^ ^^^ ^^^^^ ^^ ^^_^^^ ^^^ ^^^^^^^ ^^ 
 
 thus 
 
 h\i - 
 
 _v 1 sin k9. 
 
 for 
 
 
 sin 9. 
 J 
 
 9. /O 
 
 kp^-1 
 
 for 
 
 9. = 
 
 Where ^ = /2b - l and cos 9. = bX./p . si^ce L"^"! SUlM | < ^.k-l 
 
 sin 9 
 
 by le^a (A.2), and recalling from theorem (2.3) 
 (4. 14) becomes 
 
 (4,16) 
 
::n general, olio only i:nuv/.odga vg..^11 .h^^e oi'. ^ ia taat WY-W <\\f\\, 
 J > 1, and I l^^l I < I I ^' I I . Hence making this adjustment and summing, we have 
 |H| V/2 , 
 
 H 
 
 V 
 
 f(^^^ M2oll-if^-Mi?'ll 
 
 C 
 
 1. - 
 
 -k" 
 rN2 
 
 k^ 
 
 l?l 
 
 If 1/2 < b < b, then by a similar analysis to the above 
 
 /;7k+l sk+1 ' 
 
 (i^.17) 
 
 
 where 
 
 2b 
 
 
 k-l 3k-l 
 
 - (^i^g) 
 
 2 ^r^ - ^1 
 
 
 
 ^1-^2 
 
 ^^ 1-?^ 
 
 l?ol 
 
 ^ 
 
 1-^1 1 - ^. 
 
 1^1 
 
 (4.18). 
 
 ?^ = b\' +y b^'2 _ 2b + 1 
 
 ?2 = "b^^' -yb'^X'^ - 2b + 1 
 
 Since 
 
 1/2 < b < b and for < \'< 1 , 
 < ^2 < ^1 = ^^' + /(I -b\')^ + 2b(\'.- 1) < bX' + (1 - b\') = 1 . 
 
 Then since 
 Ix p ^ 
 
 -^ 'o-*^ 11/- -\ K- -^ ^\r ■* -1^1 
 
 ■> -P 
 
 ii'p^-^.-p'^ii < ii^iu^-^ni;, p 
 
 ''-/k^/'^ll • ('*.19) 
 
Nov if 
 
 Oi£ 
 
 'A.i ^'''-A,i"p''l < e-,^^ 
 
 
 '; 
 
 (4.20) 
 
 Hence, we have from (4.17), (4.l8), (4,19), (4,20) 
 Theorem (4.1). For the one parameter iteration scheme if 
 
 - / 2 = "^ < ^ < 1 , 
 
 1 + Vl - X'^ ' 
 
 then 
 
 
 H 
 
 \l/2 
 
 AT3 
 
 u 
 
 H 
 
 !/ 
 
 r' p - >^ 
 
 rfflN^o. 
 
 + V''"'' IIP 1 1 
 
 where 
 
 (^ = y2b - 1 , and if I/2 < b < "5 
 
 (1*.21) 
 
 r' - ^' 
 
 ^k-1 -,k-l 
 
 1^2' 
 
 ^1-^. 
 
 It 
 
 ^X-^2.,. 
 
 ^1 - ^2 
 
 I? 
 
 ^1-^. 
 
 1 - r 1 - ^: 
 
 1-^1 1 - ^, 
 
 + /Ne f 3 " . 
 
 (4.22) 
 
 minimum of the right hand side 
 
 We would like to get an estimate of the 
 Of equations (k.2,) and (4.22) with resj^ct to k. This, of course, would involve 
 a knowledge of | [d^l |. „e eaa get an upper bound for the minimum of (4.2i) and 
 (4.22) by simply letting k -» a,. Thus for 
 
 b < b < 1 
 
83 
 
 Wj J 2(b - /iTTT) . p M 
 
 (4.23) 
 Since ^ = /2b - 1, and for 1/2 < b < S 
 
 Min I || p ^k _ 1^ ^ ^Pk| I ^ ^^11^ -Pk _ I ^ ^Pk 
 
 |H| \V2 . 
 
 fH]7 - ' '- , +yN<^-^3P = M' (4.24) 
 ' UJ 2b(l - \') 
 
 since (1 - ^^)(i - ^^) = 2b(l - \'). 
 
 The minimum of M is achieved when b = b. For b = "S 
 
 H 
 
 a/2 
 
 M = 
 
 'u 
 
 111 
 
 |H|y / 2(i-(^) 
 
 + /N <f T P 
 
 On the other hand, M' grows smaller as b approaches b, and the limiting value of 
 M' as b approaches b is 
 
 M' =' ^ 
 
 iHi W^ ,,:, 
 
 '^U; 
 
 ^^ -/n^tP^ 
 
 ^ -Pv ^ . Pi 
 
 Thus we see the upper bound for the Min | |x p ^- f^ r ^ ^\\ is miniinized with 
 
 respect to b when b = b = ^ 
 
 1 + /l - \'^ 
 
 4.3 Terminating the one parameter iteration scheme 
 
 As was pointed out in section (2.4), for the averaging processes one 
 
 ca. estimate | |3 - ^J | from 7^ = G^ . h/ - ^^ or ?^ = 1 - A^^. Furthex^ore, 
 
for ^^ near the solution 5, both | |J^|1 and | [JJ | he=o.e s^all. „e conclude 
 frOM (4.1) that t^ is not difficult to calculate during the iteration process 
 vhereas ?^ may require much additional calculation. For these reasons, ve 
 propose to discuss computations Walch use equations (l».£.a) and te^inate when 
 
 where 
 
 \\\\ < ^ 
 
 6 Is a. preasslgned number 
 ■^k = H% ^ ^^''^ * (5-3*) ^ p'''= - f^ - /"^ (4.25) 
 
 and 
 
 u>^ and i<j^ are defined as for (4.2). 
 
 i^ 
 
 We may revrite z, as 
 k 
 
 k 
 
 z, = rA>, - a^\ ^ /nit^ • ."^k :> . - ^ 
 
 where 
 
 Because of roundoff errors, 6 cannot be any preassigned number. We now proceed 
 
 to develop an upper bound for the Mln \\z 11. 
 
 k ^ 
 
 We have from (4.26) and (4,5) that 
 
 = (I - s)(i p"'''^ - -^"^ ^ ^% . [(^) - p''^ - g; iS + /. 
 
 Now let 
 
 = 5(1 -AW^ [(I p"'''^ - 1 i\ , (? b"^'' . i . «'i 
 
 ■A^ ^'-(A^ -A-p'^)] 
 
by {k.S) so that by (A. 11. a) 
 
 85 
 
 I \^'\\ I <(!..') I |Q-^[J, . 4 i^^ . f^ ^ ,^^3 1 1 ^ 
 
 Then since 
 
 \\\\l<\\iJ\*my)^^'^.(S§)i\, iif-ii , 
 
 and 
 
 |(H|)i t/^ - (h|). p"^k| ^^^ 
 we have from (4,17) and (4.18) for S < b < 1 
 
 \ 
 p ^ ^ 
 
 !?JI< 
 
 IH| \V2 
 
 ^1 (l + \'))p^ 
 
 H 
 
 'ni + ki^)iiiji + kpk-i 11^, 
 
 1 + 
 
 C 
 
 -2 M-Ql 
 
 __(1 - ^)2 1 . ^ 
 
 and for I/2 < b < b 
 
 (^.27) 
 
 ^kM < 
 
 HI f/2 
 
 "' (1+X')M 
 2b 
 
 H 
 
 'X 
 
 '^f ' - ^f ^ 
 
 h-^ 
 
 5k-l ^k-1 
 
 
 + Jn i-f- ^ 
 
 h-h 
 
 l'^2' 
 
 »k 
 
 !ilii^ 
 ^1-^. 
 
 It 
 
 1-?^ x-t 
 
 1-^1 1 - ^, 
 
 (4.28) 
 
 where 
 
 l/iM< 11/' 
 
 We wish to determine an upper boimd for Min 
 letting k-^- in (4.27) and (4.26). Hence, 
 
 z-|^ I I . We may do so by 
 
86 
 Sieor^ (4,2)? For the one parameter Iteration scheme, for ^ < b < 1 
 
 then 
 
 3 /iHi y/^ 
 
 Min z < 
 
 k 
 
 u 
 
 It! 
 
 V 
 
 ^"""'^(ii^^^^^^'J^/^^^p'^^M 
 
 U 
 
 (i^.29) 
 and if 1/2 < b < b 
 
 (i^.30) 
 
 Thus if we choose an ^ which is less than the bounds given by (4.29) or (4.30), 
 there is no assurance that | |f^i | will be less than ^ and hence no assurance 
 that the computation will terminate. 
 
 4.4 The: numerical behavior of the Chebyshev iteration scheme 
 
 Theorem (3.3) showed that the b^'s converged monotonically to b. We 
 now consider the behavior of the b^'s. If we use the Chebyshev iteration scheme, 
 then the b^'s are defined by the recursive relationship 
 
 ^V+1 ~ o — 3 b = 1 . 
 2 - X'\ 1 
 
 k 
 
 Let us assume that 1/2 is exactly representable within the computer; this is 
 
 always true if p is an even integer. Then within the computer, the sequence 
 
 \+l = - 1/27 (-1 + [(^•)^t2]v b^) 
 
 \ = - 1/2 7- (-1+ [(\')^r2]) 
 is generated where ;r and 7 indicate pseudo-multiplication and pseudo- division, 
 respectively. To analyze the sequence Jb^^j , we make the following assumptions: 
 
87 
 (i) If b < c, then axb<axc if a>0; 
 
 (ii) if b < 5 < 0, then a f b < a f c if a < 0; 
 
 (lii) if a > 0, b > 0, then a / b < Min (a,b)j 
 
 (iv) if |a| < |b|, then |af b| < 1; 
 
 (v) if a < 0, b < 0, then a 7 b > 0. 
 Lema (k.3): If < (x.)^ ^ 2 < I/2, then under assumptions {i), (ii), (ni), 
 (iv), and (v), then the sequence fb^j forms a monotonically decreasing sequence. 
 Furthemore, b > 0, k>l so that for k>M, "5=5 - -h- 
 Proof ; We have be definition that 
 
 \ = -1/2 7 (-1+ [(>:')^7 2]) . 
 
 Since < (\')^ - 2 < I/2 -l < i 4. Cf ^^2 • o ^ w^ 
 
 _ \ J . ^ ^ i/.f, 1 < .1 + (X ) 7 2 < - 1/2, and hence by assumptions 
 
 (iv) and (v), 0<b2<l. Since 1 > b^ > 0, 
 
 [(:^')^7 2] X b^ < (?:')^7 2 < 1/2 
 by (iii). Then 
 
 [(X')^7 2] X bg < \'^-2<l/2 
 
 -1 + [{X^f 7 2] / b^ < -1 + (x^f ^ 2 < -1/2 
 
 .l/2^(.l^ (X.)2-^2) > -1/27(^1^ [(X 0^7 2] X £^)>0 
 and, hence 
 
 ^2 - ^3 - ° ° 
 Now assume 1 = "b^ > b^ > b^ > , . . > b^^. Since 
 
88 
 
 JX 
 
 by assumptions (.1) and (iv), and hence 
 
 > -1 ^ [(X'f f s] , i^_^ > .1 , [(^.)2 ^ 2] , ^- _ 
 Thus by assumptions (ii) and (v) 
 
 -1/2 7 (-1 + [(X-)' f 2]/ Vl' ^ -V2 7 (-1 + [(>:')' 7 2] V bj > 
 
 or 
 
 Thus we see that the h^^'s degenerate into a constant/ b', f or k > M 
 and then ve have the one parameter iteration scheme. When b^^ = b', then we may 
 use either process 1 or process 2 which are described in section (2,9). In view 
 of theorem (2,6) which is an exact analytic result, it would seem that process 1 
 is more appropriate than process 2, Thus once the b^'s become constant, they 
 shotild be allowed to remain so. 
 
 The bounds given by (1^.23) and {k.2k) for Mln I I J s'^^ - t ^ r\ I 
 
 i -P i P k /k • P I I 
 
 were arrived at from lim ||x6 ^ - ■>; -ft ^11 q-ir,^^ +v,^.. -k 
 
 xm I I A p / V ^ P I I • bmce these bounds are inde- 
 
 pendent of the initial error, and since the Chebyshev iteration scheme 
 degenerates into the one parameter iteration scheme, these bounds are equally 
 valid for the Chebyshev iteration scheme. Similarly, the bounds given by (4.29) 
 and (I..30) for Min | |lj | are independent of \\%\\ and, hence, are applicable 
 to the Chebyshev iteration scheme. Thus the termination procedures for the one 
 parameter iteration scheme and the Chebyshev iteration scheme are the same. 
 
 h,3 Nmerical behavior of the ge^^ 
 
 In chapter III, it was shown that if a matrix A has property (A) and 
 ordering ^^, then for the operator G = I - d'^ it is only necessary to compute 
 
89 
 
 (2) fo^ the iteration scheme described by {k,l). We shall assume in 
 /2k+l ' 
 
 this section that if a matrix A has property (A), then A, the digital representa- 
 tion of A, also has property (A). Thus for the numerical process described by 
 (^.2), after k iterations ve have 
 
 ^k 
 
 >k73 
 
 Note that we have modified the process slightly so that the vector^ has only 
 one scaling factor associated with it. Let 
 
 "^ " ' J(2) ^(2) 
 
 ^2k+l 
 
 2k 
 
 / \ 2k+l / 
 we now proceed to describe an upper bound for | |? | | when c = b. 
 
 Since A has property (A), the proper values and proper vectors of 
 G = I - D A have the same properties as described in section (3.2). We denote 
 the proper vectors of G as D-^/^f^^ T^^% ^'^^%, D'^/^f. so that 
 
 where M is the diagonal matrix of proper values of G. 
 From (4.12), we have 
 
 Now let 
 
 N^ -r N -r 
 
 t = 6-^2 Z 
 
 ^ t ^ -> 1 _> '2 
 
 '° ' " /itl^^'i ^ i?,^l-'i ' ,?x^^2^1 ^ ,^,^^M,.r% (^-32) 
 
90 
 
 and 
 
 -♦ 
 
 5 
 
 = D' 
 
 f:^.l/2 J V 
 
 N -r 
 
 Mg-r 
 
 L "^'^'^ ' ifi"/.i-^i ^ ,5, "^.i.sA - ^f^ -/,i.H .A 
 
 Then (4.3l) becomes 
 
 t = 
 
 -1/2 r"" -» r ^ \-^ N -r 
 
 /-O 
 
 N^-r 
 
 J,°y,i^k./^i)^ ^ 1^ «/,i.A./-Si)?i - J^ «4i.2A-/°)Pi 
 
 where 
 
 N^-r 
 
 S (fl.) 
 
 "^li '^21 
 ^li ^2i 
 
 Z. a/> . ^T S .(0)q 
 
 if ^ii^021 
 
 = ml 
 
 ;7J 
 .0 
 
 m-l 
 li 
 
 if ^, . = ^ 
 ^li ^2i 
 
 and ^11 and ^^^ are the roots of the equation 
 
 ^j ■ ^"^^J^j + (2b - 1) = (j = 1,2,... N) 
 
 and 
 
 V^j^ - V^j^^j 
 
 Ml. 
 
 'k-l^-j 
 It is not difficult to verify that S {-{1) = (-l)^^X(^.), and, hence. 
 
 S^(0) = if k is even. It is also true then that \{-Jl.) = (-l)^ T (jl.) and 
 hence T^(o) = for k odd. Then in view of theorem (3.1), 
 
2k 
 
 1=1 
 
 2k-l — 
 
 91 
 
 1 
 
 N^-r ^ 
 
 1=1 
 
 2k -1 
 
 ^.2rV0)-/^ "y,1.2Ak-/^l) 
 
 
 and 
 
 2k+l ^2 ) ^ 
 
 /1=1 
 
 r I- 
 
 
 2k - 
 
 f(2) 
 
 1 
 
 N^-r 
 
 2 
 + Z 
 
 '2k 
 Z a 
 
 1=1 L/=l ^^i+V^ 2k-/+l 
 Then by theorCTi (3.2), we see that 
 
 (0) 
 
 1^(2) 
 
 isy^jd) 
 
 2 _ 1 y. 
 
 '1 "2k " - 2 . , 
 
 1=1 
 
 2k-l 
 (a. - a )T (u ) + Z f 
 
 -l/c 
 
 l' V=0 *"^.i ' (-l'^'^/,l.r'S2k-/^l) 
 
 N^-r 
 
 2k-l 
 
 + 
 
 ,5, K2r-2.(°)-^f„-y,i.2AM'^l' 
 
 and 
 
 Jd1/2-(2) 1,2 _1 J 
 
 2 "2k+l 
 
 1 r 2k 
 
 - Z (a - a )T (a.) + Z (a, . + (-l/a^ )s , (n )T 
 
 "^ 1=1 L 1 1+^ 2k+l 1 ^^Q /^i V > /,l+r^''2k-y+l^''^l'' 
 
 N„-r 
 
 2 
 + Z 
 
 "2k 
 Z a 
 
 1=1 Uo /M^+^ 2k-/+l 
 
 (0) 
 
 Hence, 
 
 :dV2?m2 
 
 l\f - w^'Ml^if . \iti/%ii\f 
 
92 
 
 i Z 
 2 1=1 
 
 (a, 
 
 2k-l 
 
 1 1+r 
 
 K^^^,^^'£(-^,,^(-^)\,js,,./^^) 
 
 1 V 
 
 2k 
 
 1 i+r' 2k+l^ 
 
 s^ 
 
 (-. - a,.jT_,(^J .^^^a^^^ , (.ir,^ ...^S^^_,^^C^J 
 
 /,l+rr2k-/+l^^l' 
 
 N -r 
 
 + Z 
 
 i=l 
 
 2k-l 
 
 -|2 
 
 + Z 
 1=1 
 
 _ 2 
 
 '"/,M,+Ak-7+i(°) 
 
 N 
 
 z 
 
 1=1 
 
 /i^/o 
 
 I'TOI* ••• Vsk,! 
 
 where 
 
 (a. - a. ) 
 
 ^"l - Vr^ 
 
 ^2k+l('^i) 
 
 1 = 1,2,. .,r 
 
 1 = r+l, , . . _, 2r 
 
 = a. ^ T (o) 
 i+2r 2k 
 
 i = 2r+l, ..._, N +r 
 
 and 
 
 = 
 
 1 = N^+r+1, .,., N 
 
 f,. = 
 
 (a 
 
 ^ 
 
 >,i ^ <-^> °/.i..) 
 
 /^ 
 
 ^2k.^(^l) 
 
 = 1,2, ...r; A= 0,1,2,. 
 
 ..2k 
 
 / 
 
 (a/ . + (-1) a. . ) 
 
 /2- 
 
 2k 
 
 ->1^^1^ ^ = ""^^^ '"> 2r 
 
 = ^^,1.2Ak-^(0) 
 
 1 = 2r+l, ...,N^+r 
 
 -"/,l+N^+Ak-/fl^°^ 
 
 1 = Nj^+r, ..._,N 
 
Then we see by (A.l) that 
 
 ^3 
 
 ^'^'^ii^/i(;i^ro,.--/..,,r 
 
 N 2k pN ~ 
 
 If'-llJjk^-lfJjU. 
 
 /jA".-'../(4>-.)-4.A)-"i 
 
 Z _ . 
 
 ^-|_ l+2r 2k 
 
 ^.oT1(0) 
 
 yI/ ' A(v,i ^ V.g^^4.e.(^.)^ 3^2^.2,..(^,j ^ |^f«^,,...^.AWi(o) 
 
 N^-r 
 
 >l/ ^ lfi*"2/-l,i - "2/-l,l.r)'(4-2/.l(Si)- CaA^^l) - 2^ "sM,l.Sr4-£r.l(0) 
 
 r,2 / N „2 
 
 ^ ^n^,/^2K(^) - W(-) y? 2 (a, - a.^J^ , Z^ -^ 
 
 CK^O^.V i^K- 2k+r-' ^ 2 .-^ V- - a^_^^; + ^^^ a,_^g^ 
 
 ^ <k7<.. /?>' ^ 4.A) /Tf^^^w^TT^uir 
 
 * ii o5<,/'2'>^-2^'^> ^ 4-.m(^)/ i J^Ca^,., ^ «2/, 
 
 N -r 
 n2 y 2 
 
 N^-r 
 
 ^ A <^ '^^-^^^^'^ * 4-a/..(^) /| ifx'"2^-l,i - '^2M,i.r)'^J^ «I/.i, 
 
 i+2r 
 
From (^.32) we see that 
 
 9h 
 
 i=i - -- i^i 
 
 c 
 i+2r 
 
 Ng-r 
 
 iisy^f;ii^=s (« .. ^^. z- 
 
 N. -r 
 
 i+2r 
 
 ll^i^'^a^-ill' = i '"a.-x,. -..-M./ ^ Y'"^.,. 
 
 SO that 
 
 Now for G = I - D""*"! , 
 
 
 where ^' is the largest proper value of G. We shall first consider the case when 
 
 /J 
 
 b < b < 1. Then 
 
 , , sin kO , 
 V^^i^ =r ^ for 9, i 
 
 "^ sin 9. J 
 
 , -k-1 
 = V for 9. = 
 
where p = /21b - 1 
 and 
 
 and cos © = h^x./p , 
 
 By leimna (A.2) and theorem (2.3), (1+.33) hecomes 
 
 liiii<r'' 
 
 1 + 2k 
 
 
 r2 \2 
 
 -2 
 
 (5^(1 + (2k + 1) ^^^l£ 
 
 -P^Y /|Dj 
 
 1 +f' 
 
 
 D, 
 
 /lT^)(2k + 1) ^^^-^1 '^2'u 
 
 1/2 
 
 D 
 
 1%^' 
 
 95 
 
 + Z (2k 
 ^=1 
 
 2/ + 1) f 
 
 p:2k-2/-l 
 
 D I \V2 
 
 D 
 
 + I (2k - 2/+ 2) <? 
 
 A-± 
 
 
 In general, our only knowledge of f^^\ ff^\ }i^) ^ ^iH ^e that 
 
 ' ^2/ ' ^2/-l 
 
 irf'll<ll?,'^^[|, lll<5^ll<l|T(^)l|and||a)<||T(l) 
 
 2/ 
 this upper bound and sunnning, we have 
 
 ;:2 \ 2 
 
 V 1 + r/ V i + f 
 
 Then inserting 
 
 Jul ||J(1) 
 
 D 
 
 u 
 
 U 
 
 + 2 
 
 r2>k+l 
 
 :2,k 
 
 1 - (f'")"" (k -^ 1)(,°-^) 
 
 1 - e^ 
 
 (1 - ff 
 
 %\y 
 
 fi„i Y/2„-*( 
 
 51; / I5L 2 
 
 
 X.f^ 
 
 
where /o = \/ 2b - l 
 
 If, however, 1/8 < g < ,", ^hen ^y . slMlar analysis to the above 
 
 "^11 s liJ^h^zi^ r-^ 
 
 1/2 
 
 where 
 
 2k>^'- -2k+l' 
 
 W'. 
 
 2b 
 
 ¥2 I ^1 - ?, 
 
 2k+l ;,2k+l' 
 
 ltd) 
 
 D I \V2 
 2'u 
 
 Id 
 
 U 
 
 ll?^^^li 
 
 A_ Pi A - ^1 
 
 2k 
 
 1 -^: 
 
 2k 
 
 ^i-^2L/^2i-^fr/^ rr^i j^ ii'^^^^ii 
 
 r/'^ 
 
 ^Y^' 
 
 uy 
 
 1 1 A - (^1) 
 
 ^^k+l^ 
 
 ^1 - ^2 K^1^2 U - ^1 
 
 1 A - (0p) 
 
 2\k+n 
 2 
 
 hh I 1 - ^: 
 
 96 
 
 D I \V2 ^ 
 
 ^U 
 
 ^1 = bil' +yb^^'2 . 2-b +" 
 
 (^.35) 
 
 ^o = "bi^' -/^u'^ - 
 
 ^t' - 2b + 1 
 
 and 
 
 for 1/2 < b < S , 
 
 7in-l ;7in-r 
 
 m 
 
 ^M ^1 - ^2 ^ ^ ^, - ^. 
 
 1 '"2 
 
 Then since 
 
 |x 3 
 
 k ^ ' ^ ki 
 
 -Pi 
 
 we have 
 
 Theorem (h.k): For the 
 
 X 
 
 -^k -^ 
 
 successive overrelaxation scheme, if b < g < 1 then 
 
 P- - -P. p 
 
 ^ " ■ /k "^ ^^1 I < i |d I I ft ^'k -. nr^ ^ o^k 
 
^ 97 
 
 where an upper bound for | |d^| ] is given by (4.3^), and for l/2 < b < S then 
 
 vhere an upper bound for | |'^| | is given by (4.35). 
 
 mie method of termination for the successive overrelaxatlon method will 
 be to stop the iteration process when 
 
 where ^ is a preassigned number and k > K. If <f is too small, then there is no 
 assurance that the successive overrelaxatlon method will be teiminated. We shall 
 
 now proceed to develop an upper bound for Min II? - B^ - ? ^ ftP| I 
 
 k>K ^^ ' •>^'^ • " • 
 
 We note that for k > K 
 
 - fill ^ ^^ - ^'^' 3- -(fil ^ P^ - 1<^' .-) . 
 
 From (1*.3), we may develop the equation for x p-^ -? ^ pP m exactly the 
 fashion as the equation for t - V^ was derived from (4.4) and, hence, 
 
 - (ab - 1) rV2o s^_^ 0* ei/2(i e-p .f^^^p 
 
 same 
 
 q-1 
 
 i D-V2 S , 0* W^ / . 
 
where 
 
 /^^ is defined as for (I1.3). Then expanding ?s"-P - ^ - a? 
 
 X 3 P - ^j^ -^ pP, and /^_^ . in the proper vectors of G and proceeding exactly 
 in the same fashion as for (4.33), we have 
 
 + E Max |S^ ^>^(n) -S ,rLi)l llfiV2/(2) ,, 
 
 q-2 
 
 S"3 
 
 Let us consider the case vhen b < b < 1. I„ [8], It was shovn by Young 
 that if b is Slightly larger than b", then the Increase In the number of Iterations 
 necessary to reduce the Initial error by a fixed amount Is relatively small 
 whereas If b Is less than b' then there Is a Much larger Increase In the number 
 of Iterations. For b In this range all the roots are complex, and thus 
 
 Bin an9 
 
 \(^j)=r" r^ for 5=0 
 
 "^ Sin 9. <3 
 
 = ^P for 9. = 
 
 ^ 
 
where ^ = (2b - l)^/^. Then 
 
 Max |S (^x) - S (n)| = Max l^"^'- sijLm© _ -m+1 sin(m + 2)9 , 
 0<n<n' ^ °i+2 o<n<7r/2 ^ sin e ^ ' iiT© " 
 
 99 
 
 = Max \f^ r(p"l _ (5) sin(m + l) Q 
 
 0<9<r^2 L^ ^ sin Q 
 
 sin 
 
 ^— 1 
 
 cos - (j5- + ^) cos m© 1 
 
 ;rm r/^-1 
 
 <f [(f" -p)(in+ 1) + (^-1+ p)j 
 
 As a consequence of the above inequalities. 
 
 H^"^^%.,-l^^'-U,r.'\\ 
 
 < Max 
 
 " 0<M,<|I 
 
 , l^£,-2(^) - ^^Ml I |dV2(^(2) p-P . ^^^^^ . pP) I , 
 
 ' 0^^.''2*-3'^^ - Vl^-^l llSy^t''^^ P-^ -;2K^P^)|l-|2fi - H 
 
 =-2^^ c^ 
 
 1 - (f') 
 
 •fl'-T^' 
 
 52^q-l 
 
 
 i5jy^ii?^'ii 
 
 Ct.se) 
 
 where |||(2)|| < ||j(s)| 
 
 it can be shown that 
 
 Id 1^2 |!f(2) _^ p ^ . 
 2j/ ljK+q-1-P ^Jk+q'P 
 
 ^nd Mf^)]iM < 11?^^^ I I . In the same fashion. 
 
 - o5<,.l Vl(^) - Wl(^)l I l^2'/^(^'^^^ 3- vIkIi ^ 3^)1 I 
 
100 
 
 . \2i . l| ^^.Js^,Jn) - s^^(.)| ||5V2(?(l) 3-P -;(^LpP,|[ 
 
 0<|J^)a 
 
 /^2K 
 
 ^2xq .^2sq-r 
 
 ;:-l 
 
 ''-'■^^^Jf^-f^ ^r-f 
 
 ) ^^iO 
 
 s2^>l-l 
 
 •J'-'itW-'^f^j'H': 
 
 1 -f 
 
 i£!ll. 
 
 ri^aiy^iil'^' 
 
 ^iM'J'w?'^ 
 
 + D, 
 
 ly^ii/^^ 
 
 (i^.37) 
 
 We wish to determine an upper bound for the minimum of 
 
 I 'jK+q "^ ^ " Jx+q-l "^ ^^1 I ^^^"^^ is independent of | \t^^^ p"? _ 4^2) ^ ^p^ | 
 
 We do this hy allowing q -^ oo . We have from 
 
 and |l|(i)^-P -;^^^:^pP 
 
 (^.36) 
 
 Min 
 
 q. 
 
 ?(1) ^aP ^(1) 
 
 lljlcitxf^^-J^i^^^llS "HC-,f^'^-/[i^.^|| 
 
 < 
 
 2(r"^ - r) 
 
 (1 - r^)^ 
 
 r. 
 
 q-* oo 
 
 
 D, 
 
 K+q 
 \l/2 
 
 D 
 
 C 
 
 ^^U 
 
 -f" 
 
 
 1/2 
 
 D 
 
 k 
 
 U 
 
 ?(1) 
 
 .M",r> 
 
 4 
 
 ^r„?(. 
 
 l-f-^|BJ^ 
 
 "^'^'"^ff^J( 
 
 i/)Vl^2l.v^,, 
 
 D 
 
 l^ 
 
 l/(^' 
 
 and from (^^.37) 
 \{2) 
 
 r ifeir.^ -/^^3^ii < - iijit,^^^ -ii:^f 3=^11 
 
101 
 
 -1^\ fhKT ^^ji) 
 
 \ - p^ / \ ID' 
 
 r 
 
 / 
 
 Then 
 Min 
 
 SI V/2 
 
 f" |5i 
 
 '''"' lll'^'ll.s 
 
 fcLu! 1/ '"s'u 
 
 :i^ 
 
 5 I \ 1/2 
 
 -2 
 
 ^ -C /I ID 
 
 .r?2) 
 
 'X 
 
 2'u 
 
 1 - r' 1 1^1/ 
 
 lU 
 
 (2) 
 
 ;-l 
 
 f^)fii7r"i'"" 
 
 ri 1 + (^ 
 
 Id, 
 
 < 2(-- ^-1-^. / -ilii li/^^-^l|^+ ^-^ li?^^||^ 
 
 1 - f V ID 
 
 Thus we have 
 
 Theorem (4.5): If k > K and 
 
 'A 
 
 V 
 
 Jii 
 
 /\ 
 
 1 + l/l - il'^ 
 
 = b < b < 1, then 
 
 gll>.-X^P'^-jK..^^^il<2f 
 
 Min 
 q>l 
 
 ■1 /l + r 
 
 Hence, if we choose an t which is less than the boimds given by theorem (4.5), 
 there is no assurance that | \j^^^_^ ^1 pP . J^_^^ .. pP| | ^^ ^^ ^^^^ ,^^^ ^ 
 and thus no assurance that the computation will terminate. 
 
 As was shown by lemma (4.3), the b^'s computed for the modified 
 Chebyshev iteration scheme will become a constant, b', for k > M. Thus the 
 modified Chebyshev method degenerates into the successive overrelaxation method. 
 When b^ = b',then we may use either modified process 1 or modified process 2. 
 In view of theorem (3.6) which is an exact analytic result, it would seem more 
 appropriate to use modified process 1 rather than modified process 2. Thus 
 once the b^^'s become a constant, they should be allowed to remain so. 
 
Min 
 q>l 
 
 lim 
 
 The bound given by theorem (4.5) for 
 ' '/K+q-1 ~ ^ ' fx+q t P i I ■^as arrived at from 
 
 ||(1) • aP S(l) 
 
 Since these boimds are independent of 
 
 and since the modified Chebyshev iteration scheme degererates into successive 
 overrelaxation, the bound is equally valid for the modified Chebyshev method. 
 Thus the termination procedure for the modified Chebyshev method and successive 
 overrelaxation are the same. 
 
 h,6 Bounds for the error vector 
 
 Up to this point, we have only considered the errors caused by pseudo- 
 operations instead of the true numerical operations. However, there are other 
 errors introduced because G, H and y are digitalized quantities. We now develop 
 a bound for | |x - P ^(^ - p ^) I I . 
 
 Now 
 
 + p ^ 
 
 (I - G)-l [(I - G)(l p"'"^ -j^ ^ ^% , (hI) - p'^K . (g^*p-Pk ^ jH 
 
 'h 
 
 and, hence. 
 
 '■'^J 
 
103 
 
 Inasmuch as 
 
 X - X = (I - G)"V - (I - G)'^y 
 
 = [(I - G)-^ - (I - G)-^] Hi? + (I - G)-l (Hi? . Hy) 
 
 = (I - G)-l (G - G)(I .. G)-l H? + (I - 5)-l (Hi? . Hy), 
 
 . .^ ♦ |g - g| , , ^ _•» 
 
 |13-K|| < — 1 Jia ||h|||+ IM^LJdl • 
 
 |I - °> |I - G|/ |I - §1^ 
 
 ^^ '^Ij'ljl - S' '° - '''i ^ "S "y (A-6),: and If l(Hy)^ - (Hy)J < h, 
 I |Hf - Hy 1 1 < /f h. Then by (A. 5) 
 
 |I - SJ^ - |I - G + G - aj^ > |I - Gj^ - |G - 0|^ > |I - g|_^ - Ng . 
 Thus ve have;, 
 Theorem (4.6): 
 
 Where | |/^| | < | |/J | | . 
 
 Thus if Pq = P^ = ... - Pj^ = p, and if p is known, it is possible to detemine 
 I I 2^1 I SO that 1 |x - p \y^ 7 3 ) I I < ^ » 
 
IQk 
 
 V A NUMERICAL EXAMPLE 
 
 5. 1 Introduction 
 
 Up to this point we have discussed a number of methods for solving 
 the system of linear equations 
 
 where A is positive definite and symmetric. In this chapter ve shall discuss 
 a specific example in which the Illiac, the digital computer of the University 
 of Illinois, was used for performing the calculation. 
 
 The system of equations which was solved is the following: 
 
 1 
 
 
 
 
 
 -1/2 
 -1/2 -1/2 
 
 
 
 
 
 -1/2 
 
 -1/2 
 
 
 
 
 
 
 
 -1/2 
 
 -1/2 
 
 
 
 • 
 
 
 • 
 
 I ■ 
 
 
 
 
 
 ! 
 
 1 
 
 
 
 
 ; 
 
 
 
 1 
 
 
 
 -1/2 -1/2 
 
 -1/2 
 -1/2 
 
 I 
 
 i 
 
 N, 
 
 I 
 
 i 
 
 I I ! 
 I 
 
 (5.2) 
 
 1 I 
 
 N 
 
 with N^ = k9, N^ = 50, and N = 99- Since A is non-singular, the solution to 
 (5 •2; is X = where is the null vector. 
 
 The matrix in equation (5.2) arises from a finite difference approx- 
 imation to the differential operator 
 
 d^z(t) 
 dt^ 
 
 (5.3) 
 
105 
 
 on functions z(t) which satisfy 
 
 z(0) = a, z(l) = h, 
 
 A difference replacement of (5.3) is 
 
 1+1 1 1-1 d z 
 
 ^I^P ° ^ '' °(<^*))' 1-1--..,N (5.1*) 
 
 '0 = ^' Vl = " 
 
 and 
 
 N'zit = 1, 
 We may rewrite (5-^) so that 
 
 ^-5 (^.l^^-l' = ^ ft - o(m3) 1 = 1,..,,. (5.5) 
 
 Now let 
 
 ^i=^2i i=l,...N^ 
 
 ^i+N^= ^2i-l i = 1,...,N2 
 
 in (5.5); then the matrix of coefficients associated with x ,..., x is the 
 same as the matrix of coefficients in (5.2). 
 
 The system of equations (5.2) was chosen for the following reasons: 
 
 I. The matrix A has property (A) and ordering o^ and hence the 
 methods of Chapter III are applicable. 
 
 II. A great many properties of the matrix A are known, as will be 
 illustrated below. 
 
 III. Because the convergence properties of the methods to be con- 
 sidered are independent of y in (5.I), we have chosen y = since this 
 
lOo 
 simplifies the conputati^na] procedure. 
 
 VI. Since the Illlac Is a binary -machine, the only numerical 
 operations associated vith the ^trlx A are addition ax.d. shifting «hlch re- 
 ^ ,ulre considerably less time than multiplication for fixed point arithmetic. 
 For the Illiac the maximum roundoff error in performing a right 
 shift equals the maximum roundoff error in accumulating the sum of products 
 and then rounding. Thus there is no reduction in the maximum roundoff arror 
 committed during each iteration by performing a right shift, instead of accumu- 
 lating the sum Of products and then rounding; and, henca, our conclualons about 
 roundoff will be yalid for matrices whoso elements are not of such a simple 
 natyre . 
 
 The largest proper value of A is In- cos ^2., ^^^ ^^^ ^^^^^^^ 
 proper value is 1 - coe ^. The proper vector 5^ associated with the largest 
 pl'Ojjer value has components 
 
 fl,i = ^i^M 1=1, 2,..., N^ 
 ^ f21— 1 ^ir 
 
 yi,i+K'^ " ^^^ %i — i = 1, 2,..., N^ 
 
 and the proj^er vector f^ associated with th, smallest proper value has com- 
 ponents 
 
 2,i+N^ - -^^"^+r~ 1 - 1,..., W^- 
 
107 
 
 Then for G = I - d'""'" A^ 
 
 ^ = -os^ 
 
 A 
 
 b = 
 
 1 + sin ^ 
 
 W+1 
 
 and 
 
 Equation (5.2) was solved using the modified Chehyshev method and 
 
 two variants of the successive overrelaxation method. Variant one consisted 
 Of letting ^/2) ^ ^-1 ^^(2) _ a)^. ^^^ ^^^.^^^ ^^^^ ^^^^^ ^^^^ ^^ ^^^^_^^ 
 
 relationship between ^^^^^ and ^^^2)^ ^,^^ ^^^^^^^ ^^ ^^^ numerical experi- 
 ment was to investigate the following questions. 
 
 1. How are the theoretical results of Chapter III showing the 
 relation between the various methods affected by roundoff error? 
 
 2. How realistic are the various bounds given in Chapter IV for 
 the numerical process? 
 
 3. What is a good terminating procedure when we are interested in 
 acquiring as much accuracy as possible? 
 
 5-2 The numerical procedure 
 
 The choice of initial vector will greatly influence the convergence 
 properties of the three methods. In order to consider the influence of the 
 choice of initial vector, each of the three methods was used on the Illiac 
 with two different initial vectors. 
 
 A set of "pseudo random numbers" V' (i = 1, . . . , N) was generated 
 in the following manner: 
 
108 
 
 ^^3_ « £. -2 mod (2 ) i = 0,,.. 
 '^ = 2365 6938 7927 
 
 where the pseudo operation "-f 2^" indicates a right shift of five. Let 
 
 H = ^2i i = 1>..., 1^1 
 
 The first initial vector for the modified Chebyshev method and the successive 
 ^verrelaxation method, variant one, was ^^^^ (/| tf'^^jj = .119^ 53^2 82) 
 and the first initial vector for the successive overrelaxation method, variant 
 tvo,vas 7(||^(1)||= .1783 5176 4i4). The first initial vector was generated 
 in this fashion with the hope that ?q(= | - ^) would he a linear combination 
 of all the proper vectors of G. 
 
 Corollaries (3.33) and (3.3^) yield bounds for M )\[ These 
 
 11^ oil 
 
 bounds are attainable when i^^^^ = af [^^ = a^f^^^ (a^O). In order to study 
 the effect of roundoff on these bounds the following vector was generated: 
 
 fl = ^i(®i)"2 
 
 5 
 
 ©. = 2i T N+1 i - 1, .,., W 
 
 vhere s(©. ) is the machine representation of | sin ^ as generated by Illiac 
 library routine T-5 which has an accuracy of 2"^^. The vector 
 
109 
 
 f(i) 
 
 Ji 
 
 r^.^ 
 
 V 
 
 "l 
 
 l?(l)l 
 
 niji 11= .0781 2500 00) was used as the initial yector for the modified 
 Chebyahev method and successive overrelaxatlon, variant one. 
 
 Theorem (3.7) yields an upper bound for Irijr for the success- 
 ive overrelaxatlon method, variant two. It is not known if and when this 
 hound is attainable. However, a careful examination of the proof of theorem 
 (3.7) shows that 
 
 4 
 
 vrhere 
 
 rr = Ji r(^2k^) - r' ^^^-M' - (^^^.i^^) -r' ^"2^^^))'] 
 
 1+A-|I^ 
 
 -> 
 when ^ = 
 
 -> 
 
 q; ^ , and 
 
 ''k 
 
 Jl f(S2k<P) 
 
 '0 
 
 5 [(s^Jti) +(0 S2^_^(Ti))2 + fsg^^,^(p) .^2 s^j^n:))^^ (5.6) 
 
 6 
 
 ki 
 
 when £q = ae and, hence, is greater when i^ = a ? . 
 
 In view of the above, the second initial vector for the successive overrelax- 
 atlon method, variant two, was chosen as 
 
 ]l = s(e.)-2« 
 
 e. = 21 -^ N+l 1 = 1,..., Kj^ 
 
1.10 
 
 ©. = (21-1) -:- K+1 1 = 1,..., N^ 
 
 with 
 
 !?"![ = .0276 2135 86. 
 
 The six runs on the Illlac were as follows: 
 
 Run l.a - modified Chebyshev method; f*^-""^ = u^^^ 
 
 Run l.b - successive overrelaxatlon, variant one- f ^"^^ - ^^^-^^ 
 
 ' ^ 
 
 Run l.c - successive overrelaxatlon, variant two; r = -^ 
 Run 2. a - modified Chebyshev method; T = &^^^ 
 
 Run 2.b - successive overrelaxatlon, variant one: r^^^ = ^(^) 
 
 )0 J 
 
 — => _^ 
 
 Run 2.C - successive overrelaxatlon, variant two; f = ^' . 
 
 Each run consisted of 6OO Iterations. Let 
 ""^,1 = " yk,l + ^Jk^i+W + Jk,l+N^+l^-2 i = 1,...,N^ 
 
 Jk,N+l " ° 
 
 "? 
 
 "2 
 
 k,l+N^ " J'k,l+N^ + ^Jk,i-1 ■'Jlcl^ ^ 2 1 = 1,..., N. 
 
 then after each Iteration | |j J | , | | J^ - r ^ | | , and | \7 ^\ | were printed. 
 
 In addition, for the modified Chebyshev method after each iteration b and 
 _ 2k 
 
 ^2k+l ^^^^ printed. 
 
 The initial vector was chosen so that p = p = ... = 0. The 
 computation was carried out in the following manner: 
 
 K'i ' ^■'^ak ®[(Jk-i,i.N^-Jk-i,i.N^.i'-2-7,.i,il Vk-i,.i 
 
 i= 1. 2,..., N^, jVi^^^, = 
 
Ill 
 
 Jk,i+N^ 
 
 with 
 
 = 2c 
 
 = 
 
 2k-.l ® L!Jk,i-l^;k,i)"2-J^_^^ - V^^l,i-,H 
 
 ij 1 
 
 1=1,2,...,!.^, Jk,. 
 
 Jo,i+N^ = ^J 0,1-1 '^Jo,i^ r 
 
 - 2 
 
 i = 1, 2,..., M , 
 
 Jo,o = ° 
 
 for the modified Chehyshev method and the successive overrelaxation method, 
 
 variant one. For the modified Chebyshev method, c = b with 
 
 k k 
 
 .9695^ 58295 < \ < 1 
 and for the successive overrelaxation method, variant one and two, 
 
 Cj^ = b = .9695^ 58295. 
 The pseudo-operation "f 2" indicates a right shift, and "0" indicates a 
 pseudo-multiplication in which the rounding is performed to take advantage 
 of the fact that the next operation is a multiplication by two. 
 Then for i = 1, . . . , N^ and k > 0, 
 
 U 
 
 k,i ' - 
 
 ^c 
 
 2k 
 
 - 2c 
 
 Vk-l,i+N^ "^ Jk-l,i+N^+l^ 
 
 ^^2k ® f^Jk.i,i.N^-Jk-i,i.rT,.i)"2-yk.i,iJ 
 
 ^'^2k f ^Jk-l,i+N^ ■" Jk-l,i+N^+l) T 2 - Jk.i^i] 
 
 + 2c 
 
 2kl 
 
 Vk-l,i+N^ "''/k-l,i+N^+l^ .' 
 
 Vk-l,i+N, "^ Jk-l,i+W +1^ 
 - o — — ± 1 
 
 < s-s-"*" 
 
112 
 
 since for the Illiac 
 
 [2^ - 2xg)y] < 2 
 
 ■ko 
 
 '2 
 
 X T 2 < 2 
 
 -ko 
 
 The same analysis holds for L( I Ci - i w ^ t i^.^. 
 
 ^ jx iux |dj^^^^-[^ I U - i>. . ., W^j. In addition^ sin 
 
 Pq = P._L = ••' = 0, ^^^. = y^^. (i = 1^ 2,..., N). Then 
 
 An upper hound for min \\~j^\\ may be obtained by alloving k- 
 the right hand side of (k.^k); hence 
 
 "^c on 
 
 "jn \\jj,\\<^^+r 
 
 
 -2 
 1-p 
 
 1(1- e^f 
 
 2 
 
 where f = 2b - 1. Then substituting the bounds for | \f^^^ 
 
 the right hand side of (5-3), we have 
 
 ^^^^ M/i,!1 1 28,829 ' 10"^^ 
 
 In addition, from theorem (4.5), we have 
 
 
 f 
 
 -J 
 
 a:id I I #2), 
 
 into 
 
 (5.7) 
 
 mm 
 k 
 
 iij.-;k-iii-< ^r^^]l\\t''\\'^\\r^ 
 
 (5.8) 
 
 < 3567-10 
 
 -12 
 
 Instead of assuming that the maximum error occurs during each itera- 
 tion, let us assume that J, . and Y fi = 1 t\t- v v. n"i «„^ • ^ ^ ^-^ 
 
 k,i ''k^i ^ -L^ • • • ^ ^\i^ k > Oj are independently 
 
 distributed random variables which have a uniform distribution over the range 
 (-3'2 ,3-2 ). Then if the average value of a random variable X is defined 
 by E j X I , we haVe 
 
113 
 
 
 r 3-2 
 
 '3 
 Then substituting the upper hound into (5.6) and (5.7), we h 
 
 ave 
 
 average mln | ij^ -J^J | < 2059- 10"^ . (5.I05 
 
 5*3 The numerical results 
 
 We define 
 
 >» _ _ 
 
 R' - 10. "^i" 
 
 R^ = - log 
 
 ifkl 
 
 10 
 
 ttr -,_^ 11 
 
 H"' = - log 
 k ^10 
 
 
 I If" II 
 1 I j II 
 
 — 
 
 where J^ is the k iterate of the modified Chebyshev method; fj^ is the k*^ 
 
 iterate of the successive overrelaxation method, variant one; and ?'" is the 
 th ^^ 
 
 k iterate of the successive overrelaxation method, variant two. In figure 
 
 I, ve haveR^, r;;, r- fork = 15,..., ^50 when J^^^ ' = r^D" = ^(^) and 
 ^3' = ■^. The curves R', r", r"' represent the maximum theoretical value of 
 ^k' ^k' K ^^ computed from corollaries (3.33) and (3.3^) and theorem (3.7). 
 In figure II, we have R^;., r;^, rJ^' for ], - 15,..., ^50 when f^^^' = J^^)" = jd) 
 J"' =J'. For figure II, the curve r'" was computed from (5.6). Since the 
 
114 
 
115 
 
 \ ^o ?a 
 
116 
 changes in R', R^; and R- appear to te random fluctuations for k > 350, ve have 
 not plotted R^ or r;; or R- for Ic > ^50 nor the related experimental points 
 in figures III through VII for k > i+50. 
 We now define 
 
 "i = -^^^10 llJi-Ji-ilU 
 
 K - -^°gio IIJk-K-iM, 
 ^k = -^°%o lljk-jk'-ill- 
 
 In figure III, we have displayed D^^, D" D"' (k = 15, 30,..., V,0) when 
 ^0 - Jo = ^ ^^d JJ' = ^. The curves D', D% D'" represent the maxi- 
 mum theoretical value Of D', B'^, D- as computed from (3.I7). (3.18), and (3.20) 
 respectively. In figure IV, we have D^;., D^, D'" whenj^^^' = f (l)" = l(l) ^nd 
 y^ =J'. The curves D' and D" were computed from (3. 19). A computation simi- 
 lar to the one used in computing (5.6) shows f or d' = a^ 
 
 "^^"^^-^" = J^ ^''2k(^) - J2k-2(^))' - (jgk.K^) - J2k-l^^ll^oll 
 where (5.11) 
 
 J /-^ = S ,- -2 
 
 CH) - \([I) -^ f' 2^.i(fi) 
 
 a m-1 
 
 In figure IV, D"' was computed from (5,11). 
 In figure V, we have 
 
 K = -i°8,o ||?r|| 
 
 ■10 I i*k 
 
 ^" = - i°«io I l^-k 
 
117 
 
 . iQ -JO /a 
 
118 
 
 
119 
 
120 
 
 
 H 
 
 K 
 
 
 > 
 
 
 
 
 (U 
 
 crt 
 
 0) 
 
 rC 
 
 
 u 
 
 +J 
 
 r\i 
 
 p 
 
 
 
 bO 
 
 Ch 
 
 ra 
 
 •H 
 
 o 
 
 c! 
 
 1^ 
 
 
 ;3 
 
 o 
 
 CO U 
 •H O 
 U Ch 
 
 o 
 o 
 
 o 
 
 H 
 «) 
 O 
 H 
 
 o 
 
 H 
 
 O 
 
 H 
 
 II 
 
 II 
 
 ;:.'^ 
 
 "> 
 
 
 O 
 
 
 O 
 
 o 
 
 O 
 
 cn 
 
 o 
 o 
 m 
 
 o 
 
 OJ 
 
 o 
 o 
 
 OJ 
 
 o 
 
 •,H 
 -P 
 
 cd 
 ^) 
 0) 
 
 -P 
 
 •H 
 
 o 
 
 (U 
 
 X 
 
 O 
 H 
 
 I i T -^o ^-i 
 
121 
 
 H 
 H 
 > 
 
 0) 
 
 u 
 to 
 
 •H 
 
 O 
 
 Ch 
 
 •p 
 o 
 
 ch 
 
 o 
 
 O 
 
 OJ 
 
 CM 
 
 OJ 
 m 
 
 o 
 
 o 
 
 
 O 
 
 o 
 
 J- 
 
 <x 
 
 4«( 
 
 >• •« 
 
 •Jc<] 
 
 • >«'9 
 
 •>f<a 
 
 4« 
 
 Of 
 
 4K« 
 
 ft) , 
 
 -AJ -^ -^ 
 
 1-j — , .^^^ ts-^ 
 
 H H H 
 
 H H H 
 
 q o o 
 
 H H H . 
 
 II II II 
 
 -^ - 
 
 m -^ - 
 
 pq -^ 
 
 o 
 
 o 
 o 
 
 0-) 
 
 
 CO 
 
 o 
 
 a 
 
 LfN 
 
 n 
 
 OJ 
 
 •H 
 
 
 •P 
 
 
 crt 
 
 
 ^ 
 
 
 CD 
 
 
 •P 
 
 
 •H 
 
 
 IfH 
 
 
 O 
 
 O 
 
 u 
 
 o 
 
 (D 
 
 CV) 
 
 rQ 
 
 X* 4 
 
 »« < 
 
 ^ 
 
 o 
 
 * 
 
 o 
 o 
 
 H 
 
 O 
 
 LPv 
 
 O 
 
 o 
 
 OJ 
 H 
 
 1 1 .a 
 
 o 
 o 
 
 JO 
 
 .?a 
 
 O 
 
 o 
 
 O 
 O 
 
 en 
 
122 
 
 (t = 15, 30,..., 1^50) for runs l.a, l.b, l.cj and in figure VI, wq have I^, 
 ^k' K' ^^^ ^^^^ 2-^' 2.l3, 2.C. Since 
 
 '2 ^jk /k-1^ - ^^2k+l 
 
 K'?. 
 
 
 1 - 2c 
 
 2k+l 
 
 2c 
 
 ^(2) _^{2) ,,2 
 
 2k+l 
 
 lirr'-jifiii^ (5.12) 
 
 vhere 
 
 r , 
 
 For the numerical example presented here, D, = I and D =1 so that (5.12) 
 yields a bound for the residual vector. Then using (5.12) and the expansions 
 ^°^ J k ~ Jk-1 ^'^^ Jk " )\-l ^^ section (3.7) we are able to determine 
 uppef bounds for | [i^^^ | | for the three methods. In figures V and VI, the 
 curves L', L", and L"' represent the theoretical upper bounds for L/, L", and 
 
 Till 
 
 Let us define 
 
 
 
 B; = 10^^ 
 k 
 
 Ji 
 
 — y 
 
 B» = 10^^ 
 
 P 
 
 - Tk 
 
 B^' = 10^^ 
 
 
 - \fi 
 
 Where J^ is the k iterate of the modified Chebyshev method if the calcula- 
 tion were performed exactly and J^ J^ ' = a^<l); j^^ is the k"^^ iterate of the 
 successive overrelaxation method, variant one, if the calculation were performed 
 exactly and /q^^ " = «/i^^'' ^^^ f^ i^ the k^^ iterate of the successive over- 
 
123 
 
 relaxation method, variant two, if the calculations were performed exactly and 
 /o"= ^f2' ^"^ ^^g"^^ ^^> ^^ have plotted B^, B^J, and Bj^' for k = I5, 30,...,1|5Q 
 
 5.^ Discussion of the numerical results 
 
 From figure VII, ve are able to observe the growth and behavior of the 
 roundoff errors. We note that for all three methods, the contribution of the 
 roundoff error is approximately the same for k < 315- This is not too sur- 
 prising since the only difference between the modified Chebyshev method and 
 
 the successive overrelaxation method is in the computation of the b 's which 
 
 k 
 
 which vary from b by at most k%. We would expect that the contribution of the 
 roundoff error would be approximately the same also for 315 < k < 1^50. How- 
 ever, we note that for k fixed and greater than 315, there is considerable 
 variation between B^, B,;;, and B^' . It is not known whether this is due to 
 statistical variation or some other cause. 
 
 Equation (5.6) yields an upper bound for min | |? | | which is actually 
 the maximum contribution of the roundoff error and is given by (5.7) as 
 2883-10 . From figure VII, however, we see that for k between I05 and 315, 
 Bj^, B«, and B^^' vary between 760- lo'^^ and 879-10- ^1. Thus the contribution 
 for k in this range is only about | as great as indicated by the upper bound 
 for min |jjj^||. If, however, we consider the roundoff errors to be independ- 
 ently distributed random variables which have a uniform distribution over the 
 range (-3-2 ,3-2 ), the upper bound for the average min | |r 11 is by 
 
 (5.9) 1664-10 which is approximately twice as great as B' or B" or B"' for 
 
 k k k 
 
 k between 105 and 315. For k between 330 and ^50, we have 
 
 max [B^, b;^, B^'] = 1652- 10*^^ 
 
 k=330,...,450 
 and this occurs for B^ when k = 36O. Thus for k in this range, the maximum 
 
12k 
 
 contribution of roundoff agrees very closely with the upper bound for the aver- 
 
 :^ 
 age min \\J^\\. We note that max [B' B" B"'] never exceeds 
 1664-10-". k=15,...,U50 >^- - 
 
 In figure II, we see that for k = I5,..., 325 the experimental points 
 
 ^k' ^k' ^k ^Sree very closely with the theoretical curves R', R", and R'". 
 
 However, for k = 255, we note that R^ and R^ depart from the theoretical values 
 
 whereas Rj^' continues to agree approximately with R'" until k = 3i^5. 
 
 We shall now explain why R"' agrees more closely with R'" than R' 
 
 '^ " k 
 
 agrees with R' or RJJ agrees with R". It is not difficult to see that 
 
 ,r9 
 
 Ro.) - B, = log,, (1 . ^^y^) ■ 
 
 Thus the difference between the theoretical curve and the experimental points 
 
 is a function of the relative error; and, hence, R(k) - R may be large even 
 
 though I IJ^I I - I I r I I is small. From figure VII we see that for k< 315 
 
 I Jj I ~ I Ij'J I for all three methods and 
 
 there is very little variation in 
 
 i^^^ Kjk I I > I lyil I ^^^ I i Jk I I > I Ijkl h R'" (^) agrees most closely with 
 
 
 As a direct consequence of theorem(3.^) for k fixed, we have 
 max R'(k) > i^ax R"(k). 
 
 kT'iM° ii4'^iMo 
 
 The question arises whether this result is anulled by roundoff errors. As was 
 
 pointed out in section (5.2), the maximum of R'(k) and R"(k) is attained when 
 
 ^0 ^ ^Jl (^^^)- From figure II we note that for k = 15,..., 315 R' > R". 
 
 For k > 315 neither R^^ nor Rj^ is uniformly better than the other with respect 
 
 I i f I I 
 to k. Nevertheless, to reduce —^jjr to lO"^, the modified Chebyshey method 
 
 I I Jo II 
 
 required 13^ fewer iterations than the successive overrelaxation method, variant 
 one. 
 
125 
 In fignjre I we observe that for k = 15, ..., 3^5 r" >r'. jhe rea- 
 son for this is that for ^^^^ = 4'^^ max R- and max R," are not 
 
 obtained. Thus we see that although the modified Chebyshev method guarantees 
 fewer iterations for a given accuracy the successive overrelaxation method, 
 variant one, may actually require fewer iterations for a given accuracy. 
 
 In both figures I and II we see that for k = 15,..., 330 R" > R'" 
 and R^ > R^' and that | | JJ^'j | actually increased during the early iterations. 
 
 Indeed, for run l.c, ^7 iterations were necessary so that U ^^- 1 I < 1, and for 
 
 I I r"<\ I ■ ' 
 jrun 2.C, 95 iterations were necessary so that 'i JO'' 
 
 IJS' 
 
 < 1. 
 
 Comparing run l.b with l.c and run 2.b with run 2.c, we see that by beginning 
 the iteration process in a very simple fashion, i.e. y^^^ = D""*" (y^^^ - S^^-*-^) 
 the number of iterations necessary so that the initial error is reduced by a 
 fixed percentage is considerably diminished. 
 
 In figure III and IV we observe the behavior of the difference vector 
 J'k ~ Tk-1' ^°^ ®^^^ °^ *^® three methods. In figure IV we note that there is 
 more agreement between the experimental points and theoretical curves than in 
 figure II. A possible explanation for this is that in computing the difference 
 vector a cancellation of roundoff error takes place which does not occur in the 
 computation of the error vector n . This is verified in the calculation of 
 the upper bounds for min | l]"j^ | | and min \\X:^~ T _^ i | since from (5.7) and 
 
 (5.8) we see that the upper bound for min | | ^^ ~ Jk-i I I ^^ ^^0""t k the upper 
 
 bound for min | | f , | |, and the upper bounds describe the maximum contribution 
 k -^^ 
 
126 
 
 of the roundoff errors. 
 
 For some numerical problems one wishes to iterate until the length 
 of the error vector \\f^-t\\ is as small as possible. If we base our ter- 
 mination procedure on | \f^ - J^_^ | | , the questions arises when do we stop 
 iterating. In both figures III and :I7 we see that D^, D^;, and D- tend to in- 
 crease and then "level off", i.e. remain relatively constant. If we compare 
 figure I with figure III and figure II with figure IV we note that the leveling 
 off occurs for | \f^ - f^^^ | | after the leveling off of | | Jj | . For any 
 
 terminating procedure which is based on the "leveling off" of the I I f - r II 
 
 "y'k ^k-l'l 
 curve, more iterations are involved than would be warranted by the decrease in 
 
 the quantity | | J^ - x i |. The question arises whether we would not tend to 
 overiterate if we based our termination procedure on | 1 ?j^ | |. From figures V 
 and VI we see that the behavior of | | ? | | is very nearly the same as that of 
 "Jk ~ Jk-1 I I ^"^^ ^®^^^ ^^e^s is no advantage to be gained in spending the 
 additional time to calculate | | ?J | . It is not surprising that I I ? 
 I ijk ~J°k-l I I ^^^® similar behavior for this problem in view of (5.12). 
 
 We now wish to consider how to easily sense the leveling off of 
 ' ' Jk ~ Jk-1 ' I' ^^® termination procedure which has been used for s( 
 ical processes is to continue to iterate until some quantity whic:i ordinarily 
 decreases increases or remains constant. Although it appears from figure III 
 that I I J^ - Jk.l I I decreases as k increases, we see from figure IV that for 
 
 k< 15 lljk" Jk-lll actually increases. Thus the aforementioned termina- 
 tion procedure would not be a very good method here. 
 
 For the termination procedure we now wish to propose we are unable 
 to give a rigorous argument; however, for this example it seemed to work suf- 
 ficiently well. From (5. 10) we have an upper bound for the average 
 
 ■ 1 II and 
 k ' ' 
 
 some numer- 
 
127 
 
 "k" "Jk /k-lH- Sence we are reasonably assured that ||r -f n „iii 
 be less than or equal to the bound given by (5. 10). Even ii \\f Sf |[ 
 is smaller than the .Inlmum given by (5.10) one should continue t^ Iterate as 
 long as ||;^-/,.J1 decreases. If ||J,-J^_J| does not change or In- 
 creases then one should terminate the Iteration process since it is not known 
 "''"° lljk -/k-lll "1" a«=rease again. In the table below, we have recorded 
 the value of k for which the iteration process would have been terminated and 
 llJkl I and I IJ^ -J^_^ 1 1 if this iteration procedure had been used for the 
 six runs. 
 
 
 
 TABLE II 
 
 Run 
 
 Terminating value 
 
 of 1: 
 
 % 
 
 l.a 
 
 304 
 
 810-10"^^ 
 
 l.b 
 
 291 
 
 6ko -10" ■'■■'- 
 
 l.c 
 
 382 
 
 702 •lo"-^-'- 
 
 2.a 
 
 309 
 
 90l'l0"-'--'- 
 
 2.b 
 
 330 
 
 1073 •10"-'-^ 
 
 2.C 
 
 366 
 
 336-10"^^ 
 
 lljk-lll 
 
 157- 10" ■'■■'- 
 
 181. 10'^^ 
 
 6.10-^1 
 
 2 -10 
 
 -11 
 
 20-10 
 
 ■11 
 
 24-10 
 
 -11 
 
 ^ We note from table II that by using this terminating procedure 
 
 IIJJI ^for each of the six runs is less than the upper hound for the average 
 "^^ lllkll gi^e° ^y (5.9), that is, average min | |/j^| | < 1664-10-^1. 
 
 5.5 Conclusions 
 
 In chapter III several methods are derived and discussed for solving 
 
128 
 
 Ai?= i^^when A i^ syiometric, positive definite, possesses property (a) and 
 ordering ^^. m this chapter ve have shovn the results of using these methods 
 for solving a particular example in which a high speed computer was used for 
 performing the calculation. In section (5.I) three questions vere posed with 
 respect to the numerical experiment; we shall now discuss these questions in 
 the light of the numerical results. 
 
 The first question asks how the theoretical results of chapter III 
 showing the relation between the various methods is affected by roundoff errors. 
 From figure II we note that for k < 315, 
 
 ^k > «k > K (5.13) 
 
 vhich is as predicted by the analytic results. However, for 315 < k < U50, 
 (5.13) no longer holds true and, hence, the roundoff errors vitiate the analy- 
 tic results of chapter III. 
 
 The question then arises: for a given problem, which of the three 
 methods Should be used? Let us assume that after k iterations the effects of 
 roundoff error is the same for all three methods. From a stochastic model it 
 mj be Shown that the expected value of | | J^ - Jj |2 ^, ^^^^^ ^^ ^^^ ^^_ 
 pected value of | |JJ f - | |f J f, m section (5.4) it was observed that 
 for k < 315 the effect of roundoff as measured by the quantity of 
 
 ll/kll- iijkll 
 was substantially the same for all three methods. This is the basis for the 
 assumption made above. Then under this assumption since 
 
 and since for k and the length of the initial vector fixed | |J - x | | is 
 smallest for the modified Chebyshev method, the modified Chebyshev method 
 should be used in preference over the successive overrelaxation method, variants 
 one and two. 
 
129 
 
 In question two ve query how realistic are the hounds given in 
 Chapter IV for the numerical process. If ve compare the non- stochastic upper 
 bound given hy (5.7) for min | | f J | with the experimental values of ( [J J [ 
 given in table II, then ve see the bound is too large by a factor between 2.7 
 and 8.6 vhereas the stochastic upper bound of (5.9) is too large by a factor 
 between 1.6 and 5.0. Thus the stochastic upper bound is more realistic than 
 the non-Btochastic upper bound for min | |J J | . The non-stochastic upper 
 
 bound given by (5.8) for min Mr - r 11^^ ^4.1- 
 
 ;f \j ) j-ox min \\ij^ - J ^_^\\ exceeds the experimental 
 
 values of | J^^ - /k^il I in table II by a factor betveen 2.0 and I78 while 
 
 the stochastic upper bound is too large by a factor between 1.1 and IO3. Thus 
 
 the stochastic upper bound for min ||/^ - J^_ J | agrees more closely with 
 
 I ITk ""Jk-ll I ^^^"^ ^^^ non- stochastic upper bound. 
 
 The last question asks for a good terminating procedure when we are 
 
 interested in acquiring as much accuracy as possible. In the preceedlng 
 
 section, a procedure was given for terminating the iteration process when it 
 
 was desired to make | | Jj^ - f | | as small as possible in the fewest number of 
 
 iterations. This procedure seemed to vork adequately for the example given in 
 
 this chapter. It would be desirable in future investigations to establish 
 
 rigorously, if possible, a terminating procedure with optimal properties. 
 
130 
 Appendix 
 A.l Basic matrix relationships 
 
 For an n X m matrix, the elements will be denoted by a . or ^A ? . An 
 th , -4 Ij / jij 
 
 n order column vector f has elem-^^nts f f ^ a 
 
 j- i^^ exem_nxs ]^> ^2.' '" fn' ^ ^°^ vector is denoted 
 
 "%> * 
 as 9 . 
 
 Let q;^) = J^ ^^^^, ther, 1 1/| |2 , (J,j?), j,, ..,pp,, ^„^^„ |^|^ ^^^ 
 "lower bound" \k\^ are defined as follows: 
 
 |A| = Max iJALlL 
 " ilflMo ||||| ' 
 
 and 
 
 Thus If A Is symmetric, |a|^ Is the largest proper value of A in magni- 
 tude, and |A|^ is the smallest proper value of A in magnitude. Some basic 
 relationships are given in [7]. They are as follows: 
 
 "A Vs^ ■■• Vkll S li/lll * ■•• - l/jl (A,l) 
 
 l*"'lu = \^\t (A.2) 
 
 l^^lu = l«l |A|u (A.3.a) 
 
 1°^!? = l"l lAy (A.3.b) 
 
 ^^'\ > |AI^ - |B|„ (A.5) 
 
 K--- ^=lu < Klu Usl^ ■■■ |A 
 
 k'u 
 
 (A. 6) 
 
Furthermore! if A is such that 
 
 /or all i,j then 
 
 131 
 
 a. . < C 
 
 l^lu S ^'^ ■ (A.7) 
 
 Also, if A is the transpose of A, then 
 
 *, 
 
 and 
 
 I* lu = lA|u (A.8.a) 
 
 l*\ ' M/ (A.8.b) 
 
 |A*A|^ = |AA»1^ = |A|2 . (,.5,^) 
 
 |A*A|^ = |AA*|^ = lA|j . (A.9.b) 
 
 If is orthogonal, then 
 
 l^lu = 1^1/ = 1 • (A.10) 
 
 Since 
 
 l|Ay|l = li/ll -jMU for ||?|i ^0, 
 
 •>! I ^1 |->l 
 
 II^^M < M^M |A|, (A.ll.a) 
 
 and thi9 obviously holds true for | |^| | = 0. Smilarly, we have 
 
 I 1^1 I >|A|^ M^ll . (A.ll.b) 
 
 A.2 The solution of a finite difference equation 
 
 The following lemma is well known: 
 I^emma (A.l); If 
 
 ^k+l + ^^k + ^^k-l = \ (k > 1) 
 
then 
 
 132 
 
 \ ^2 
 
 k-1 k-1 
 
 ""- = 5^-^^ ^1-^1^2-xr^ 
 
 = k\ 
 
 1 "2 
 k-1 
 
 1 '2 
 k-1 
 
 k-1 \^"J - x^"J 
 y + Z -1 2 
 
 j=l '^1 
 
 1 yi - (k - i)xJyo+"Z (k - j)xf-JS 
 
 j=l 
 
 ^J ^ * \ 
 
 if x.^ = X2 
 
 where \^ and X^ are the roots of the equat 
 
 ion 
 
 >». + a\ + "b = 0. 
 
 A. 3 A usefijil inequality 
 Lemma (A. 2): 
 
 Max 
 O<0<T 
 
 sin kO 
 
 sin 
 
 < k. 
 
 Proof : 
 
 For k = 2, s1il20 ^ p cos Q sin Q . sin 29 
 
 sin 9 "^^ sin 9 > ^^^> hence, . ^'' ^ 
 
 Assiming 
 
 sin(k-l)9 
 
 sin 9 
 
 sin k9 
 sin 9 
 
 < (k - l) , then ve have since 
 
 sin (k-l)9 
 sin 9 
 
 cos 9 + cos(k-l)9, 
 
 < 2 
 
 sin k9 
 sin 9 
 
 < k 
 
BIBLIOGRAPHY 
 
 ^'^ LtialTff ^ Convergence Rates of Iterative Treatments of 
 
 P^ltial Differential Equations , M.T.ATcTTTT ig^o)^ pp. '^^^^ 
 
 ^^■' ^lutlortn^'T^'i^'' Laboratory, A Stu^ of a Numerical 
 
 Solution to a Tvo-Dimension a] HydrodVnami cIT Pr ohi^rr. t^; ah 
 Scientific Lab^tory, Report li:2lif7ifj,^i^:^ ^''"'^ 
 
 [^] J. D. Riley, Iteration Procedures for the DirirhiP+ ■n■;^:■^:• 
 
 Problem, M.T.A:cT^195J^T71SFr]2fri3r ^ ' P^^^"^^"" - " - 
 
 [5] E. L. Stiefel, Kernel Polynomials in Linear Algebra and ThP-i.- 
 iHSpical AE^ncatior^T^^^thiTcoT^rib^t^ 
 
 sLfnv^^ '''°": P'^'"" Equations and the Determination of 
 Eigenvalues, Nat. Bur. Stds . Appl. Math. Series No. 49 (1958) 
 
 ^^^ u'^t'/^""^^' - ^Q"^^a^^son of the Successive Overrelaxation 
 Method aM Semi:,Iterative Meth^s-^gi^^hib ysl^^P^^^ 
 J. Soc. Indust. Appl. MathT7TTl957yrpirf9ri^ Polynomials , 
 
 [7] J. von Neumann and H. H. C-oldstine, Numeric al Inverting of 
 ^tri^J^High order. Bull. Amer. ^^uT^c.Tlfv^.f 
 
 [8] D. M Young, Iterative Methods for Solving Partial Difference 
 
 [9] D. M. Young, Iterative Methods for Solving Partial Mfferenc^e 
 aHaS|2a|^of Elliptic S^, TraSr MS^UaTir^o.fwiM},