LIBRARY OF THE 
 
 UNIVERSITY OF ILLINOIS 
 
 AT URBANA-CHAMPAIGN 
 
 510.84 
 1463 c 
 *o.5l-GO 
 
 [fr^-Tr^ ^; , *^T;/^- 
 
 ENGINEERIN tr 
 
AUG 51976 
 
 The person charging this material is re- 
 sponsible for its return to the library from 
 which it was withdrawn on or before the 
 Latest Date stamped below. 
 
 Theft, mutilation, and underlining of books 
 are reasons for disciplinary action and may 
 result in dismissal from the University. 
 
 UNIVERSITY OF ILLINOIS LIBRARY AT URBANA-CHAMPAIGN 
 
 EN 
 
 «.■{! II a?" a* n 8 
 
 OCT 1 8 fa 
 
 Mr 
 
 A\ 
 
 
 OCT 1 J 
 MAY 
 
 
 o 
 
 6 
 
 FEB 27 \m 
 
 M « 5 IKU 
 
 L161 — O-1096 
 
Digitized by the Internet Archive 
 
 in 2012 with funding from 
 
 University of Illinois Urbana-Champaign 
 
 http://archive.org/details/algorithmforfitt54belf 
 
ENGINEERING LIBRARY 
 SSSslTV OF ILLINOIS 
 
 U URBAHA.ILUHOIS 
 
 Mlll#lll»llL# 
 
 UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN 
 
 URBANA, ILLINOIS 61801 
 
 CAC Document No. 5^ 
 
 AN ALGORITHM FOR FITTING 
 RELATED SETS OF STRAIGHT-LINE DATA 
 
 By 
 
 Geneva G. Belford 
 
 December 13, 1972 
 
CAC Document No. 5k 
 
 AN ALGORITHM FOR FITTING 
 RELATED SETS OF STRAIGHT-LINE DATA 
 
 By 
 
 Geneva G. Belford 
 
 Center for Advanced Computation 
 University of Illinois at Urbana-Champaign 
 Urbana, Illinois 61801 
 
 December 13, 1972 
 
 This work was supported in part by the Advanced Research Projects 
 Agency of the Department of Defense and was monitored by the U. S. 
 Army Research Office-Durham under Contract No. DAHCOlj-72-C-OOOl. 
 
*WeWFERfW6 UBfWl 
 
 ABSTRACT 
 
 Many physical experiments give rise to sets of curves related by 
 the requirement that, although certain of the curve parameters may 
 vary from curve to curve, others should be the same for all of the 
 curves. To get the "best" values of the common parameters, one would 
 like to fit all of the curves simultaneously by the appropriate 
 theoretical expressions. This paper deals with this problem, present- 
 ing an algorithm for its solution, in the case that the curves are 
 straight lines with common slope and "best" fit is defined in the 
 uniform (or minimax) sense. 
 
TABLE OF CONTENTS 
 
 Page 
 
 1. INTRODUCTION 1 
 
 2. CHARACTERIZATION AND UNIQUENESS 3 
 
 3. THE ALGORITHM 6 
 
 k. DISCUSSION 10 
 
 REFERENCES 12 
 
1. INTRODUCTION 
 
 In numerous physical problems data is obtained which is essentially- 
 linear, and parameters best describing the linear relationship are 
 desired. In some situations related experiments give rise to sets of 
 straight-line data which should all have the same slope although the 
 intercepts may vary. For example, one might be observing an exponen- 
 tial decay process 
 
 (1) x(t) = ae" 3t 
 
 in which the initial amount, a, varies from experiment to experiment, 
 but 6 should be constant. Taking logarithms in (l), one is led to the 
 straight-line analysis problem with (3 appearing as the slope. 
 
 Formally, we state the problem as follows. Given a set of experi- 
 mental curves g , g , ..., g , find parameters a , a , ..., a , 6 
 
 such that the deviations E. (x) = g.(x) - a. - Bx are small in some 
 
 li l 
 
 sense. One obvious approach is to minimize 
 
 N 
 
 I I [E.(x )] 2 
 
 where the x.'s are the points of observation. This is a standard least- 
 J 
 
 squares problem, solvable by the usual technique. It is perhaps 
 
 interesting that if one simply computes least-squares lines a. + $.x 
 
 to fit each g., then the 3 which minimizes (2) is just 3 = (V g.)/N. 
 i ^ i 
 
 1 
 
 Any least-squares approach suffers, however, from the drawback that 
 errors at individual points - or for certain curves - may be very large. 
 To avoid this problem, we suggest using a uniform or minimax approxima- 
 tion procedure; i.e., minimization of 
 
 (3) M = max max |E. (x) | , 
 
 i xeX x 
 
where X here represents either a closed, bounded interval or a finite 
 number of points. A theorem characterizing solutions to this type of 
 problem in a very general context has been given in [l]. In this 
 paper we state hov the theorem applies to the problem at hand and 
 discuss a numerical algorithm, based on the theorem, for the calcula- 
 tion of these best uniform approximations. 
 
2. CHARACTERIZATION AND UNIQUENESS 
 
 Let A = ( a -, , • ••» a w j ^) be a parameter set defining a proposed 
 approximation to g , ..., g„. Define an extremal (x, k) of the approxi- 
 mation A to be a number pair such that M = |E (x)|. Let a = sgn K (x). 
 The following theorem then follows immediately from Theorem 3 of [l]. 
 
 Theorem 1. A necessary and sufficient condition that A define a 
 best uniform approximation to g , •••> g^ is that there exists no para- 
 meter set A' = (a' a', .... 0* , 3 ) such that 
 
 1 = (a^, o^, . .., a^ , 3') 
 
 (1+) a 1 [a« + 3'x] > 
 
 xk k 
 
 for every extremal (x, k) of the approximation A. 
 
 This result is, of course, completely analogous to the usual Chebyshev 
 single-function result that an approximation can be improved if and only 
 if some member of the approximating family takes on the same signs as the 
 error curve does at its extremal points. In fact, a better approximation 
 may be constructed by adding on to the original approximation some small 
 multiple of any function satisfying the sign condition. Although this 
 "inequality" approach can be used to construct best single-function 
 approximations [2], it is very seldom done, presumably because Remez-type 
 algorithms [3] are better studied and known to converge rapidly. 
 For our problem, however, Remez-type algorithms appear to be diffi- 
 cult to construct, and we shall here discuss an algorithm based on 
 (U). 
 
 There are two conclusions to be drawn from Theorem 1 which are useful 
 in simplifying the procedure. 
 
 Theorem 2. A necessary and sufficient condition that A define a best 
 uniform approximation to g , . . . , g is that one of the following two 
 conditions hold: 
 
(i) For some index i, a. + 3x is the best straight-line approxima- 
 tion to g., and for all J ± i 
 1 
 
 max |g.(x) - a - 3x| < max |g (x) - a - 0x| . 
 xeX J 3 xeX 
 
 (ii.) For some pair of indices i, J, the parameter set (ou, a^ , 3) 
 defines a best simultaneous approximation to the pair g ± , gy and for 
 
 all k ^ if J 
 
 max |g k (x) - a k - 3x| * max |g.(x) - a. - 0x| = max |g.(x) - a. - 3x 
 
 This theorem may be verified by carefully checking the various possibilities 
 for (h) and noting that no more than two indices k can enter into the 
 characterization in a significant or non-redundant fashion. In the same 
 manner, one may verify the following important fact. 
 
 Theorem 3. Although there are in general many parameter sets A 
 defining a best approximation, the parameter 3 is unique. 
 
 This result is of extreme importance for applications; it is very 
 often this best slope that one wants to find. Putting together Theorems 
 2 and 3, we see that the main problem is then one of finding this best 
 3 , and that this 3 must lie among those obtained by constructing single- 
 function approximations ((i) of Theorem 2) or by constructing best 
 approximations to all function pairs ((ii) of Theorem 2). For the 
 latter, we use a procedure based on (U), which is not very complicated 
 when only two functions are involved. In fact, we have the following 
 simple "alternation" theorem. 
 
Theorem h_. 
 Suppose that the best simultaneous approximation to the pair g n , g_ is 
 not characterized by condition (i) of Theorem 2. Then there exist four 
 points x , , x , x , x in X such that 
 
 (a) x 1± < x 12 ; x 21 < x 22 ; 
 
 (b) (x 11 , 1), (x^, 1), (x 21 , 2), (x 22 , 2) are 
 extremals of the approximation, and 
 
 (c) the error signs satisfy 
 
 °J1 °j2 = - 1 ' (J = X ' 2) 
 
 "ll °21 = - 1 
 
 (where a.. = sgn E.(x..)). 
 
 J -L J J - 1 - 
 
 2 3 
 Example . Suppose g = x , g = x and X = [-1, l]. One can quickly verify 
 
 that the best approximation is provided by a =0.65, a = 0, 8 = 0.325. 
 
 Extremal x values are x = -1, x^ = 0.1625, x = -1, and x = +1. 
 
 The error norm M is 0.675- The reader is invited to provide his own 
 
 quick sketch to see that the sign condition (c) of Theorem k is satisfied. 
 
3. THE ALGORITHM 
 
 f v \ a (y) a (x) defined on X, one proceeds 
 
 Given functions g-^x), S 2 { ' y ""' g N 
 
 as follows: 
 
 1. For i = 1, ..-, N > use the Remez algorithm (available in most 
 subroutine libraries) to compute the best straight-line approximation 
 C + 3.x to g.(x) on X. Let p = max |g (x) - £. - S.x|. 
 
 Ill x X£A 
 
 2. For each 3. (i = 1, . • • , N) found in Step 1, and for all j ¥ i, 
 compute the best constant approximation Cj to gj - 3^, and let 
 
 =max|g.(x)-3.x-c |. (This is a trivial computation. Simply 
 find r^ max ( gj (x) - 6.x) and r g = min ( gj (x) - 3.(x)). Then 
 c. = (r x + r 2 )/2 and p ±J = (^ - r g )/2. ) If for some i, p.. < P- for all J, 
 then a best approximation is constructed by taking a. = c. (j * i), 
 
 a = c and 6 = S.. (Note: This is the situation described in case (i) 
 
 i i' i 
 
 of Theorem 2, where the best overall slope 3 is just that obtained for 
 
 the best individual approximation to g.. The nonuniqueness of the a.'s 
 
 also shows up clearly here, since if p.. < p. for some J, a small change 
 
 in a . can be made without affecting the max error.) If a best approxi- 
 
 J 
 
 mat ion has not yet been found, one proceeds to Step 3. 
 
 3. One now examines function pairs. Clearly, for any pair i, J 
 
 such that either p.. < p. or p < p a best simultaneous approximation 
 
 ij i J x *) 
 will be characterized by slopes of p. or g. respectively. Since these 
 
 3 values have already been shown not to yield a best- overall approxima- 
 tion, it is only for i, J pairs satisfying neither of the above 
 inequalities that we now wish to compute a best simultaneous approximation 
 
{y . + nx, Y. + tix). The procedure is iterative. Begin with initial 
 
 J 
 
 guesses of the parameters. We have used 
 
 n (0) = (3. + 3.)/2, 
 i J 
 
 y. 0) = c. + (3. - 3.) (a + b)A, and 
 y/ 0) - C. + (3. - 3.) (a + b)A, 
 
 but a better choice may be possible. (Here the parameters a. b define 
 
 the interval of approximation; i.e. X = [ a , b]. If X is a finite 
 
 Q 
 
 point set {x } , , then we assume a = x., < x < ... < x- = b.) 
 
 ^ q q=l 12 Q 
 
 Letting 
 
 F^ (x) = g k (x) - Y k - n x. 
 
 compute, for l = 0, 
 
 R = max {max |E. (x)|, max |E. (x) } , 
 
 x x x J 
 
 and then iterate. (For simplicity, superscripts indicating the iteration 
 number will be generally omitted. ) 
 A. Determine points 
 
 and 
 
 X il < X i2 < 
 
 x.., < x < 
 Jl J2 
 
 where E.(x) (or E.(x)) takes on the value ± R (to within some preassiened 
 i J 
 
 tolerance, of course), and let associated error signs be a n • The 
 
 km 
 
 inequalities (k) are now readily treated by ad hoc methods. For each k, 
 the list {a } may contain no more than one sign change. (Otherwise we 
 would have a best individual approximation - a possibility already elimina- 
 ted in Step 2.) One therefore checks each of the two lists for whether 
 
8 
 
 it (a) is empty (b) has no sign changes, or (c) has one sign change. 
 If both lists have a sign ehange, and the changes are in opposite 
 directions (i.e. one ♦! to -1 and the other -1 to + l). then W has 
 no solution. In fact, this is just the situation described in 
 Theorem k; we know, therefore, that the current values of Yj, Yj, n 
 are the best and we go on to Step It. 
 
 Otherwise, one readily constructs a convenient solution to the 
 i„e 9 ualities depending upon the outcome of these tests plus the direc- 
 tion of the sign change (♦ to -, or - to ♦) if one occurs. For example, 
 suppose that list i has a sign change, say o. 2 0.3 = -1 with a ±2 = +1, 
 
 «,„+ list i is empty. Then (M may be satisfied by choosing 
 and suppose that list j is "*"■' 
 
 8 ' . -1 .I - (x.„ ♦ x. J/2, and a\ - 0. A simple scheme taking care 
 
 of all possibilities is easily designed. 
 
 -, *• v, ' i"™ 1 + «'* a' + 6 x} to the inequalities 
 B. Suppose a solution ic^ + B x. «j * P w 
 
 ( ^ - mfl y Ie (x) - e(o' + g'x)| for k = i, j, and 
 has been found. Let m^U) - max ji^W ev k I 
 
 let M (e) = max { m. , m.>- For some small positive value of b, M(e) < R. 
 After finding such an e (see later discussion), set 
 
 ( & +i) = U) + ' , k = i, j, 
 
 Y k ^k k 
 
 ^ - n (£) + .3 1 . 
 
 andR ( * +l) =M(s). 
 
 Then return to Step A. 
 
 k. At this point we have obtained a best simultaneous slope n(i, j) 
 and corresponding approximation error R(l. j) for the pair g,, Sy 
 Precisely as in Step 2 one now checks whether the rest of the » curves 
 
can be fitted by straight lines of slope n(i, j) and with errors no 
 larger than R(i, j). If they can, we have solved our problem with 
 6 = n(i, j) and criterion (ii) of Theorem 2 is satisfied. If they cannot, 
 we go back to Step 3 and look at the next function pair. Theorem 2 
 guarantees that at least one pair will yield a 3 that works. 
 
10 
 
 h. DISCUSSION 
 
 The algorithm has been programmed (by Dr. Joseph Garber) and tests 
 on sets of simple functions indicate that it works well. 
 
 In view of Theorem h, one would expect that a minimum of three 
 iteration steps (cycles through Steps 3A - B) would be required to 
 obtain a best approximation to any pair. That is, in general there 
 will be only one extremal point for the initial guess and each itera- 
 tion step is not likely to add more than one additional extremal 
 point. In tests on four function pairs, the actual number of steps 
 varied from a near-minimal 5 to a reasonable 9« Small changes in 
 algorithmic details may improve this further. 
 
 A key factor affecting the number of iteration steps required is 
 the choice of the adjustment parameter e (Step 3B). It therefore 
 seems worthwhile to include here a brief discussion of how this para- 
 meter is chosen. Initially, a guess for e based on theory (cf. proof 
 of Theorem 2 in [l]) is computed: 
 e Q = H/2U 2 , 
 
 where H = max max [a + 3 x| , 
 k=i,j x 
 
 and 
 
 U = min {R k +8x.|l , 
 
 the minimum being over all extremals (x. .,k). 
 
 If the test M(e ) < R is not satisfied, we then try e = 2~ J e for 
 j = 1, 2, ... until for some j = J we find M(e ) < R. Examination of 
 the tests shows an average J of about 5. Furthermore, an easy way to 
 decrease this is immediately evident. The larger J values were often 
 associated with an iteration step in which the solution to (h) was 
 identical to that obtained on the preceding step. It is reasonable 
 
11 
 
 that, for two successive corrections in the same direction, the second 
 should be of smaller size; a test and adjustment of e for this 
 situation may he readily incorporated into the program. 
 
 After e. is obtained, this value is improved by computing 
 M (e p/10) for p = 1, 2, ..., 10, 
 and choosing as e that value of e p/10 which minimizes M. It is 
 possible that the number of iteration steps (Steps 3A - B) may be 
 decreased by sweeping a finer grid at this point, but this may not 
 effect any overall saving in time. It seems likely, however, that 
 some very different approach from that described here may lead to a 
 really efficient way of computing a near-optimal e. 
 
12 
 
 REFERENCES 
 
 [l] G. G. Belford, 'Uniform approximation of vector-valued functions 
 with a constraint', Math. Comp. 26, 1+87-^92 (1972). 
 
 [2] E. Stiefel, 'Numerical methods of Tchebycheff approximation', 
 
 in On Numerical Approximation , R. E. Langer, ed., University of 
 Wisconsin Press, Madison. 1959- 
 
 [3] G. Meinardus, Approximation of Functions , Springer-Verlag, New- 
 York, 1967, pp. 105-116. 
 
UNCLASSIFIED 
 
 Security Classification 
 
 DOCUMENT CONTROL DATA -R&D 
 
 (Security claf Ideation of tlllm, body of aba tract mnd Indamrnt annotation mil ba antarad whan tha ovarall report la claialtlad) 
 
 I. ORIGINATING ACTIVITY (Corporal* author) 
 
 Center for Advanced Computation 
 University of Illinois at Urbana-Champaign 
 Urbana, Illinois 
 
 ia. REPORT SECURITY CLASSIFICATION 
 
 UNCLASSIFIED 
 
 2b. CROUP 
 
 3 REPORT TITLE 
 
 An Algorithm for Fitting Related Sets of Straight-Line Data 
 
 4. DESCRIPTIVE MOTE* (Trpa o I taper t mnd htelualra dmtaa) 
 
 Research Report 
 
 8 authoR(S) (Fttat nmmta, mxlddta Initial, laat nama) 
 
 Geneva G. Belford 
 
 a REPORT DATE 
 
 December 13, 1972 
 
 Ta. TOTAL NO. OF PACES 
 
 JLS. 
 
 76. NO. OF REFS 
 
 Sa. CONTRACT OR GRANT NO. 
 DAHC04 72-C-OOOl 
 b. PROJECT NO. 
 
 ARPA Order No. 1899 
 
 •a. ORIGINATOR'S REPORT NUMBER(S) 
 
 CAC Document No. 5h 
 
 OTHER REPORT NO(S) (Any othat number* that may ba aaalanad 
 thla tapatt) 
 
 10 DISTRIBUTION STATEMENT 
 
 Copies may be requested from the address given in (l) above. 
 
 II. SUPPLEMENTARY NOTES 
 
 12. SPONSORING Ml LI TARY ACTIVITY 
 
 U. S. Army Research Office-Durham 
 Duke Station, Durham, North Carolina 
 
 13. ABSTRACT 
 
 Many physical experiments give rise to sets of curves 
 related by the requirement that, although certain of the curve 
 parameters may vary from curve to curve, others should by the 
 same for all of the curves. To get the ,f best" values of the 
 common parameters, one would like to fit all ot the curves sim- 
 ultaneously by the appropriate theoretical expressions. This 
 paper deals with this problem, presenting an algorithm for its 
 solution, in the case that the curves are straight lines with 
 common slop and "best" fit is defined in the uniform (or minimax) 
 sense. 
 
 DD ,'°?..1473 
 
 UNCLASSIFIED 
 
 Security Classification 
 
UNCLASSIFIED 
 
 S»curity Cl>««iflc«tion 
 
 K CV WORDS 
 
 Interpolation; Functional Approximation 
 Uniform Approximation 
 Linear Approximation 
 Simultaneous Approximation 
 Construction of Best Approximations 
 
 LINK A 
 
 IOH *T 
 
 KOLI WT 
 
 LINK C 
 
 ROLI "T 
 
 IMHASaiEIEII 
 
 Security Claamification