LIBRARY OF THE 
 
 UNIVERSITY OF ILLINOIS 
 
 AT URBANA-CHAMPAICN 
 
 510.84 
 
 ho. 22-59 
 
 <p-0 H 
 
 COI 
 
Digitized by the Internet Archive 
 in 2013 
 
 http://archive.org/details/upperboundfornum58mull 
 
UNIVERSITY OF ILLINOIS 
 GRADUATE COLLEGE 
 DIGITAL COMPUTER LABORATORY 
 
 INTERSIL REPORT NO. 58 
 
 AN UPPER BOUND FOR THE NUMBER OF CERTAIN 
 ERROR CORRECTING CODES 
 
 BY 
 
 D. E. MULLER 
 
 August 9, 195^ 
 
An Upper Bound for the Number of Certain 
 Error Correcting Codes 
 
 Let S "be the set of n dimensional vectors whose components 
 may take only values or 1- An error correcting code is a sulset 
 T of S having the property that any pair of vectors in T differ in 
 at least d components. The general question of how numerous a set 
 T may he found for given n and d is of importance in information- 
 theory and will he treated in this discussion. 
 
 Fano has shown that if a set T can he found having the 
 property that every member of S lies a distance of less than d/2 
 
 from some member of set T, then the number of members of the set 
 
 i 
 T approaches 2 
 
 "by the formula 
 
 nK 
 T approaches 2 as n approaches infinity. The quantity K is given 
 
 K = 1 + (log d/2n) d/2n + (log (l - d/2n))(l - d/2n) 
 
 (1) 
 
 nK 
 This number 2 represents an upper hound for the numbesr of members 
 
 of the set T,and if it could he achieved the resulting set T would 
 
 he an error correcting code of maximum efficiency. 
 
 Fano's formula has not been achieved by actual codes in 
 
 which d/2n is held constant as n -» OO , and there may even be 
 
 some doubt as to whether any non-zero constant K may be found to 
 
 satisfy the formula 2 . In the special case of n = 2? where p is 
 
 an integer it is possible to construct a set T whenever d is a power 
 
 of 2 , say 2 . The number of members of this set T is given "by the 
 
 formula 
 
 EI*]. (2) 
 
 i=0 
 
 The sum of binomial coefficients in the exponent will be a polynomial 
 
 p 
 in p of degree p-m and consequently cannot be proportional to n = 2 . 
 
 Robert M. Fano "Notes for Statistical Theory of Information" (1953) 
 Page IV- 3k, Mass. Inst, of Technology 
 
-2- 
 
 In the case m ■ p-1 we get d = 2 P " = n/2 and the number of Members 
 of set T is merely 2 p - 2n. The theorem given here treats the case 
 of general n with d = n/2 and shows that the set T can centain no 
 more than 2n members and that the formula of Fano gives, therefore, a 
 much higher than necessary upper bound in this ease. 
 
 Let us designate the number of components in which two vecters 
 a and b differ by the symbol L(a,b). The condition that d = n/2 may 
 then be written as L(a,b) > n/2 if a and b are in T« 
 
 THEOREM: It is not possible to pick a subset T of more than 
 2n members from the set S of n dimensional vectors whose components are 
 or 1 if one requires that any pair a, b from set T satisfy L(a,b) > n/2. 
 If the condition is made L(a,b) > n/2 it is never possible to pick 
 more than n+1. 
 
 PROOF t Let (a Q , a.,..., a ,) represent the n component vector 
 "a M whose components a. are either or 1. Let us adopt the notation 
 (A- , A_ , . . . ,A , ) to represent the transformed vector A, where the vector 
 A will be transformed by the formula 
 
 k ± - (l-aa^/fi (1) 
 
 Thus if a. = 0, A = l/ fn and if a ■ 1, A = -l/ fn". These are the 
 only two possible choices for A. and consequently 
 
 n-1 o 
 
 z (A.r = i. (2) 
 
 i=0 x 
 
 We shall let V represent the broader class of vectors satisfying (2 ) 
 (these are unit vectors). Transforms from set S are in V but, of course, 
 V contains vectors which are not transforms from S. The theorem will be 
 proven for set V and as a result will apply to the smaller Set S. 
 
 Formula (l) allows a numerical representation of the quality 
 L(a,b). If A. is the transform of a. and B. is the transform of h 
 
-3- 
 
 then l/2 - n/2 A. B. will be If a and b agree and 1 if a. and b 
 disagree. By summing such expressions we obtain 
 
 n-1 
 L(a,b) = (n/2) (l - 2 A, B ) . (3) 
 
 1=0 
 
 This formula permits the condition L(a,b) > n/2 to be written 
 
 n-1 n-1 
 
 Z A . B < and the condition L(a,b) > n/2 to be written Z A.B < 0. 
 
 1=0 "" 1=0 1 1 
 
 Let us assume that a set T has been found containing e members 
 
 r ,r , ...,r ~ which satisfy the condition L(r ,r ) > n/2 if j 4 k » 
 
 The transformed set V contains e members R , R , . . . , R and satisfies 
 
 the condition 
 
 n-1 . , 
 
 Z R^ R? CO (k) 
 
 1=0 
 
 The set of vectors V may be rotated if each vector is post multiplied 
 
 by a unitary matrix M. Under such a rotation the scalar product 
 
 a-1 
 
 Z A.B ** invariant and condition (k) will therefore continue to hold. 
 
 1=0 
 
 n-1 
 To show that Z A B is invariant under a rotation we need merely 
 
 1=0 1 i 
 
 write Z A.B. = AB . But A' = AM and B' = BM. Hence A' (B T ) = 
 1=0 
 
 rn rp rp. . rp — "t 
 
 AM (BM) = AMM B '. M, however, is unitary so that M"^ = M and 
 
 AMM'V = AIB T = AB T . 
 
 We now show that it is possible to rotate V into V so that 
 (R )' = (1,0,0,0, .. .,0). lit other words we shall perform a rotation M 
 so that R transforms into (1,0,0, .. .,0). We begin by performing a 
 rotation by M. , where M. is a unitary matrix defined as follows, 
 
-4- 
 
 , "5 
 
 < 
 
 w * w 
 
 i^f * («;) 2 
 
 
 < 
 
 W ♦ (»lf 
 
 \|(» °) 2 + (»?) 2 
 
 
 
 
 
 
 
 
 
 . .. 
 
 ... 
 
 ... 
 ... 
 
 When one post multiplies R by M. no components axe changed except 1L. 
 
 ■which becomes \(R n ) + (R, ) and R, which becomes . Similarly a 
 
 u L f 4 
 
 matrix VL may be constructed which alters only R Q and R_, making Rp 
 
 equal to etc. The matrix M = M_ ML M,,...,M , will thus rotate R 
 
 into (1,0,0, ...,0). 
 
 We now remove (R )' from V f and place it in a set W. If the 
 
 •vector (-1,0,0, .. .,0) is in V T we also place it in W, removing it from 
 
 V . We see that all remaining vectors in V must have nonr^positive 
 
 n " 1 1 1 
 first components since Z (R )' (R^) 1 = (R^)' < 0. Thus we may write 
 
 i=0 1 1 
 
 the condition 
 
 n-1 
 
 Z 
 
 i=0 
 
 Z (rJ)« (R*)< <0 
 
 whenever (R )' and (R )' are in V as 
 
 n-1 
 
 (r£)» (Rq)' + Z (R^)« (R*)« <0 . 
 
-5- 
 
 But since (R^)' s-nd (R^V ore both non~poaltiv» we have (Uj*)' (R^)' > 
 
 n-1 . . 
 and hence Z (X ) 1 (R )' < (5) 
 
 1=1 l 1 " 
 
 Condition (5) say now replace condition (fc) with one less dimension 
 
 1 
 
 and with one or two less vectors In V than we had in V. It is tr^e that 
 the normalization condition (2 ) may not hold for the n-1 dimensions hut 
 condition (5) will not he affected if we multiply any vectors in V hy 
 suitable constants to make it hold. We see that any vector in V whose 
 last n-1 components are zero has been removed from V and placed in W 
 so that normalization is always possible. 
 
 The entire process described here may be repeated for the set V* 
 in n-1 dimensions and a new set V" constructed. Bach time this process 
 is repeated we put at most two vectors in W and lose one dimension. Thus, 
 after n repetitions we have placed all e vectors in W and hence © may be 
 not greater than 2n. This proves the first half of the theorem. 
 
 To prove the second part of the theorem we follow the same process 
 but use the condition 
 
 E R^ R*<0. (6) 
 
 1=0 
 
 Using this condition we see that if both the vectors (l, 0, 0,...,0) and 
 (-1,0,0, ...,0) are placed in W then no other vector may be left in V and 
 satisfy (6 ) . Thus only one vector is placed in W at each step prior to 
 the last one since to place two vectors in W would cut short the process 
 (make e less than necessary). At the last step two vectors may be placed 
 in W since no more vectors will be left In V. Hence e may not be greater 
 than n+1 in this case and the second half of the theorem is proved. 
 
(