-X I B RAHY OF THE U N IVLRSITY Of ILLINOIS 0.84 I£6r no. 257-264 cop. 2 CENTRAL CIRCULATION AND BOOKSTACKS The person borrowing this material is re- sponsible for its renewal or return before the Latest Date stamped below. You may be charged a minimum fee of $75.00 for each non-returned or lost item. Theft, mutilation, or defacement of library materials can be causes for student disciplinary action. All materials owned by the University of Illinois library are the property of the State of Illinois and are protected by Article 16B of Illinois Criminal Law and Procedure. TO RENEW, CALL (217) 333-8400. University of Illinois Library at Urbana-Champaign JUN 2 8 1999 When renewing by phone, write new due date below previous due date. L162 Digitized by the Internet Archive in 2013 http://archive.org/details/computationalexp259baug ■ Report No, 2 S^? CCstt \ * *1 T' ^- / COMPUTATIONAL EXPERI i ■.: CM ALL- INTEGER, BINARY VARIABLE, INTEGER PROGRAMMING PROBLEMS USING GOMORY'S ALL- INTEGER ALGORITHM by Charles R. Baugh Toshihide Ibaraki Saburo Muroga April k, 1968 Report No, 259 COMPUTATIONAL EXPERIENCE IN ALL-INTEGER, BINARY VARIABLE, INTEGER PROGRAMMING PROBLEMS USING GOMORY'S ALL- INTEGER ALGORITHM by Charles R , Baugh loshihide Ibaraki Saburo Muroga April k, 1968 Department cf Computer Science University of Illinois Urbana, Illinois 61801 ABSTRACT Our computational experience with Gomory's algorithm for the integer linear programming problem of synthesis of optimum network with NOR gates is presented. The problem is briefly described and accompanied with statistics such as the size and density of the coefficient matrix. Upon successful solution of problems with 90 variables and 2^0 rows, the effect of constraint orderings and adding additional inequalities was investigated. The difference in convergence of two of Gomory's pivot selection rules is noted. Also the behavior in convergence of feasible versus non-feasible problems is demonstrated. Rn The convergence rate is conjectured to be A'10 where n is the number of variables of a switching functi on to be synthesized and A, B are constant coefficients. This rate is compared to an exhaustive method developed by Hellerman to solve the same logical design problem. TABLE OF CONTENTS Page 1. INTRODUCTION 1 2. PROBLEM STATEMENT 2 3. OUTLINE OF RESULTS 7 k. FEASIBLE AND NON-FEASIBLE 11 5. ORDER OF INEQUALITIES . 12 6. ADDITIONAL CONSTRAINTS Ik 7. FIXING THE VALUES OF A VARIABLE 17 8. RELATIONSHIP BETWEEN THE NUMBER OF ITERATIONS AND VARIABLES 18 9. COMPARISON OF HELLERMAN'S ALGORITHM AND GOMORY'S. . 20 10. CONCLUSION 21 REFERENCES 22 1 . INTRODUCTION Our computational experiments were performed using Gomory's all- integer integer linear programming method even though our variables are restricted to be 1 or 0. The problems are unusual in that the number of inequalities is larger than the number of variables . This is just the opposite of the vast majority of the other published computational reports. Also the size of the coefficient matrix is much larger - up to 90 variables and 2^0 constraints. The problem formulation is derived from the design of logic circuits in digital computers. Specifically it is concerned with the optimum synthesis of a NOR element network complicated by considering fan-in and fan-out restrictions. In the next section the set of inequalities will be stated and briefly explained. Previous publications have demonstrated the very erratic be- havior in convergence. Some problems have been solved in only a few iterations while others needed several hundred thousand iterations for a solution. ' The convergence rate of Gomory's method seems to be highly dependent upon the characteristic of the constraints of each particular problem. We will examine the rate of convergence and the effect that certain factors have upon it. Some of the factors we will consider are the order of the inequalities, the addition of other constraints, and the difference between feasible and non-feasible solutions. 2. PROBLEM STATEM Fig. 1: Feed Forward NOR Circuit For the Boolean Function f(x, , x , x ) Our integer problem arises from the attempt to synthesize a Boolean function f of three variables, x. , x , and x„ with a feed forward NOR element circuit as is shown in figure 1. In the figure we denote the weight of an input variable x. to the j-th NOR element by w. and the weight of the output from the k-tn NOR element to the e-th element by a, . The logical design problem is to determine which a' s and w's are 1 (connected) and which are zero (disconnected) by using Gomory's all-integer algorithm. It is possible that the given number of elements R is in- sufficient for a particular function, i.e. the problem is infeasible. Therefore we may have a considerable number of infeasible problems if R is small . For a detail description of the general feed forward network synthesis formulation of which this problem is a special case see reference (8). ii order to mods..! the clrcuil with Linear .in [ualil man; variables must be I tl original w's and ex's. The for tb Lemenl t Forv ard bwork ar • ' i i -i : ;, •• ' | ; cr s , v. , c, and f:'ol. I ows : For the k-th element (k - I. 2, ..., R-l) A: u: Z w. k x. (j) + u P (j) + L (a., -a./^) _k — 1 1 k ._ lk ik B • -1 :> -Z v. k x. (j) - U P ( J } - ¥ (a.. - a. v ( ' ]) ) k -..ii k . , ik ik — 1=1 i=l C • > a , - a n ^ - P ^ k — ek ek e (i) >-a . - a . wJ ' — eK ek 1 > P (J) +a , (J) for e * 1 ' ■•" k " X — e ek 3 k k-.l D. : 3 - w. L a., — 1=1 1=1 R 'i > V, cy ~ i=k+l kl for j = 1, 2, ..., 8 For the Rth element E: > Z v. R x. U) + E (a. R - a. R (j) ) 1=1 " ! for f(r. (j) , x (j) , J J) ) =1 -1 > i w. R x. ( J } - ¥ (« -a. R Q) ) - . , 1 1 . , lR iR / . V / . N ( . .. 18,1 ^ for f(x n {j U, (j) . x> ]j ) = C > a _ - a n ^' - P ^' - eR eR e > a - a )? U — eR eR 1 ' "2 3 1 > P ^ J + a ,, ■'' for e = 1, 2 R-l — e e R * For a detailed derivation of these equations see reference (8) 3 R 3 > Z w . + L a . „ i=l i=j for J = 1, 2, ..., 8 For all R elements G : all w k ' s, a ' s, a. ^^'s. andP/^'s <1 - ij ke i - These seven groups of constraints A-F compose the complete set of inequalities. The constant U is a sufficiently large positive value so that if P, = the j -th inequality of A is always satisfied and if -(j) . ,-_ (j) . (o) „ (j), = 1 the j -th inequality of B is always satisfied. The input vector X [x ', x n ' , x " ' ) for the three variable Boolean function f(x) \1 ' 2 ' 3 assumes all eight possible combinations of 1 and 0. Inequalities D and F make the further restrictions that no NOR element can have more than 3 inputs or more than 3 outputs. The fan-in, fan-out constraints (inequalities D and F) arise from practical engineering restrictions on the actual electronic circuits which are used to realize the NOR elements. Therefore cur all -integer integer linear programming problem is: 3 i :: ;r :> . -,: 2-1 R a ek k i k=e+l under the con c Lnts A through G By minimizing g we are minimizing the total number of connections. The procedure for determining a network for a given Boolean function is out- lined in figure 2. Fig. 2: Flow Chart Of Procedure of Synthesizing A Boolean Function In this manner we will obtain a network with the fewest number of NOR elements and with minimum connections. The number of variables, inequalities, entries, and non-zero entries in the coefficient matrix all increase with the number of elements R. Figure 3 shows this growth. Slack variables are not included. Fig. 3: Characteristics Of The Coefficient Matrix R Matrix Size Coefficients Constraints Variables Total No. Non-zero* "^Non-zero 1 11 3 33 18 55 = 50 2 64 23 1472 183 12.40 3 142 52 7380 447 6.08 4 245 90 22,040 810 3.68 5 374 137 51,250 1277 2A9 6 528 193 101, 800 1836 1.81 7 707 258 182,200 2487 1.37 This number is for the Boolean function f(x)=l. See the E inequal- ities for the effect of other Boolean functions. 3. OUTLINE OF RESULTS The algorithm was programmed in assembly language NICAP for the ILLIAC II computer at the Univer; : nois. The computer has a 1.75 microsecond core cycle time, an 8K - 52 bi t word memory, and the facility to operate on 13 bit quarter words. All the coefficient matrix entries were stored in the 13 bit quarter words. Therefore whenever the algorithm generates a number greater in absolute value than ^+096, the computation terminates with overflow, Gomory in proposing his all -integer integer program suggested several, variations to the basic algorithm. We used the variation which chooses the pivot row which minimizes the lexiograpbic rank of the pivot column. In order to ascertain the factors important in convergence of the algorithm, we solved several problems. These same problems were then reformulated by changing the order of inequalities, by adding additional, constraints, by changing the value of U, etc. In our tests the number of problems run was not large enough to guarantee statistical, accuracy. This is especially true since convergence if very unpredictable. However the results do demonstrate the qualitative tendency of integer programming. The integer linear programming formulation with R = M models a feed-forward network of M NOP elements If a Boolean function is realizable with M or fewer elements, the corresponding integer linear program has an optimal feasible solution. But if a function requires more than M elements, the problem is not feasible. The set of Boolean functions upon which we performed our exper- (1?) iment.p is shown in figure 4, Hellerman's " network numbering system was used to uniquely identify each function and will be used throughout this report. ( Fig. k: Complete Set of Boolean Test Functions Boolean Function Hellerman netwprK number Number of gates in network Objective function value a J+D 1 1 ab 5D 1 2 a ^ b 6E 2 3 ab 7D 2 3 a v b 8D 3 k ab 9D 3 k aBc 1 1 3 a ^ b v c 2 2 k ab ^ ac 3 2 k aBc l+ 2 k ab ^ c 5 3 5 a ^ b v c 6 3 5 aB v ac 7 3 5 ac v be 8 3 5 ab v c 9 3 6 ab ^ c 10 3 5 a s ^ B 10D 4 5 ab s ' aB 11D 4 8 ab v c 11 4 6 a ^ B ^ c 12 4 6 a v Be 13 4 6 aB v ac Ik 4 6 ac ^ be 15 k 6 ab v ac 16 k 7 abc 17 k 6 ab v ac v be 18 k 9 a v be 19 k 7 ab ^ be 20 k 8 abc v abc 21 k 9 aB v be v ac 22 h 9 abc v be 23 k 9 aB v ab v c 24 k 10 abc ^ ac ^ be 25 k 10 The density of non-zero coefficients in the coefficient matrix after each iteration is also of interest. The density of the initial matrix is low (see figure 3)- In a seemingly typical problem the density starts at 5$ and increases gradually until it reaches about 15$. It oscillates between 10$ and 20$ until convergence at which time it drops down to about 10$, First we noticed the size of U in equations A and B con - siderably effects the occurrance of overflow. Generally the larger the value of U the more often overflow was encountered. By reducing the value of U from 9 down to h we were able to prevent overflow from occurring as early. This is especially true for problems which require a large number of iterations, (1) Gomory also suggested another approach which he called the row combination method. Using this method on a set of Boolean functions for R = 3- The comparison between the minimum rank method and the minimum rank method appended with the row combination technique is seen in figure 5, Fig. 5 : Evaluation of Adding Row Combination Row combination Without row combination Ave, number of iter, 62,1 6-,6 Time per iteration 46^ msecs 166 rrsecs $ of problems with overflow 17 . 6$ 0.0$ Thus in considering the increased computation for each iteration, the row combining technique should be disregarded. 10 h. FEASIBLE AND NON-FEASIBLE For example, when a Boolean function realizable with R = k {h NOR elements) is tried with th< R = 3 formulation, the resulting integer linear program is not feasible. In Figure 6 the average number of iterations for all Boolean test functions and for three different orderings of the constraints are shown, Fig. 6: Average No, of Iterations For 3 Different Orderings of the Inequalities „ 1 2 3 Feasible 72,5 66.0 66.6 Non- feasible 85,5 , ., j 83*3 98,2 , 1 The difference in the average number of iterations for any case is not too great. However in the non-feasible cases the number of iterations deviates from the average much more than in the feasible case, For example in the ordering of column 3 of figure 6 all non-feasible functions required less than 92 iterations ex- ept for #17 and fflQ of figure k which require k6l and 296 iterations respectively, The convergence varies over a much larger number of iterations in non -feasible problems than in feasible problems, Figure 6 also demonstrates ' feasible problems are much more difficult to solve than the feasible problems, See figure 7 for details of the constraint ordering 11 5. ORDER OF INEQUALITIES Gomory's method with the incorporation of the lexiographic (1) row rank variation is sensitive to the order of inequalities since the lexiographic ordering is changed by altering the order of the constraints. Figure 7 displays the average number of iterations for all the functions of figure h for four different constraint orderings . The objective function is denoted by O.F. and slack variables always Fig. 7: Average Number of Iterations Under k Different Constraint Orderings 1 2 3 k O.F, O.F. O.F. O.F. G E E *L E B i G B l B i B 2 B i A 2 B 2 D 2 B 2 B 2 D 2 F D 2 °2 F A 3 F E A i C 3 A 3 A 3 ** A 2 C 3 C 3 C 2 C 2 A 2 G A 3 A l C 2 B 2 °3 G A l F 2 Feasible 72.5 66.0 66.6 50.0 Non-feasible 85.5 83-3 98.2 118.0 12 appear after all other constraints. It should be noted that B } B , and E are the only constraints which initially have negative entries in the constant column. It is difficult to draw any conclusive results from figure 7 • However, it seems best to place the E constraints first since we have more non-feasible than feasible problems. These inequalities are the only ones which change according to the Boolean function which is being tested. Since the rest of the constraints remain the same regardless of the Boolean function being solved, the E constraints are likely the most important. Therefore we place them at the top. One of the reasonable orderings may be to put the more important inequalities at the upper part of the tableau. However, strictly speaking we don't have any measure of the importance of an inequality, One intuitive and reasonably successful measure is to count the number of non-zero entries in an inequality - the greater the number of coefficients the more important the constraint. 13 6. ADDITIONAL. CONSTRAINTS Another technique for increasing the speed of convergence is to incorporate additional constraints which exclude unnecessary solutions without loss_of generality. For example the two networks; x realize the same Boolean function. But it is not important to get both solutions. Either one will be acceptable. Therefore additional constraints should be added to exclude either one but obviously not both. By considering other properties of a h element NOR network (R = k-) we establish the following constraints; a 2^ 3 L. < * lk -* 2k < a 2 ^ 2k < 1 a 2^" a 34 - - 1 ,111, ■(w 1 +w 2 +w j 2 3 (w, ^+w^ ' 3 3 3 1 w 1 +w 2 « 1+4 4 -lw 1 +w 2 +w •° 3 u < -1 a 12 -*a 13 -Kx lh < 2 a l2 4a l3 ~ a 2k < 1 a i2" a i3 **2k < 1 "12^13^23^24 < 2 +a 12 ) < -1 ^12 < -1 "^TQ^O V < -1 + a 2k' KX ^h' < -I -(rt +a. +rv , ) 4 12 13 Ik- < -1 (au.,+au.. ) < -1 lA Note that some of the inequalities require that the network consist of exactly 4 elements while others require that the Boolean function have exactly 3 input variables. The additional constraints for the R = 3 network are obtained by neglecting the 1st element and relabeling the re- maining elements 2, 3 and 4 as 1, 2, and 3 respectively. Figure 8 shows the effect of adding these constraints to the first 3 row orderings of figure 7 for the R = 3 network and for the six Boolean functions #5 -#10. Figure 9 demonstrates the improvement on the R = 4 network for all the 4 gate realizable Boolean functions, Without the additional constraints all the R = 4 network problems either did not converge after about 1200 iterations or generated an overflow. Al- though all the infeasible R = 4 problems tested with additional constraints did not converge after 1200 iterations, the additional constraints were extremely successful, for feasible problems in the R = 4 case. Fig. 8: Effectiveness of Additional Constraints on R = 3 Network, Average Number of Iterations Row Ordering 1 2 3 without with without with without with Feasible 87,5 68.2 103.7 70.1 87.7 68.1 Non-feasible 98.2 129.8 I.45.8 137 = 4 83»3 146.2 , 15 Fig. 9 : Convergence with Additional Constraints on R = k Network without with °lo problems with no convergence * 35.3/o °Jo problems with overflow 11.8$ Average ff= iter, for sol. 355.6 From the results we notice the fact that the additional constraints facilitate convergence if the problem is feasible but make convergence worse if it is non-feasible,, Furthermore, the incorporation of the added inequalities is much more effective with the R = k problems than the R = 3 problems . A few functions were tried, However all exceeded the preassigned bound of 1200 iterations . 16 7. FIXING THE VALUES OF A VARIABLE We tried splitting the given problem into two smaller problems by fixing a particular variable to 1 in one and to in the other. By comparing the results, we can pick the optimal network. By picking a, _ in "the R = 3 network and by using the formulation of column h of figure 7 we obtained the results of figure 10. Fig. 10: Average Number of Iterations For All Test Functions oc not preset a 13 = o "- a 13 = 1 Feasible 71.0 20.0 18,5 Non-feasible 81.0 32.9 55^5 Generally it is difficult to judge whether fixing a variable speeds up the computation. However, it may be worthwhile if the right choice of a variable is made. 17 8. RELATIONSHIP BETWEEN THE NUMBER OF ITERATIONS AND VARIABLES In order to determine the increase in the number of iterations we preset some randomly picked variables to their solution values. (Solution of our problem is already known by using another approach. ) This equivalently shrinks the size of integer linear programming problems, but the general characteristics of the problem may not change very much. The test was performed on one particular Boolean function ac v be with the following order of inequalities OF A E C 3 F G Each size of the coefficient matrix was run 5 times with a different set of variables being preset. The average number of iterations is plotted in figure 11. Fig. 11: Number of Iterations l(n) as a Function of the Number of Variables n For the Boolean Function ac ^ toe. 60 In I(n) 5 ■ (12,5.6 o 10' 20' 30' n w :h8,5k.o] 50 1 i Q The curve of figure 11 demonstrates that the increase in the number of iterations grows exponentially and can he expressed as I(n) B 2.5 x 10" 02T7n for the Boolean function ac ^ be. Our conjecture is that the number of iterations is generally "Rn A* 10 where A and B are constants which depend upon the particular type of integer linear programming problem. 19 9. COMPARISON OF HELLERMAN "S ALGORITHM AND GOMORY'S Hellerman in reference 12 determined the optimum NOR circuit for all Boolean functions f(x,, Xp, x ) by an exhaustive method,, He generated all possible circuits and chose the best one for each function. It is interesting to compare the integer programming approach using Gomory's method with Hellerman' s to determine if it is better. For the 3 and k element formulation (R = 3> M the computational efficiency of Gomory's method is inferior to Hellerman 's approach, The number of 2 CR iterations in Hellerman 's method increases as T-10 with the number of elements R. In the previous section we showed that Gomory's method appar- ently increases as A-10 . Therefore it is unlikely that Gomory's algorithm would be better than Hellerman 's exhaustive method since n 2 increases as R . However, this cannot be a definite conclusion since we do not have sufficient computational experience. Also some techniques to reduce the number of iterations for Gomory's method may be developed in the future which could make it more effective tha.n Hellerman' s approach. 20 10. CONCLUSIONS Using Gomory's algorithm ve duci es i'ully solved a few rather large 0-1 value all integer problems of 90 variables and 2^0 constraints Gomory's work was followed by the implicit enumeration methods of Balas^ , GeoffriorT ' and others^ ' ' . These algorithms further restrict the problem by requiring all variables to be 1 or 0. The number of iterations for implicit enumerations reportedly k (ID grow as n . Since our problems are 0-1 problems, we are now experimenting with these implicit enumeration methods. We are very encouraged with the initial results and feel these methods may be better for our problems. Our results will be published. 21 (1) R.E. Gomory, ''An all-integer integer programming algorithm", Industrial Scheduling edited by J.R. Muth and G.L. Thompson, Prentice-Hall, I963. (2) E. Balas, "An additive algorithm for solving linear programs with zero- one variables", Operations Research , vol. 13 ; > No. k, pp 517-5^, July - August, 1965. (3) F. Glover, "A multiple phase-dual algorithm for the zero-one integer programming problem", Operations Research , vol. 13, No, 6, PP 879~919^ November - December, 1965° (k) A.M. Geoff rion, "integer programming by implicit enumeration and Balas' method", SIAM Review , vol. 9, No. 2, pp 178-I9O, April, I967. (5) R.J. Freeman, "Computational experience with a 'Balasian' integer programming algorithm", Operations Research , vol. Ik, No. 5> PP 93^-9^1^ September - October, I966. (6) B, Fleisehmann, "Computational experience with the algorithm of Balas", Operations Research , vol. 15, No. 1, pp 153"155> January - February, 1967° (7) C.C. Petersen, "Computational experience with variants of the Balas algorithm applied to the selection of R and D projects". Manage - ment Science , vol, 13, No. 9, pp 736-7^5, May, 1967. (8) S. Muroga, "Threshold logic", lecture notes for EE ^97 and EE i+98, Department of Computer Science, University of Illinois, I965-I966. (9) J. Haldi and L.M. Isaacson, "A computer code for integer solutions to linear programs'', Operations Research , vol. 1.3, No. 6, pp 9^+6 " 959> November - December, I965 . 22 (10 ) G.T. Martin, "Directed network models for discrete programs '. Paper presented at the International Symposium on Mathematical Programming, London School of Economics, July 196 J i-. (11) A.M. Geoff rion, "Recent computational experience with three classes of integer linear programs", Rand Report P-3&99; October, 1967. (12) L. Hellerman, "A catalog of three -variable OR- invert and AND- invert logical circuits", IEEE Transactions on Electronic Computers , vol EC-12, pp. 198-223, June 1963. (13) F.T. Chen, "Code for integer linear programming problem for Gomory's algorithm II", Department of Computer Science, University of Illinois, I968. 23 JUN 2 <. 1966 JUN 2 1969