UNIVERSITY OF ILLINOIS UBRARY AX URSAiNA-CHAMPA'GN ENGINEERING NOTICE: Return or renew all Library Materials! The Minimum Fee tor each Lost Book is $50.00. II III rt m «^._ JUN 2 7 IQftft The person charging this material is'res]»nsible for its return to the library from which it was withdrawn on or before the Latest Date stamped below. Theft, mutilation, and underlining of books are reasons for discipli- nary acttorkand may result in dismissal from p>e University. To renew c>ll Telephone Center, 333-8400 _ UNIvisifvlcfcllfiU^Qli^L^nlrS^ljyANA-CHAMPAIGN Digitized by the Internet Archive in 2012 with funding from University of Illinois Urbana-Champaign http://archive.org/details/quadraticassignm71purd ENGINEERING LIBRARY UNIVERSITY OF ILLINOIS URBANA, ILLINOIS UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN URBANA, ILLINOIS 61801 P /*-/-* CAC Document No. 71 THE QUADRATIC ASSIGNMENT PROBLEM By George B. Purdy February 1973 CAC Document No. 71 THE QUADRATIC ASSIGNMENT PROBLEM By George B. Purdy Center for Advanced Computation University of Illinois at Urbana-Champaign Urbana, Illinois 6l801 February 1973 This work was supported in part by the Advanced Research Projects Agency of the Department of Defense and was monitored by the U.S. Army Research Office-Durham under Contract No. DAHC0U-72-C-0001. uwttmm Limn* ABSTRACT In this report we discuss some seemingly reasonable approaches to the quadratic assignment problem, and we give some evidence from automata theory that the problem is insoluble. INTRODUCTION We present here a discussion of some reasonable approaches to the quadratic assignment problem and an incomplete proof of its insolubility. For the former we reduce the problem to an integer programming problem, for the latter we relate the problem to some very difficult graph problems that are believed to be non-computable in any reasonable sense. We shall define the quadratic assignment problem to be the following: Let A, B be given symmetric matrices with non-negative T elements. Determine the permutation matrix P such that tr(APBP ) is maximized, where tr(A) denotes the trace of A. Let t be a permutation on {1, ..., n}, and let P = {p. .} be ij the permutation matrix p.. = 6 ,.\.. Then ij tt(i)j -T-T, r,s Identity (l) is easily verified: (APB T P T ). . = 7 a. p b p. 4 iJ r s t ir rS ts Jt Bu_t P>. c = 5 (~\ e is unity if s = Tr(r) and zero otherwise; hence (APB T P T ). , = I a. b , . w . ij J; ir Tr(jjTr(r) and (l) follows. Thus we see that the quadratic assignment problem may also be formulated as follows: Given symmetric matrices A and B, with non-nega- tive elements, find the permutation ^ which maximizes ^ rs Tr(r)iT(s)" r,s tr(APB T P T ) = Y a b . w . (1). L rs Tr(r)n(s) v " Since the number of possible permutations is n! for n by n l8 ~ 32 matrices and since 20! = 2,4 x 10 and 30! = 2.6 x 10 it is not practical to solve the quadratic assignment problem by enumerating all possible sums unless n is very small. The quadratic assignment problem can be stated as a boolean (i.e., zero-one) integer programming problem - a fact which follows from theorems one and two. Theorem one is of interest in itself. It states that the quadratic assignment problem on matrices A and B can be expressed as a boolean quadratic programming problem with objective T function x Qx, where Q is B&A., the Kronecker product of A and B (also called tensor product). Theorem 1 Let A and B be n x n real matrices. Let Q = B©A, the Kronecker product of B and A. Then the maximum of m m tr (PAP B ) taken over all permutation matrices P is equal to the maximum of T x Q x taken over all boolean (zero-one) vectors x = (x_, * 2 ' •"' x n 2^ subject to the 2n linear constraints n . L . n(i-l)+j i=l and n V x / x.=l 1 5 i i n. .£ n(i-l)+j Proof Let A, B be n x n matrices. "Then tr (PAP T B T ) = I (PAP T B T ). . 11 T , T r ^, ir rs *st ti 1 rst - I P- a Px t.^ ; let . u , ir rs ts it irst X- (x, ..., X o)» x / • -r \ , = V . fn-r — 1 n c % n(i-l)+r *ir lor 1 < i,r < n. Let us write q = q [i,j], where Q = B ©A. Then q [n(i-l) + r, n(j-l) + s]'= b., a , 1 < i,j,r,s < n» and ij rs I p. a p b., .*• , *ir rs ts it irst = I x , . v q[n(i-l)+r, n(t-l)+s] x , . . * , n(i-l;+r n(t-l)+s irst = T x. q[i,j] x. = [ x. q. . x. = x Q x . 7: 1 J 7 m 1 IJ J - - 13 J iJ * 1J J The zero-one matrix P is a permutation matrix if and only if I P, . = 1 i < j < n i J and 5" P. , - 1 1 < i < n . 3 1J The resulting constraints on x are the ones claimed in the statement of the theorem. Theorem 2 Let Q be an n x n real matrix, let A be an m x n matrix, and let b_ be an m- rowed vector, where m > n. T Then the maximum of x Q x_ taken over all boolean (i.e. zero-one) vectors x subject to the constraint Ax = b is equal to the maximum of c_ . w taken over all boolean w subject to the constraint Dy = d, P 2 2 where £ is a ( 3n - 2n)-rowed vector, D is a (2n - 2n + m) x ( 3n - 2n) 2 matrix, and d is a (2n - 2n + m)-rowed vector. Proof In the function T -3 J x Q x = I x. q. . x. — — L i l. we wish to make the change of variable (2) y.j * x. Xj 1 < i,j < n. We need to introduce conditions on the y. . to force them to be of the form (2). Since all of our variables are zero-one, y. . is of the form (2; if and only if *u = y ii y jj (1 - i,j - n) - This is equivalent to 1 - y.. = (1 - y..)Q(l - y .) (1 < i,J ^ n), where (+) denotes boolean addition. Now boolean addition can be defined by adding variables and constraints The only zero-one value z lying between x + y aXid 2 ^ x + y) is x ©y- Hence the equations 2z = x + y + u z = x + y - v, where u and v are zero-one variables, have the unique solution z = x©y . We may now restate our original optimization problem as follows: Maximize ) q. . y. . over all zero-one variables y. . , u. . , v. . satisfying the constraints, n I a. . y . . = b. (l < i < m) and (3a) 2(1 - y ) = (1 - y ) + (1 - y ) + u *o 11 3 3 1 3 (3t) l - '« " (1 - Til) + (1 - r i3 ) - t„ 1 < i,3 < n i ? j We observe that u. . and v. . are not needed when i = j. The total number of equations (constraints) in the new problem is 2n - 2n + m, and they are independent if the original m equations were. The number of variables is 3n 2 - 2n. The problem is therefore of the type claimed in the statement of the theorem. Nov that theorems one and two have been proved, we are ready- to make some remarks about this approach to the quadratic assignment problem. If we relax the boolean requirement in the boolean quadratic programming problem of Theorem 1, then we get an ordinary QP (quadratic programming) problem, and it is well known [l] that good algorithms exist when Q is negative definite or negative indefinite; otherwise, it is doubtful whether there exist good algorithms. Indeed, it was shown in [2] that Hilbert's tenth problem is expressible as a quadra- tic integer programming problem. (This differs from the problem of Theorem 1 in that the boolean variables are replaced by integer varia- bles). Since Hilbert's tenth problem is known to be undecidable there can, of course, not be an algorithm. So, let us suppose that Q is negative definite. There is a theorem about Kronecker products which says that if A. (l < i < n) are the eigenvalues of A and y. (l < i < n) are the eigenvalues of B, then the eigenvalues of B8A are A.y. (l < i, j < n). Therefore, if A is negative definite so that all the A. are negative, and B is positive definite, so that all the y. are positive, then all the A.y. will be negative, and B8A will be X J negative definite, and the quadratic programming problem T „ max x Qx subject to /X/. n \.. = l H j f n n and J X/. 1 n j .=1 1 -: i < n .**_ n(i-l)+j i=l (u) can be solved for real numbers x. . Now we are interested in boolean 1 solutions (h) t and there is a theorem which says that the set of matrices P, P.. = x /. . v satisfying (k ) (called doubly stochastic matrices) is a convex set whose extreme points are precisely the n! permutation matrices of order n. Thus a solution x to the QP problem will be boolean only p if x is at a vertex of the polytope in n -dimensional space defined by (k). It is more typical, however, for such solutions to be interior points. The QP algorithms usually find a local maximum; the negative definiteness of Q implies that the local maximum is the global maximum. If, on the other hand, we arrange to make Q positive definite e.g., by choosing A and B to be positive definite, then any real solu- tion x to the QP problem is guaranteed to be an extreme point, and therefore, boolean. However, there are n! extreme points and there are no known algorithms better than branch and bound methods to decide which of the n! extreme points is the right one. The problem is indeed a difficult one! We now go on to discuss some evidence that the quadratic assignment problem is ill-posed, and that partial enumeration (also called implicit enumeration) is essentially the fastest algorithm for the problem as posed. Naturally such a method would be too slow for most purposes. 8 Stephen Cook [3] introduced the concept of polynomial time to measure the difficulty of problems.. A class of problem is solvable in polynomial time if there exists an algorithm •which solves any member of the class within P(n) steps on a Turing machine, where P(x) is a poly- nomial and n is some reasonable measure of the length of the problem. For example, in the quadratic assignment problem, n could be the number of entries in the matrices A and B; but then it may as well be the dimension of A and B. Stephen Cook has shown [h] that a number of ideal machines which resemble contemporary computers more than does a Turing machine can be used in place of a Turing machine in the defini- tion without changing the classes of problems that are solvable in po ly nomi al t ime . We say that one problem P is polynomial reducible to another problem P_ if the solvability in polynomial time of P p would imply the solvability in polynomial time of P . We now define a few problems. Such expressions as F = (p & ~ p ) V (p & p, ) will denote propositions and the p. are variables over the set {T, F} . The binary operators & and V obey the usual truth table, as does the unitary operator The expression F is a tautology if F takes the value T for all values of the p.. This particular F is in disjunctive normal (D.N.F.) form because it is of the form F = EJ R n V ... V R. , where 12 k R =S & S.„ & . . . & S and each S. . is of the form p or ~p . 1 il i2 im. ij r r The problem of determining tautologyhood of propositions in D.N.F. is probably not solvable in polynomial time, but it has never been proved. It is suspected that the tree search method, which requires 2 opera- k tions, where n is V m is closer to the theoretical limit than any i=l ± polynomial p(n). The major reason for thinking that D.N.F. tautologyhood cannot "be decided in polynomial time is a theorem by- Stephen Cook [ 3] which states that this would imply that every problem solvable on a non-deterministic Turing machine in polynomial time would be soluble on an ordinary Turing machine in polynomial time. A non-deterministic Turing machine (N.T.M. ) is able to make copies of itself whenever it has a decision to make, each copy taking a differ- ent course. Thus there may be 2 Turing machines after n steps. Practically any finite problem can be solved in polynomial time on an N.T.M. ; therefore D.N.F. tautologyhood must be about the most diffi- cult to determine, since D.N.F. solubility in polynomial time would imply solubility of almost everything else in polynomial time. We shall prove theorems showing that D.N.F. tautologyhood is polynomial reducible to the quadratic assignment problem! To do this, we must first introduce the subgraph isomorphism problem. A graph is a set of n elements called vertices together with a set of certain of the unordered pairs of these vertices, called edges. The complete graph of order n is the unique graph with n vertices and („) edges. Two graphs G and H are isomorphic if their vertices may be labeled in such a way that vl , . . . , u denote the vertices of G, v n , ..., v denote the vertices of H, and {u. , u.} is an edge of G if In ' 1 j and only if (v., v.} is an edge of H. It is an unsolved problem whether graph isomorphism can be determined in polynomial time. There are n! different ways of numbering the vertices. If a certain conjecture of Corneil [5] is true, then graph isomorphism can be determined in poly- nomial time. It has never been shown that D.N.F. tautologyhood is polynomial reducible to the graph isomorphism problem. For our purposes 10 therefore, we must seek a harder problem. Let G be a graph with vertices v , v , . .., v and edges e , ..., e . A graph H is a subgraph of G if H is obtained from G by possibly deleting some of G's vertices and deleting their incident edges and possibly deleting additional edges. The subgraph isomor- phism problem is to determine, given graphs G and H, whether H is isomorphic to some subgraph of G. Theorem 3 The problem of determining tautologyhood of propositions in disjunctive normal form (D.N.F.) is polynomial reducible to the complete subgraph isomorphism problem. Proof Let the given proposition be F = R VR V ... VR where R. = S. n & S.„ & ... & S. and the S. . are metavariables which 1 ll i2 im. ij take the letters p , p , ... or the symbols " p , " p 5 , ... as their values; here ~ denotes "not". m Let the graph G have vertices the m = / m. vertices v. . u l n i=l J for 1 < j < m. , 1 < i < k; let v. . be joined to v if i / r and if - l ~ ij rs S. . and S are not opposing - that is (S. ., S ) is not of the ij rs ij rs form (p , ~ p ) or (~ p , p ) for any t. (Graph theorists will observe that G is a k-partite graph). It is now obvious that F is a tautology if and only if it is not the case that the complete k-graph K_ is isomorphic to a subgraph of G. To see this, suppose first that F is not a tautology. Then there is an assignment of truth values to p , p , ... which falsifies F. This assignment must falsify every R., and consequently for every i, some S.. must be 11 false. Hence the subgraph of G having vertices v is a complete k- graph. We now suppose that G has a complete subgraph of order k. Since v.. and v are only joined when i is different from j, we may lj rs suppose that the vertices of the complete subgraph are v. . (l < i < k) We then assign truth values to p n , p^, ... so that S. . (l < i < k) 1 2 ij . i are falsified and we see that F is not a tautology. Theorem h The complete subgraph problem is polynomial reducible to the quadratic assignment problem. Proof Let G be a graph with n vertices and let k be a positive integer not exceeding n. We wish to determine whether G contains a complete subgraph of order k. Let A = {a. .} be the adjacency matrix for G; that is, if -^- J the vertices of G are v., , v^, .... v , then let a. . be 1 if v. and 1 2 n lj l v. are joined by an edge and otherwise. Thus A is a symmetric J n x n matrix with zeros on the diagonal. We then let B be the k x k adjacency matrix of a complete k-graph. That is, t J = J ,.=f llfi 1J L if i B be n x n. o and let B = Let M = max tr (APBP ) where the maximum is taken over all n x n P permutation matrices P. Then G has a complete subgraph of order k if and only if M = k(k-l), and the theorem is proved. 12 Theorems three and four show that the quadratic assignment problem is at least as hard as D.N.F. tautologyhood, and Cook's theorems tell us that that is very serious. One could discuss some ways of limiting the quadratic assignment problem to a special class of matrices so that D.N.F. tautologyhood is excluded. We shall merely show that limiting A and B to he so-called distance matrices does not exclude D.N.F. tautologies. Theorem 5 (Robert Ray III) Any quadratic assignment problem T max tr (APBP ) P where A, B are non-negative and symmetric is reducible to another quadratic assignment problem (QAP) max tr (CPDP T ) P where c, . is the distance between x. and x , d is the distance between ij i j ij y. and y . , and x., , . ... x , y_ , .... y are 2n points in (n-l)- ^ l j l n 1 n dimensional euclidean space. We call this the geometric quadratic assignment problem (GQAP). Consequently D.N.F. tautologyhood is poly- nomial reducible to GQAP. This result is due to Robert Ray III [6] and is part of his work on QUASCO, a heuristic program to solve the quadratic assignment problem. Proof Let E = {e. . }. e. . =1, the matrix of all ones. The following ij ij identifies are useful: Let a be any real number. J\ , T. (i) tr{(A+al)PBP } = tr(APBP ) + a tr(B) (ii) tr{AP(B+al)P T } = tr(APBP T ) + a tr(A) (iii) tr {(A+ E)PBP T } = tr(APBP T ) + a tr(EB) (iv) tr {AP(B+ E)P T }= tr(APBP T ) + a tr(AE) 13 Now let A and B be as In the statement of the theorem. It is well known that there exist constants a, 6 such that C = A + aE - al and D = B + BE - $1 have the properties claimed and the identities T (i) - (iv) show that max tr (CPDP } will occur at the same P as T max tr (APBP ), proving our theorem. Remark There is one more useful identity (Robert Ray, III [6]) (v) tr{(aE-A)PBP T } = a tr (EB) - tr (APBP T ). If we apply this with a = max a. . and put A = aE-A, then we get max tr (APBP T ) = max{a tr(EB) - tr (APBP T )} or max tr (APBP T ) = a tr(EB) - min {tr (APBP T )}. P P Hence every maximizing QAP is equivalent to a minimizing QAP (and vice versa). Ik References [l] Nonlinear and Dynamic Programming , G. Hadley, 196U, Addi son-Wesley. [2] "There cannot be any algorithm for integer programming with quadratic constraints," Mimeographed notes by Robert G. Jeroslow, University of Minnesota. Received December 1971. [3] Stephen Cook, Proceedings of the Third Annual ACM Symposium on the Theory of Computing, Shaker Heights, Ohio, 1970, p. 151-158 [h\ Stephen Cook and Robert A. Reckhow, "Diagonal Theorems for Random Access Machines," University of Toronto, Computer Science Department report number U2, June 1972. [5] D. G. Corneil, "Graph Isomorphism," University of Toronto, Computer Science Department report number 18, April 1970. [6] Robert Ray, III, oral communication. UNCLASSIFIED S«>curityClasaificatiori DOCUMENT CONTROL DATA R&D (Security claatltleatlon ol till*, body of rtiWcl and Indaatng annotation muat ba antatad whan tha overall report It etaaalllad) I. ONIGIN1TINS ACTIVITY (Corpormf exithot) Center for Advanced Computation University of Illinois at Urbana-Champaign Ilrbana. Tllinoi s 6l801 i». »EPO«T IECUAI IY CLASSI FIC * TIO» UNCLASSIFIED 2b. GROUP S. REPORT TITLE The Quadratic Assignment Problem 4. DESCRIPTIVE MOTH (Type at rapatt and rncluelre dataa) Research Report 8 AuTHOPim(Flfiliw», middle Initial, laml nama) George B. Purdy «. REPORT DATE February 1973 7a. TOTAL NO. Or PACES 19 7b. NO. OF REFS 6 Sa. CONTRACT OR GRANT NO. DAHCOU 72-C-OOOl b. PROJECT NO. ARPA Order No. 1899 am. ORIGINATOR'S REPORT NUUtERUI CAC Document No. 71 •b. OTHER REPORT NOI1I (Any other numbers that nay ba aaalgnad thla report) 10. DISTRIBUTION STATEMENT Copies may be requested from the address given in (1) above. II. SUPPLEMENTARY NOTES 12. SPONSORING MILITARY ACTIVITY U. S. Army Research Office - Durham Duke Station, Durham, North Carolina IS. ABSTRACT In this report we discuss some seemingly reasonable approaches to the quadratic assignment problem, and we give some evidence from automata theory that the problem is insoluble. DD ,"7.1473 UNCLASSIFIED Security Classification UNCLASSIFIED Security Classification kev WONO* Metatheory Graph Theory Non-Linear Programming HOLE «T HOLE WT ROLE WT UNCLASSIFIED Security Classification