MySQL Web Server, Tomcat, Jsp, Struts Hosting Blog

4.7 ANSWERS TO EXERCISES 657 10. Form y = X(1 + a15 + a& + . . . )l O = ~(1 f CIZ + c2×2 + . .) by means of Eq. (9); then revert the latter series. (See the remarks following Eq. 1.2.11.3-11.) 11. Set W0 + Uo, and set (Tk, Wk) + (Vk,O) for 1 5 Ic < N. Then for n = 1, 2I …t N, do the following: Set Wj +-Wi + UnT3 for n 2 j 2 N; and then set q +-T~-~VI +. . . + TnV,-, for j = N, N -1, . . . , n + 1. Here T(z) represents V(z) . An on-line power series algorithm for this problem, analogous to Algorithm T, could be constructed, but it would require about N2/2 storage locations. There is also an on-line algorithm that solves this exercise and needs only O(N) storage locations: We may assume that VI = 1, if Uk is replaced by UkVf and Vk is replaced by Vk/Vl for all k. Then we may revert V(z) by Algorithm L, using its output as input to the algorithm of exercise 8 with G1 = Ul, G2 = U2, etc., thus computing U((V- )-l(z)) -UO. Brent and Kung have constructed several algorithms that are asymptotically faster. For example, we can evaluate U(Z) for 3: = V(z) by a slight variant of exercise 4.6.4- 42(c), doing about 2fi chain multiplications of cost M(n) and about n parameter multiplications of cost n, where M(n) is the number of operations needed to multiply power series to order n; the total time is therefore O(fiM(n) + n ) = O(n ). A still faster method can be based on the identity U(Vo(z) + z Vl(z)) = U(Vo(z)) + z u (vo(z))vl(z)+z u (v,(z))Vl(z) /2!+. . . , extending to about n/m terms, where

656 ANSWERS TO EXERCISES 4.7 2. We have Vt+ w, = vo u, -(V, Wo)(V, -1%) -(V, Wl)(VJ- IL,) -. . * - (V, W,-,)(I@ ,). Thus, we can start by replacing (Uj, V,) by (ViUj, Vi- V,) for j 2 1, then set W, +-U, -xOck<,, wkvn-k for n 2 0, finally replace Wj by W,lVo 3+1 for j > 0 Similar techniques are possible in connection with other algorithms -. in this section. 3. Yes. When LY = 0, it is easy to prove by induction that WI = Wz = . .. = 0. When cy = 1, we find W, = V,, by the cute identity k -(n -k) &l/n-k = v,v,. n 4. If W(z) = ev( ), then W (z) = V (z)W(z); we find WO = 1, and wn = c ;vkw,-k, for n 2 1. l

4.7 ANSWERS TO EXERCISES 655 62. The rank is 3, by the method of proof in Theorem W with P = (i i). The border rank cannot be 1, since we cannot have al(zl)bl(u)cl(u) = al(u)b~(zl)cz(u) = ud and al(~)bz(u)ci(u) E ai(u)bi(u)cz(u) G 0 (modulo &l). The border rank is 2 because of the realization (L A), ( f T), (A -A). 63. (a) Let the elements of T(m, n, s) and T(A4, N, S) be denoted by t(2,J~)(3,k~)(k,z~) and T(I,J~)(J,K~)(K,I~), respectively. Each element ~I,J,)(J,K,)(K,I,) of the direct product, where 1 = (6 I), J = (j, J), and K = (b K), is equal to t(,,j,)(j,k,)(k,zl)~(~,~/)(J,KI)(K,r ) by definition, so it is 1 iff I = I and J = J and K = K. (b) We have M(mns) 5 r3, since T(mns, mns, mns) = T(m, n, s) @ T(n, s, m) @ T(s, m, n). If M(P) 5 R we have A4(Ph) 5 Rh for all h, and it follows that M(N) 5 J,f(pk= 1) 2 RhP Nl 2 RN1 gR/logP, [This result appears in Pan s 1972 paper.] (c) We have Md(mns) < r3 for some d, where ?&(n) = rankd(T(n,n,n)). If l&(P) 2 R we have ?&d(Ph) 5 Rh for all h, and the stated formula follows since WPh) 5 2 >Rh by exercise 61. In an infinite field we save a factor of log N. ( hdt2 [This result is due to Bini and Schonhage, 1979.1 (d) Let P = mns; we can perform p 3h independent Ph X Ph matrix multi- plications with at most (hdzfz)p3hr3h noncommutative scalar multiplications. Reduce M(P2h) to A4(Ph) matrix multiplications of size Ph x Ph; thus we have M(Pzh) 5 (hdt2)~3hM(Ph)(l f O(~/T)~~). Iterating this recurrence gives 2 M(N) = O(N p(m,n,s,r)) exp(O(log log n)2). 64. (a) Let a = xij, A = U&z, b = yjk, B = Yt3, c = u&z, C = zjk, so that CZ = O(u ) can be eliminated. [When m = s = 7 and n = 1, this gives M(N) = O(N2e6).] (b) Take cyi = s -1, ~2 = –oc , t23=Q4=-1,C25=1,(-Y6=S-1, and d > 4. [We assume that o-l exists in the field.] (c) Taking the direct product of T(m, n, 2s) $ T(2n, s, m) $ T(s, 2m, n) with itself 3h times gives a tensor whose border rank is at most (2(m + l)n(s + 2))3h. This tensor is the direct sum of 33h terms of the form T(m (2n)3sk, n2s3(2m)k, (2s)%m3nk) where i +j + k = 3h, and (3h)!/h!3 of these have i = j = Ic = h. Thus if we let P = (2mns)h and p = (3h)!/h!3, the border rank of pT(P, P, P) is at most pr, where r = (2(m + l)n(s + 2))3h/p. Exercise 63(d) now implies that M(N) = O(N P(P,PZP, )fe) for all 6 > 0; here P and r are functions of h. We complete the proof by letting h be large in p(P, P, I , r) = log r/log P = (3h log 2(m + l)n(s + 2) -3h log 3 + O(log h))/(3h log Zmns), which equals ,@m, n, 2s, $(m + l)n(s + 2)) + O((log h)/h). [The best value is obtained for m = 5, n = 1, s = 11, p = 31og,,, 52 < 2.522.1 SECTION 4.7 1. Find the first nonzero coefficient V,, as in (4), and divide both U(z) and V(z) by zm (shifting the coefficients m places to the left). The quotient will be a power series iff UO = . . . = U,-1 = 0.

654 ANSWERS TO EXERCISES 4.6.4 It is easy to verify that at most n extra bits of precision are needed for the intermediate variables in this calculation; i.e., if ]si] 2 M for 0 2 i < 2n at the beginning of the algorithm, then all of the 5 and X variables will be bounded by 2 M throughout. Algorithm N performs A, addition-subtractions, D, halvings, and M, multiplica- tions, where A1 = 5, D1 = 0, MI = 3; for n > 1 we have A, = Ln/2]2 + + 2 n z +1A,+, + ([n/2] + 1)2 + + 2 , D, = 2 n z +1DLn,zJ + ([n/2] + 1)2 +l, and M, = 21n 2J+1 ML+J. The solutions are A, = 11. 2n- +r1gn1 -3.2 + 6. 2nS,, D, = 4. 2 - +bn1 -2.2 + 2. 2 S,, M, = 3 . 2n-1+r gnl; here S, satisfies the recurrence Sr = 0, S, = 2Sl,/zl+ [n/2], and it is not difficult to prove the inequalities +n[lgnl 2 S, 5 inlgn + n. Algorithm C does approximately the same amount of work as Algorithm N. It would be interesting to find a simpler way to carry out the additions and subtractions in step N3 (and the reverse operations in N5), perhaps analogous to Yates s method. The operation Xi + Xi + wkXl sketched above can be done with a procedure that generalizes the data-rotation algorithm of Fletcher and Silver in CACM 9 (1966), 326, but there might be a better way. 60. (a) In Cl, for example, we can group all terms having a common value of j and k into a single trilinear term; this gives y2 trilinear terms when (j, k) E E xE, plus y2 when (j, k) E E X0 and y2 when (j, k) E OX E. When 3 = k we can also include -xjj y3j z?j in Cl, free of charge. [In the case n = 10, the method multiplies 10 by 10 matrices with 710 noncommutative multiplications; this is fewer than required by any other known method, although Winograd s scheme (35) uses only 600 when commutativity is allowed.] (b) Here we simply let S be all the indices (i, j, k) of one problem, S the indices of the other. [When m = n = s = 10, the result is quite surprising: We can multiply two separate 10 X 10 matrices with 1300 noncommutative multiplications, while no scheme is known that would multiply each of them with 650.1 (c) Corresponding to the left-hand side of the stated identity we get the terms summed over (i, j, k) E S and 0 5 E, 5, n 5 1, so we get all the trilinear terms of the form Xz3 yjk ski except when [i/2] = [j/2] = [k/2]; h owever, these missing terms can all be included in Cl, Cz, or Es. The sum Cr turns out to include terms of the form ~i+~,j+< yi+v,j+, times some sum of z s, so it contributes 8v2 terms to the trilinear realization; and CZ, Cs are similar. To verify that the aBC terms cancel out, note that they are z(-l)Sfq xz+r,j+c Yk+c,i+c Zj+c,k+f, so 7 = 1 cancels with 7 = 0. [This technique leads to asymptotic improvements over Strassen s method whenever ijn +6n2 -4n < nlg7, namely when 36 5 n 5 184, and it was the first construction known to break the lg 7 barrier. Reference: SIAM J. Computing 9 (1980), 321-342.1 61. (a) Replace oij(U) by uail(~). (b) Let Q(U) = a+~~, etc., in a polynomial realization of length r = rankd(tijk). Then tijk = xCl+Y+O=d cl

4.6.4 ANSWERS TO EXERCISES 653 (~0,. . , yzn-1). The algorithms are presented in unoptimized form, for brevity and ease in exposition; the reader who implements them will notice that many things can be streamlined. Cl. [Test for simple case.] If 72 = 1, set so + ~oYo+mYl, 21 + (2o+51)(Yo+Yl)-zo, and terminate. Otherwise set m + 2+-l. C2. [Remainderize.] For 0 2 Ic < m, set (5k, xm+k) c (5k + ZnL+k, Xk -xm+k) and (Yk, ymfk) + (yk + ym+k, yk -ymfk). (Now We have X(U) mod (2~ - 1) = x0 f.. . + xm–lum-l and ~(~1)mod (urn f 1) = xm + … + ~2~~1; we will compute z(u)y(u) mod (2~~ - 1) and x(u)y(u) mod (urn + l), then we will combine the results by (57).) C3. (Recurse.] Set (20,. , ~~-1) to the cyclic convolution of (zo, . . . , xrn-i) with . , Ym-1). Also set (zm, . . . , ZZ,,-~) to the negacyclic convolution of PXT?%,…, x2,+-1) with (yn, . . . , yz,,+i). C4. [Unremainderize.] For 0 I Ic < m, set (zk, zmfk) e i(zk + zrnfk, Zk -zm+k). Now (~0,. . . , zm–l) is the desired answer. i Nl. [Test for simple case.] If n = 1, set t t xs(yo + yi), zs e t -(x0 + xl)yl, s1 + t + (xi -xs)ys, and terminate. Otherwise set m + 2Ln 2J and r + 2mi21. (The following steps use 2 +l auxiliary variables Xtj for 0 2 i < 2m and 0 < j < r, to represent 2m polynomials Xi(w) = Xi0 +Xilw+. . . +X,(,-l)~ - ; similarly, there are 2nf auxiliary variables Yij.) N2. [Initialize auxiliary polynomials.] Set Xij t X(i+,)j c xmj+i Yij + ~i+,)j + ymj+%, for 0 < i < m and 0 I j < r. (At this point we have x(u) = XO(U ) + UXl(U ) + . . . + u–l Xm-i(um), and a similar formula holds for y(u). Our strategy will be to multiply these polynomials modulo (urn7 + 1) = (u + I), by operating modulo (w f 1) on the polynomials X(W) and Y(W), finding their cyclic correlation of length 2m and thereby obtaining x(z~)y(u) = Zo(zlm) + u,&(u ) + . . + U2m–1Z2m–1(Um).) N3. [Transform.] (Now we will essentially do a fast Fourier transform on the poly- nomials (X0,. . . ,Xm–l, 0,. . . ,0) and (Yo, . . . , Y,-1, 0, . . . ,O), using w as a (2m)th root of unity. This is efficient, because multiplication by a power of w is not really a multiplication at all.) For j = [n/2] -1, . . . , 1, 0 (in this or- der), do the following for all m binary numbers s + t = (~1~12~. . . sj+lO . . .O)z + (0.. . Otj-1 . . . &~)a: Replace (Xs+t(w), X,+t+2j(w)) by the pair of polynomials (Xs+t(w) + ~(~~~)(~~~)X,+~+~j(w),X~+t(w) -w(7 )(s 2)X,+,+23(~)). (See Section 4.3.3 and Eq. 4.3.3-33. The operation Xi(w) + Xi(w) + wkXl(w) means, more precisely, that we set Xi3 +- Xij + X[(j+k) if j + k < r, otherwise Xij t Xi3 -for 0 5 j < r.) Do the same transformation on the Y s. xl(j+k-r), N4. [Recurse.] For 0 < i < 2m, set (Zie, . . . ,2,(,-l)) to the negacyclic convolution of (X0,. . . ,X+-I)) and . . . , Yz(,–I)). (Y,o, N5. [Untransform.] For j=O, 1, . . . , [n/2] (in this order), set (Zs+t(w), Zs+t+2j(w))+ ~(Zs+~(w)+ZS+t+23(w), ~–(~~~)(~~~)(Z,+t(w)–Z~+~+~~(w))), for all m choices of s and t as in step N3. N6. [Repack.] (Now we have accomplished the goal stated at the end of step N2, since it is easy to show that the transform of the Z s is the product of the transforms of the X s and the Y s) Set zi + ZZo -Z(,+i)(,-1) and Z,j+i + Ztj +Z(m+z)()-l) for 0 < j < r, for 0 5 i < m. 1

652 ANSWERS TO EXERCISES 4.6.4 57. Let N be the smallest power of 2 that exceeds 2n, and let un+l = . . . = UN-1 = vn+1 = . = UN-1 = 0. If u, = CO

4.6.4 ANSWERS TO EXERCISES 651 complex multiplications are all simple since they involve only two real multiplications and no real additions.) We can construct a normal scheme for the two-dimensional m x m case by applying the m scheme to vectors F(t , *) of length m . Each si step becomes m additions; each rnj becomes a Fourier transform on m elements, but with all of the (Y S in this algorithm multiplied by crj; and each tk becomes m additions. Thus the new algorithm has (a m + c a ) complex additions, t t trivial multiplications, and a total of c c complex multiplications. Using these techniques, Winograd has found normal one-dimensional schemes for the following small values of m with the following costs (a, t, c): m = 2 ( 2,2,2) m = 7 (36,1, 9) m = 3 ( 6,1,3) m=8 (2% 6, m = 4 ( 8,4,4) m = 9 (46,1,12) m=5 (17,1,6) m = 16 (74,8,18) By combining these schemes as described above, we obtain methods that use fewer arithmetic operations than the fast Fourier transform (FFT) discussed in exercise 14. For example, when m = 1008 = 7.9.16, the costs come to (17946,8,1944), so we can do a Fourier transform on 1008 complex numbers with 3872 real multiplications and 35892 real additions. It is possible to improve on Winograd s method for combining relatively prime moduli by using multidimensional convolutions, as shown by Nussbaumer and Quandalle in IBM J. Res. and Devel. 22 (1978), 134-144; their ingenious approach reduces the amount of computation needed for 1008-point complex Fourier transforms to 3084 real multiplications and 34668 real additions. By contrast, the FFT on 1024 complex numbers involves 14344 real multiplications and 27652 real additions. If the two-passes-at-once improvement in the answer to exercise 14 is used, however, the FFT on 1024 complex numbers needs only 10936 real multiplications and 25948 additions, and it is not difficult to implement. Therefore the subtler methods are faster only on machines that take significantly longer to multiply than to add. [References: Proc. Nat. Acad. Sci. USA 73 (1976), 1005-1006; Math. Comp. 33 (1978), 175-199; Advances in Math. 32 (1979), 83-117.1 54. max(2eldeg(pl) -1,. . ,2e,deg(p,) -1, q + 1). 55. 2n -q , where n is the degree of the minimum polynomial of P (i.e., the manic polynomial p of least degree such that p(P) is the zero matrix) and r is the number of distinct irreducible factors it has. (Reduce P by similarity transformations.) 56. Let tzjk + t @ = rijk + rjik, for all i, j, k. If A, B, c iS a realization Of (tijk) Of rank 7, then cl m, otherwise ?-ijk = 0. Now rank( r%,k) + rank(?-jik) 2 rank(t,,k), since we obtain a realization of (tijk) by suppressing the last n rows of A and the first m rows of B in a realization A, B, c of (7ijk f $:32k).

650 ANSWERS TO EXERCISES 4.6.4 (a) Let n(k) = (p-l)pePk- = (p(pePk) for 0 5 k < e, and n(k) = 1 for k 2 e. Represent the numbers { 1, . . . , m} in the form aipk (modulo m), where 0 2 k 2 e and 0 2 i < n(k), and a is a fixed primitive element modulo pe. For example, when m = 9 we can let a = 2; the values are {2 30,,2130, 2 31,2230,2530,2131,2430,2330, 2032}. Then na pk) = COllle COlj

4.6.4 ANSWERS TO EXERCISES 649 48. If A, B, C and A , B , C are realizations of (tzjk) and (t:3k) of respective lengths r and r , then A = A @ A , B = B @ B , C = C @ C , and A = A @ A , B = B @J B , C = C @ C , are realizations of (tyJk) and (tyik) of respective lengths r + r and r . r . Note: Many people have made the natural conjecture that rank((t,,k) @ (tLJk)) = rank(t,,k) f rank($,), but the construction in exercise 60(b) makes this seem much less plausible than it once was. 49. By Lemma T, rank(tijk) 2 rank(ti(jk)). Conversely if M is a matrix of rank r we can transform it by row and column operations, finding nonsingular matrices F and G such that FMG has all entries 0 except for r diagonal elements that are 1; cf. Algorithm 4.6.2N. The tensor rank of FMG is therefore < r; and it is the same as the tensor rank of M, by exercise 44. 50. Let i = (i , i ) where 1 5 i 5 m and 1 2 i 5 n; then t(zr,2/t)3k = &u~&,~, and it iS Clear that rank(ti(jk)) = mn since (i&k)) is a permutation matrix. By Lemma L, rank(tijk) 2 mn. COnVf?rSdy, since (tljk) has only mn nonzero entries, its rank is clearly 2 mn. (There is consequently no normal scheme requiring fewer than the mn obvious multiplications. There is no such abnormal scheme either [Comm. Pure and Appl. Math. 3 (1970), 165-1791. But some savings can be achieved if the same matrix is used with s > 1 different column vectors, since this is equivalent to (m x n) times (n X s) matrix multiplication.) 51. (a) si = y0 + yl, s2 = y0 -yl; ml = $(x0 + ZI)SI, mz = +(x0 -sl)sz; w. = ml +mz, wi = ml -ms. (b) Here are some intermediate steps, using the methodology in the text: ((~0 -~2) + (m -ZZ)U)((YO -YZ) + (YI -~z)u)mod(u~ + u + 1) = ((zo -az)(yo -yz) -(21 -52)(Yl -y2)) + ((x0 - z)(Yo -y2) -(21 -ZO)(Yl -yo))u. The first realization is The second realization is The resulting algorithm computes si = yc + ~1, sz = yc -yl, ss = y2 -yc, s4 = y2 -yl, s5 = so + y2; ml = +(x0 + 21 + 22).5x, m2 = +(x0 + ZI -2z2)s2, m3 = +(x0 -2×1 + x2)53, m4 = &(-220 + zl +x2).%&; tl = ml + m2, t-2 = ml -m2, t3 = ml + m3, w0 = tl -m3, wl = t3 + m4, w2 = t2 -m4. 52. Let i = (i , i ) when i mod n = i and i mod n = i . Then we wish to compute W(k’,k”) = c x(i’,i”)y(~’,j”) summed for i + j = k (modulo n ) and i + j = Ic (modulo n ). This can be done by applying the n algorithm to the 272 vectors X,, and Y$ of length n , obtaining the n vectors wk . Each vector addition becomes n additions, each parameter multiplication becomes n parameter multiplications, and each chain multiplication of vectors is replaced by a cyclic convolution of degree n . [If the subalgorithms use the minimum number of chain multiplications, this algorithm uses 2(n -d(n ))(n -d(n )) more than the minimum, where d(n) is the number of divisors of n.]

648 ANSWERS TO EXERCISES 4.6.4 /3so = 0. The result set now has at most m + 1 + cl 3. This single canonical form involves m2 + 2m parameters. As the cr s run through all integers and as we run through all chains, the /3 s run through at most 2m2f2m sets of values mod 2, hence the result set does also. In order to obtain all 2n polynomials of degree n with O-l coefficients, we need m2 + 2m > n. (c) Set m + 161 and compute z2, x3, . . . , xm. Let U(X) = ZL~+~(X)X(~+ )~ + . . . + ul(x)xm f u,J(x), where each ~~(5) is a polynomial of degree 5 m with integer coefficients (hence it can be evaluated without any more multiplications). Now evaluate U(X) by rule (2) as a polynomial in xm with known coefficients. (The number of additions used is approximately the sum of the absolute values of the coefficients, so this algorithm is efficient on O-l polynomials. Paterson and Stockmeyer also gave another algorithm that uses about 6 multiplications.) Reference: SIAM J. Computing 2 (1973), 60-66; see also J. E. Savage, SIAh4 J. Computing 3 (1974), 150-158. For analogous results about additions, see Borodin and Cook, SIAM J. Computing 5 (1976), 146-157; Rivest and Van de Wiele, Inf. Proc. Letters 8 (1979), 178-180.1 43. When ai = aj $ ak is a step in some optimal addition chain for n $1, compute xi = z3xk and pi = p& fpj, where pi = x2-l+. . .+x+1; omit the final calculation of xnf . We save one multiplication whenever ak = 1, in particular when i = 1. (Cf. exercise 4.6.3-31 with c = a.) 44. It suffices to show that (TZjk) s rank is at most that of (&k), since we can obtain (tijk) back from (Tijk) by transforming it in the same way with F-l, G-l, H- . If t ajk = c 1 <1<7 azl bjl ckl then it follows immediately that [H. F. de Groote has proved that all normal schemes that yield 2 X 2 matrix products with seven chain multiplications are equivalent, in the sense that they can be obtained from each other by nonsingular matrix multiplication as in this exercise. In this sense Strassen s algorithm is unique.] 45. By exercise 44 we can add any multiple of a row, column, or plane to another one without changing the rank; we can also multiply a row, column, or plane by a nonzero constant, or transpose the tensor. A sequence of such operations can always be found to reduce agive 2 X 2 x 2 tensor to one of the forms ( )( )00 00 9 ( )( )00 7 (lo)(oo),01 (~~)(~~), 00 00 (A y)( z t). The last tensor has rank 3 or 2 according as the polynomial u2 - TU -9 has one or two irreducible factors in the field of interest, by Theorem W (cf. (72)). 46. A general m X n X s tensor has mns degrees of freedom. By exercise 28 it is impossible to express all m X n X s tensors in terms of the (m + n + S)T elements of a realization A, B, C unless (m + n + S)T 2 mns. On the other hand, assume that m 2 n 2 s. The rank of an m X n matrix is at most n, so we can realize any tensor in ns chain multiplications by realizing each matrix plane separately. [Exercise 45 shows that this lower bound on the maximum tensor rank is not best possible, nor is the upper bound. Thomas D. Howell (Ph. D. thesis, Cornell Univ., 1976) has shown that there are tensors of rank 2 [mns/(m + n + s - 2)1 over the complex numbers.]