Orthogonality

<< LinearAlgebra`MatrixManipulation`
<<LinearAlgebra`Orthogonalization`

Part 1.  Basic operations

Experiment with randomly generated vectors, and determine the meaning of "Dot".  (Note Mathematica's ambivalence about whether a vector is a row or a column.)

\(x\  = {Random[Integer, \ {\(-99\), 99}\ ], \ \ Random[
          Integer, \ {\(-99\), 99}\ ], 
		\ \ \ \ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)
\(y\  = {Random[Integer, \ {\(-99\), 99}\ ], \ \ Random[
          Integer, \ {\(-99\), 99}\ ], 
		\ \ \ \ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)
x  .  y
Dot[x, y]

Confirm your observation symbolically.

Clear[x, a, y, b]
x  =  Array[a, 4]
y  =  Array[b, 4]
Dot[x, y]

Experiment with randomly generated vectors, and determine the meaning of "VectorNorm", in particular, the 2-norm. (If you are curious about other norms, change the "2" to some other integer, and see what happens.)

\(x\  = {Random[Integer, \ {\(-99\), 99}\ ], \ \ Random[
          Integer, \ {\(-99\), 99}\ ], 
		\ \ \ \ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)
Sqrt[ Sum[x[[i]]^2,  {i,  1,  4} ] ]
Sqrt[ Dot[x, x] ]
N[%]
VectorNorm[N[x],  2]

Enter the matrix A, and calculate (A^T)A.  What do you conclude about the columns of A?

\(A\  = \ {\ {\(-6\), \(-3\), 6, 1}, \ {\(-1\), 2, 1, \(-6\)}, \ {3, 6, 
          3, \(-2\)}, 
		\ \ \ \ \ \ \ \ {6, \(-3\), 
          6, \(-1\)}, \ {2, \(-1\), 2, 3}, \ {\(-3\), 6, 3, 
          2}, 
		\ \ \ \ \ \ \ \ {\(-2\), \(-1\), 2, \(-3\)}, {1, 2, 1, 
          6}\ }; \ A // \ MatrixForm\)
Transpose[A]  .  A  //  MatrixForm

Check your conclusion(s) by using the Dot and VectorNorm commands.   (Note that taking columns of the transpose of A is the same as taking rows of A.)

A1  =  Transpose[A][[1]]
A2  =  Transpose[A][[2]]
A3  =  Transpose[A][[3]]
A4  =  Transpose[A][[4]]

Part 2.  Orthogonal projections

Construct a random vector in (R^8).

\(y\  = \ {Random[Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ], 
		\ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ], 
		\ \ \ \ \ \ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ], 
		\ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)

Find the projection of y in the direction of A1.

Dot[y, A1]/Dot[A1, A1]*A1

Find the projection of  y on the subspace W=Col(A).

Part 3.  Orthonormal bases

Enter a matrix U whose columns are the normalized columns of A.  Find (U^T)U and U (U^T).

Part 4.  Orthogonal matrices

Construct random 4-vectors.

\(v1\  = \ {Random[Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ], 
		\ \ \ \ \ \ \ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)
\(v2\  = \ {Random[Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ], 
	\ \ \ \ \ \ 	\ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)
\(v3\  = \ {Random[Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ], 
		\ \ \ \ \ \ \ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)
\(v4\  = \ {Random[Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ], 
	\ \ \ \ \ \ 	\ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)

Apply the Gram-Schmidt process to these vectors.

u1 = 
      N[GramSchmidt[{v1, v2, v3, v4},  
            Normalized -> False][[1]]]
u2 = 
      N[GramSchmidt[{v1, v2, v3, v4},  
            Normalized -> False][[2]]]
u3 = 
      N[GramSchmidt[{v1, v2, v3, v4},  
            Normalized -> False][[3]]]
u4 = 
      N[GramSchmidt[{v1, v2, v3, v4},  
            Normalized -> False][[4]]]

Normalize these vectors.

A1 = u1/VectorNorm[u1, 2]
A2 = u2/VectorNorm[u2, 2]
A3 = u3/VectorNorm[u3, 2]
A4 = u4/VectorNorm[u4, 2]

Construct the orthogonal matrix A.

A  =  Transpose[ {A1,  A2,  A3,  A4} ];   A  //  
      MatrixForm

Compute (A^T)A and A (A^T).

Enter random vectors.

Clear[x, y]
\(x\  = {Random[Integer, \ {\(-99\), 99}\ ], \ \ Random[
          Integer, \ {\(-99\), 99}\ ], 
		\ \ \ \ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)
\(y\  = \ {Random[Integer, \ {\(-99\), 99}\ ], \ \ Random[
          Integer, \ {\(-99\), 99}\ ], 
		\ \ \ \ \ Random[
          Integer, \ {\(-99\), 99}\ ], \ 
        Random[Integer, \ {\(-99\), 99}\ ]}\)

Compute Dot[x,y] and Dot[Ax,Ay].

Compute lengths of x and Ax.

Compute the eigenvalues of A.

Compute the absolute values of the eigenvalues (use Abs).

Compute the determinant of A.

Part 5.  Summary