Linear Algebra 4. Matrix multiplication.

When you apply one LT and then the other LT (example: 90° clockwise rotation and then shear (shift) the overall effect is another LT which is composition of two LT. This LT will capture overall effect of applying 2 LTs into a single LT.
Applying several LT to one vector is like using several functions and using output of one function as input to the other:
ƒ(g(x)) where:

1. g is the first LT with input "x"
2. ƒ is the second LT with input from the previous LT

So the same as with functions - we apply LT from right to the left. 

The composition of two LTs is multiplication / product / dot product of two LT - product of two matrices:

C  = AB  where:

1. A have the same number of columns as B has rows or mathematically
1.  A_{l x m} and B_{m x n} 
2. easy way - to check dot product possibility
1.  example: A_{2 x 3} and B_{3 x 4}
2. Write dimensions of matrices one after the other with "=" sign between them:
1. 2 x 3 = 3 x 4  as you see 3 = 3, so we can dot-product these matrices
3. example: A_{2 x 2} and B_{3 x 3}
1. 2 x 2  = 3 x 3 as you see 2 = 3 is not true, so we can't multiply that matrices

2. Ci,j =	m	Ai,k Bk,j
   	∑
   	k=1

   this means: starting with k=1 and till k=m multiply each A_i,k by B_k,j and also i = {1,2,...,l} and j={1,2,..,n}

For example if we have two matrices and want to find their dot-product:

⌈	0	2	⌉	⌈	1	-2	⌉
⌊	1	0	⌋	⌊	1	0	⌋

A_{2 x 2} and B_{2 x 2} find k:  2 x 2 = 2 x  2 , 2 = 2, k = 2, k shows how many times we sum product of factors A and B:

Change general formula for this particular case:

Ci,j =	2	Ai,k Bk,j
	∑
	k=1

Then:

C1,1 =	2	A1,k Bk,1	=	A1,1 B1,1 +	A1,2 B2,1 =	0*1 + 2*1 = 2
	∑
	k=1

C_1,2 = A_1,1 ⋅ B_1,2 + A_1,2 ⋅ B_2,2= 0*(-2) + 2*0 = 0
C_2,1 = A_2,1 ⋅ B_1,1 + A_2,2 ⋅ B_2,1= 1*1 + 0*1 = 1
C_2,2 = A_2,1 ⋅ B_1,2 + A_2,2 ⋅ B_2,2= 1*(-2) + 0*0 = -2

The simplest way to calculate matrix dot product is to approach it as matrix vector product:
First we find î of the right matrix after applying left matrix (LT):

⌈	0	2	⌉	⋅	⌈	1	⌉
⌊	1	0	⌋		⌊	1	⌋

Secondly we find ĵ hat of the right matrix after applying left matrix (LT):

⌈	0	2	⌉	⋅	⌈	-2	⌉
⌊	1	0	⌋		⌊	1	⌋

​
Matrix product properties:
A(B + C) = AB + AC distributive property
A(BC) = (AB)C associative property
But AB≠BA (because matrix is LT, and LT is like function, so apply right to left)

Also matrix element wise product (or Hadamard product) exists. It is supported only for matrices of the same shape:
C = A ⊙ B where C_i,j = A_i,j B_i,j

With 3-D Tensor basis are î , ĵ and k̂ and it's linear combination is:

v =	î	xî + yĵ + zk̂
	ĵ =
	k̂

These materials were used while preparing this blog-post:

1. https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab
2. https://www.deeplearningbook.org/
3. NBGtLA by https://minireference.com/