Linear Algebra 3. Linear transformations and matrices, matrix operations.

Linear transformation (LT) is like a function that transforms (changes) vector: ƒ(x) => L(v⃗)

So linear transformation takes some input vector and produces some output vector.
A transformation is linear if:

1. all lines (of the coordinate system grid) are not become curved after transformation (horizontal, vertical and diagonal lines). In other words grid lines remain parallel and evenly spaced
2. the origin remains fixed in place

Example of LT - 90° clockwise rotation about the origin. How we can describe LT numerically? We have input vector with coordinates [x_in , y_in] and output vector with coordinates [x_out , y_out] . We know that each vector is just linear combination of the basis/unit vectors, so we can rewrite coordinates like:

1. [x_in , y_in] = x_in î + y_in ĵ
2. linear combination remains the same even after applying LT, so we just use transformed versions of the î and ĵ => LT(î) and LT(ĵ)
3. [x_out , y_out] = x_in LT(î) + y_in LT(ĵ)

Example of 90° clockwise rotation LT:

1. Take squared sheet of paper and draw two unit vectors; for convenience - each with length of 2 squares. 
2. If we make 90° clockwise rotation LT then:
1. we move î  90° clockwise - now î is down y axis and LT(î) coordinates (in terms of old greed - before transformation) are [0, -1].
2. we move ĵ  90° clockwise - now ĵ lies on x axis and LT(ĵ) coordinates (in terms of old greed - before transformation) are [1, 0].
3. if we have some vector v with coordinates [3,2]:
1. LT(v) = 3LT(î) + 2LT(ĵ) = 3[0, -1] + 2[1,0] = [0, -3] + [2,0] = [2, -3] in terms of greed before transformation

We can describe 2D (Cartesian plane) with 4 digits - 2 for î coordinates and 2 for ĵ coordinates. We can package this coordinates in two-by-two grid of numbers - array of numbers, or in terms of LA - matrix. Matrix will have 2 columns and 2 rows:

1. columns - 1st is î coordinates and 2nd is  ĵ coordinates
2. rows - 1st is x axis coordinates of î and ĵ , and 2nd - y axis coordinates of î and ĵ

⌈

0

1

⌉

⋅

⌈

3

⌉

=

3

⋅

⌈

0

⌉

+

2

⋅

⌈

1

⌉

=

⌈

0⋅3

+ 1⋅2

⌉

=

⌈

2

⌉

⌊

-1

0

⌋

⌊

2

⌋

⌊

-1

⌋

⌊

0

⌋

⌊

-1⋅3

+ 0⋅2

⌋

⌊

-3

⌋

Above we rewrote our linear combination as matrix-vector multiplication.

By convention we denote matrix in bold upper-case, like A . And we denote elements of a matrix upper-case non-bold, like A.

A_mxn is matrix with height of m (rows) and width of n (columns)
A_1,1 is element at 1 row and 1 column intersection, using above example of LT matrix, A_1,1= 0
To denote real valued matrix A_mxn : A∈ℝ^mxn
Colon symbol ":" represents "all" - all rows or all columns:

1. All numbers/elements of matrix on intersection with i column: A_:,i 
1. A_:,1 equals to the set {0, -1} - 1st column of A
2. All numbers/elements of matrix on intersection with i row: A_i,: 
1. A_2,: equals to the set {-1,0} - 2nd row of A

If we use more than 2 axes (2 axes is 2D) then we'll call such a matrix - tensor.

We can add matrices of the same shape by adding their corresponding elements/numbers:
C = A + B   where   C_i,j = A_i,j + B_i,j

⌈

0

1

⌉

+

⌈

-3

8

⌉

=

⌈

0+(-3)

1+8

⌉

=

⌈

-3

9

⌉

⌊

-1

0

⌋

⌊

-1

-5

⌋

⌊

-1+(-1)

0+(-5)

⌋

⌊

-2

-5

⌋

To add scalar to a matrix or to multiply matrix by a scalar, we must perform addition or multiplication of each element of a matrix:
D = aB + c where D_i,j = aB_i,j + c

 These materials were used while preparing this blog-post:

1. https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab
2. https://www.deeplearningbook.org/
3. NBGtLA by https://minireference.com/