Matrix 101 - Identity Matrix

2023-12-30

1 minute read

Matrix is very important in understanding math in machine learning. I will spend a few posts to explain the basics and then focus on the applications in ML.

symbols used in the post

Symbol	Meaning
$A$	the matrix, $A_{m n}$ means a $m \times n$ matrix
$A_{i j}$	the $(i, j)$ element of the matrix
$A^{- 1}$	the inverse of matrix $A$
$I$	the identity matrix, alternatively, use $I_{n}$ means dimension $n$ identity matrix
$0$	the zero matrix, or null matrix, all elements are 0
$diag$	the diagonal of matrix

Definition

Identity matrix is a matrix where elements on the main or principal diagonal are 1 and all other elements are 0.

Alternatively, unit matrix is also used.

$[\begin{matrix} 1 & 0 & 0 & \dots & 0 \\ 0 & 1 & 0 & \dots & 0 \\ 0 & 0 & 1 & \dots & 0 \\ ⋮ & ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & 0 & \dots & 1 \end{matrix}]$

The math notation of a identity matrix can be described via a diagonal matrix $I_{n} = diag (1, 1, 1, \dots, 1)$

Properties

Identity matric has some good peroperties in term of matrix mulitplication

multiplying any matrix by the identity matrix results in the matrix itself

$I_{m} A_{m n} = A_{m n} = A_{m n} I_{n}$

Any matrix multiplied by its inverse reulsts in identity matrix

$A A^{- 1} = I$

All the powers of identity matrix are equal to the identity matrix $I^{n} = I$

The identity matrix is analogous to the number “1” in scalar algebra.

References

https://en.wikipedia.org/wiki/Identity_matrix