Matrix 101 - Identity Matrix
Matrix is very important in understanding math in machine learning. I will spend a few posts to explain the basics and then focus on the applications in ML.
symbols used in the post
| Symbol | Meaning |
|---|---|
| $\mathbf{A}$ | the matrix, $\mathbf{A}_{mn}$ means a $m \times n $ matrix |
| $A_{ij}$ | the $(i,j)$ element of the matrix |
| $\mathbf{A}^{-1}$ | the inverse of matrix $\mathbf{A}$ |
| $\mathbf{I}$ | the identity matrix, alternatively, use $\mathbf{I}_n$ means dimension $n$ identity matrix |
| $\mathbf{0}$ | the zero matrix, or null matrix, all elements are 0 |
| $\text{diag}$ | the diagonal of matrix |
Definition
Identity matrix is a matrix where elements on the main or principal diagonal are 1 and all other elements are 0.
Alternatively, unit matrix is also used.
$$
\begin{bmatrix}
1 & 0 & 0 & \cdots & 0 \\
0 & 1 & 0 & \cdots & 0 \\
0 & 0 & 1 & \cdots & 0 \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
0 & 0 & 0 & \cdots & 1
\end{bmatrix}
$$
The math notation of a identity matrix can be described via a diagonal matrix $$ \mathbf{I}_n = \text{diag}(1, 1, 1, \cdots, 1) $$
Properties
Identity matric has some good peroperties in term of matrix mulitplication
multiplying any matrix by the identity matrix results in the matrix itself
$$ \mathbf{I}_{m}\mathbf{A}_{mn} = \mathbf{A}_{mn} = \mathbf{A}_{mn}\mathbf{I}_{n} $$
Any matrix multiplied by its inverse reulsts in identity matrix
$$ \mathbf{A} \mathbf{A}^{-1} = \mathbf{I} $$
All the powers of identity matrix are equal to the identity matrix $$ \mathbf{I}^{n} = \mathbf{I} $$
The identity matrix is analogous to the number “1” in scalar algebra.