Matrix Singular Value Decomposition (SVD): A Complete Guide with Examples

Singular Value Decomposition (SVD) is a matrix factorization technique that decomposes a matrix into three components: ${U}$, ${\Sigma}$, and ${V^T}$. It simplifies complex matrix operations and is widely used in applications like data compression, machine learning, and signal processing to reduce dimensionality and extract meaningful patterns from data.

Get Homework Help

1. Introduction to Singular Value Decomposition (SVD):

Singular Value Decomposition (SVD) is a powerful tool in linear algebra, often used to simplify complex matrix operations. It breaks down a matrix into three components, making it easier to analyze, compute, and apply to various real-world problems. SVD is widely used in data science, machine learning, image processing, and more, as it can uncover hidden patterns, reduce data dimensionality, and improve computational efficiency.

2. What is Singular Value Decomposition (SVD):

SVD is a mathematical technique that factors a matrix ${A}$ into three distinct matrices: ${U}$, ${\Sigma}$, and ${V^T}$. These matrices are derived from the original matrix and offer insights into its structure. The decomposition can be written as: ${A = U \Sigma V^T}$

Where:

• ${U}$ is an orthogonal matrix containing the left singular vectors.
• ${\Sigma}$ is a diagonal matrix containing the singular values.
• ${V^T}$ is the transpose of the orthogonal matrix ${V}$, containing the right singular vectors.

Each matrix has specific properties and serves a unique purpose in understanding the structure of the original matrix.

3. Mathematical Representation of SVD:

The SVD formula is: ${A = U \Sigma V^T}$

Here's a breakdown of each component:

• ${U}$: This is an orthogonal matrix (meaning ${U^T U= I)}$, and it contains the left singular vectors of ${A}$.

• ${\Sigma}$: This diagonal matrix contains the singular values of ${A}$. The singular values represent the magnitude of the transformations applied by the matrix.

• ${V^T}$: The transpose of matrix ${V}$, which is also orthogonal. It contains the right singular vectors of ${A}$.

Orthogonal matrices preserve the length and angle of vectors, making them highly useful for transformations. The diagonal matrix ${\Sigma}$ simplifies the original matrix by concentrating the key information along the diagonal.

4. How to Compute Singular Value Decomposition (SVD):

Computing SVD involves several steps:

1. Find Eigenvalues and Eigenvectors: Compute the eigenvalues and eigenvectors of ${A^T A}$ and ${A A^T}$. These eigenvectors form the columns of the orthogonal matrices ${U}$ and ${V}$, while the square roots of the eigenvalues are the singular values in ${\Sigma}$.

2. Construct Matrices ${U, \Sigma, \text{and} \space V^T}$: The left singular vectors form ${U}$, the singular values are placed in ${\Sigma}$, and the right singular vectors form ${V^T}$.

3. Matrix Decomposition: Multiply these three matrices together to verify the decomposition.

This method highlights the relationship between SVD and eigenvalue decomposition, showing how SVD leverages these fundamental concepts to decompose matrices.

5. Properties of Singular Value Decomposition:

SVD has several important properties:

• Orthogonality of ${U}$ and ${V}$: Both ${U}$ and ${V}$ are orthogonal matrices, meaning ${U^T U = I}$ and ${V^T V = I}$. This orthogonality ensures that the columns of these matrices are mutually perpendicular.

• Uniqueness of Singular Values: The singular values in ${\Sigma}$ are unique and non-negative. They represent the magnitude of the principal components of the matrix.

• Stability Under Small Perturbations: SVD is stable even when small changes or noise are introduced to the original matrix. This makes it an excellent choice for real-world applications where data can be imperfect.

6. Applications of Singular Value Decomposition (SVD):

SVD has a wide range of applications in various fields:

1. Data Compression (Image Compression): SVD can reduce the dimensionality of data by representing it with fewer components, which is crucial in image compression. By truncating smaller singular values, we can approximate images with fewer data points.

2. Principal Component Analysis (PCA): SVD is a key part of PCA, where it helps reduce the dimensionality of data while preserving its essential features. This is widely used in machine learning for feature extraction.

3. Latent Semantic Analysis (LSA): In natural language processing, SVD helps in uncovering the hidden relationships between words and documents by reducing the dimensionality of term-document matrices.

4. Signal Processing: SVD is used in noise reduction and filtering applications, particularly in signal and audio processing.

7. Singular Value Decomposition Solved Examples:

Question: 1
Find the singular value decomposition SVD of given ${2 \times 2}$ Matrix
Given the matrix: ${A =\begin{bmatrix} 1 &2 \\ \\ 2 & 3 \end{bmatrix}}$

Step By Step Solution :-
The Singular value decomposition (SVD) of the given matrix can be obtained as the multiple of three matrices represented by the formula.
${A = U \cdot \sigma \cdot V^T}$
Where ${A}$ is the given matrix of any order

Step-1
First, we will find the Transpose of the given matrix ${A}$ as
$A^T = \begin{bmatrix} 1 & 2 \\ \\ 2 & 3 \end{bmatrix}^T = \begin{bmatrix} 1 & 2 \\ \\ 2 & 3 \end{bmatrix}$

Step-2
Now we will multiply the original matrix by its transpose.
$W = A \cdot A^T = \begin{bmatrix} 1 & 2 \\ \\ 2 & 3 \end{bmatrix} \cdot \begin{bmatrix} 1 & 2 \\ \\ 2 & 3 \end{bmatrix} = \begin{bmatrix} 5 & 8 \\ \\ 8 & 13 \end{bmatrix}$

Step-3
Now we will find the eigenvalues & eigenvectors related to matrix ${W}$ obtained in above step.

EigenValue ${(\lambda_1) = 17.94427190999916}$, Eigenvector ${=\begin{bmatrix} \frac{(\sqrt(5)-1)}{2} \\ \\ 1 \end{bmatrix}}$

EigenValue ${(\lambda_2) = 0.05572809000084078}$, Eigenvector ${=\begin{bmatrix} \frac{(-\sqrt(5)-1)}{2} \\ \\ 1 \end{bmatrix}}$

Step-4
Now we will find the square roots of the non $-$ zero eigenvalues ${(\sigma_i)}$.

${\sigma_1 = \sqrt{17.94427190999916} = 4.236068}$

${\sigma_2 = \sqrt{0.05572809000084078} = 0.236068}$

Now we can generate the ${\Sigma}$ matrix which is a zero matrix with ${\sigma_i}$ on its diagonals.
$\Sigma = \begin{bmatrix} 4.236068 & 0 \\ \\ 0 & 0.236068 \end{bmatrix}$

Step-5
Now we can generate the ${U}$ matrix which is made up of the normalised unit vectors related to each eigenvector.
${U_1} = \begin{bmatrix} \frac{{\sqrt{5} - 1}}{2 \cdot \sqrt{-\frac{\sqrt{5}}{2} \frac{5}{2}}} \\ \\ \frac{1}{\sqrt{-\frac{\sqrt{5}}{2} \frac{5}{2}}} \end{bmatrix}$

${U_2} = \begin{bmatrix} -\frac{{\sqrt{5} 1}}{2 \cdot \sqrt{\frac{\sqrt{5}}{2} \frac{5}{2}}} \\ \\ \frac{1}{\sqrt{\frac{\sqrt{5}}{2} \frac{5}{2}}} \end{bmatrix}$

${U} = \begin{bmatrix} \frac{{\sqrt{5} - 1}}{2 \cdot \sqrt{-\frac{\sqrt{5}}{2} \frac{5}{2}}} & -\frac{{\sqrt{5} 1}}{2. \sqrt{\frac{\sqrt{5}}{2} \frac{5}{2}}} \\ \\ \frac{1}{\sqrt{-\frac{\sqrt{5}}{2} \frac{5}{2}}} & \frac{1}{\sqrt{\frac{\sqrt{5}}{2} \frac{5}{2}}} \end{bmatrix}$

Step-6
Now we can generate the ${V}$ matrix by using the given formula
${V_i = \frac{1}{\sigma_i} \begin{bmatrix} 1 & 2 \\ \\2 & 3 \end{bmatrix}^T \cdot U_i}$

${V_1 = \frac{1}{4.236068} \begin{bmatrix} 1 & 2 \\ \\ 2 & 3 \end{bmatrix}^T \cdot \begin{bmatrix} \frac{{\sqrt{5} 1}}{2 \cdot \sqrt{-\frac{\sqrt{5}}{2} \frac{5}{2}}} \\ \\ \frac{1}{\sqrt{-\frac{\sqrt{5}}{2} \frac{5}{2}}} \end{bmatrix}= \begin{bmatrix} 0.526 \\ \\ 0.851 \end{bmatrix}}$

${V_2 = \frac{1}{0.236068} \begin{bmatrix} 1 & 2 \\ \\ 2 & 3 \end{bmatrix}^T \cdot \begin{bmatrix} -\frac{{\sqrt{5} 1}}{2 \cdot \sqrt{\frac{\sqrt{5}}{2} \frac{5}{2}}} \\ \\ \frac{1}{\sqrt{\frac{\sqrt{5}}{2} \frac{5}{2}}} \end{bmatrix}= \begin{bmatrix} 0.851 \\ \\ - 0.526 \end{bmatrix}}$

Therefore ${V = \begin{bmatrix} 0.526 & 0.851 \\ \\ 0.851 & - 0.526 \end{bmatrix}}$

Hence, the obtained matrices ${U}$, ${\Sigma}$ and ${V}$ are such that the ${A=U. \Sigma . V^T}$

Final Answer
The SVD of the above given matrix is

${U} = \begin{bmatrix} \frac{{\sqrt{5} - 1}}{2 \cdot \sqrt{-\frac{\sqrt{5}}{2} \frac{5}{2}}} & -\frac{{\sqrt{5} 1}}{2 \cdot \sqrt{\frac{\sqrt{5}}{2} \frac{5}{2}}} \\ \\ \frac{1}{\sqrt{-\frac{\sqrt{5}}{2} \frac{5}{2}}} & \frac{1}{\sqrt{\frac{\sqrt{5}}{2} \frac{5}{2}}} \end{bmatrix}$ or ${\begin{bmatrix} 0.526 & -0.851 \\ \\ 0.851 & 0.526 \end{bmatrix}}$

$\Sigma = \begin{bmatrix} 4.236068 & 0 \\ \\ 0 & 0.236068 \end{bmatrix}$ or ${\begin{bmatrix} 4.236 & 0 \\ \\ 0 & 0.236 \end{bmatrix}}$

${V = \begin{bmatrix} 0.526 & 0.851 \\ \\ 0.851 & - 0.526 \end{bmatrix}}$ or ${\begin{bmatrix} 0.526 & 0.851 \\ \\ 0.851 & - 0.526 \end{bmatrix}}$

Question: 2
Find the singular value decomposition SVD of given ${3 \times 3}$ Matrix
Given the matrix: $A=\begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & 1 \\ \\ 0 & 0 & 2 \end{bmatrix}$

Step By Step Solution :-
The Singular value decomposition (SVD) of the given matrix can be obtained as the multiple of three matrices represented by the formula.
${A = U \cdot \sigma \cdot V^T}$
Where ${A}$ is the given matrix of any order

Step-1
First, we will find the Transpose of the given matrix ${A}$ as
$A^T = \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & 1 \\ \\ 0 & 0 & 2 \end{bmatrix}^T = \begin{bmatrix} 3 & -1 & 0 \\ \\ 1 & 3 & 0 \\ \\ 1 & 1 & 2 \end{bmatrix}$

Step-2
Now we will multiply the original matrix by its transpose.
$W = A \cdot A^T = \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & 1 \\ \\ 0 & 0 & 2 \end{bmatrix} \cdot \begin{bmatrix} 3 & -1 & 0 \\ \\ 1& 3 & 0 \\ \\ 1 & 1 & 2\end{bmatrix} = \begin{bmatrix} 11 & 1 & 2 \\ \\ 1 & 11 & 2 \\ \\ 2 & 2 & 4 \end{bmatrix}$

Step-3
Now we will find the eigenvalues & eigenvectors related to matrix ${W}$ obtained in above step.

EigenValue ${(\lambda_1) = 12.898979485566356}$, Eigenvector ${=\begin{bmatrix} \frac{(\sqrt(6) 2)}{2} \\ \\ \frac{(\sqrt(6) 2)}{2} \\ \\ 1 \end{bmatrix}}$

EigenValue ${(\lambda_2) = 3.1010205144336442}$, Eigenvector ${=\begin{bmatrix} \frac{(-\sqrt(6) 2)}{2} \\ \\ \frac{(-\sqrt(6) 2)}{2} \\ \\1 \end{bmatrix}}$

EigenValue ${(\lambda_3) = 10}$, Eigenvector ${=\begin{bmatrix} -1 \\ \\ 1 \\ \\ 0 \end{bmatrix}}$

Step-4
Now we will find the square roots of the non $-$ zero eigenvalues ${(\sigma_i)}$.

${\sigma_1 = \sqrt{12.898979485566356} = 3.591515}$

${\sigma_2 = \sqrt{3.1010205144336442} = 1.760971}$

${\sigma_2 = \sqrt{10} = \sqrt{2} \sqrt{5}}$

Now we can generate the ${\Sigma}$ matrix which is a zero matrix with ${\sigma_i}$ on its diagonals.
$\Sigma = \begin{bmatrix} 3.591515 & 0 & 0 \\ \\ 0 & 1.760971 & 0 \\ \\ 0 & 0 & \sqrt{2} \cdot \sqrt{5} \end{bmatrix}$

Step-5
Now we can generate the ${U}$ matrix which is made up of the normalised unit vectors related to each eigenvector.
${U_1} = \begin{bmatrix} \frac{\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \\ \\ \frac{\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \\ \\ \frac{1}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \end{bmatrix}$

${U_2} = \begin{bmatrix} \frac{-\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{-2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \\ \\ \frac{-\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{-2 \cdot {\sqrt{2} \cdot\sqrt{3} 6}}} \\ \\ \frac{1}{\sqrt{-2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \end{bmatrix}$

${U_3} = \begin{bmatrix} -\frac{1}{\sqrt2} \\ \\ \frac{1}{\sqrt2} \\ \\ 0 \end{bmatrix}$

${U} = \begin{bmatrix} \frac{\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & \frac{-\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{-2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & - \frac{1}{\sqrt2} \\ \\ \frac{\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & \frac{-\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{-2 \cdot{\sqrt{2} \cdot \sqrt{3} 6}}} & \frac{1}{\sqrt2} \\ \\ \frac{1}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & \frac{1}{\sqrt{-2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & 0 \end{bmatrix}$

Step-6
Now we can generate the ${V}$ matrix by using the given formula
${V_i = \frac{1}{\sigma_i} \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & 1 \\ \\ 0 & 0 & 2 \end{bmatrix}^T \cdot U_i}$

${V_1 = \frac{1}{3.591515} \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & 1 \\ \\ 0 & 0 & 2 \end{bmatrix}^T \cdot \begin{bmatrix} \frac{\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \\ \\ \frac{\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \\ \\ \frac{{1}}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \end{bmatrix}= \begin{bmatrix} 0.375 \\ \\ 0.751 \\ \\ 0.544 \end{bmatrix}}$

${V_2 = \frac{1}{1.760971} \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & 1 \\ \\ 0 & 0 & 2 \end{bmatrix}^T \cdot \begin{bmatrix} \frac{-\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{-2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \\ \\ \frac{-\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{-2 \cdot {\sqrt{2} \cdot\sqrt{3} 6}}} \\ \\ \frac{{1}}{\sqrt{-2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} \end{bmatrix}= \begin{bmatrix} -0.243 \\ \\ -0.487 \\ \\ 0.839 \end{bmatrix}}$

${V_3 = \frac{1}{\sqrt{2} \cdot \sqrt{5}} \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & 1 \\ \\ 0 & 0 & 2 \end{bmatrix}^T \cdot \begin{bmatrix} -\frac{1}{\sqrt{2}} \\ \\ \frac{1}{\sqrt{2}} \\ \\ 0 \end{bmatrix}= \begin{bmatrix} -\frac{2 \cdot \sqrt{5}}{5} \\ \\ \frac{\sqrt{5}}{5} \\ \\ 0 \end{bmatrix}}$

Therefore ${V = \begin{bmatrix} 0.375 & -0.243 & -\frac{2 \cdot \sqrt{5}}{5} \\ \\ 0.751 & - 0.487 & \frac{\sqrt{5}}{5} \\ \\ 0.544 & 0.839 & 0 \end{bmatrix}}$

Hence, the obtained matrices ${U}$, ${\Sigma}$ and ${V}$ are such that the ${A=U. \Sigma . V^T}$

Final Answer
The SVD of the above-given matrix is

${U} = \begin{bmatrix} \frac{\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & \frac{-\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{-2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & - \frac{1}{\sqrt2} \\ \\ \frac{\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & \frac{-\frac{\sqrt{2} \cdot \sqrt{3}}{2} {1}}{\sqrt{-2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & \frac{1}{\sqrt2} \\ \\ \frac{1}{\sqrt{2 \cdot {\sqrt{2} \cdot \sqrt{3} 6}}} & \frac{1}{\sqrt{-2 \cdot {\sqrt{2} \cdot\sqrt{3} 6}}} & 0 \end{bmatrix}$ or ${\begin{bmatrix} 0.674 & -0.214 & -0.707 \\ \\ 0.674 & -0.214 & 0.707 \\ \\ 0.303 & 0.953 & 0 \end{bmatrix}}$

$\Sigma = \begin{bmatrix} 3.591515& 0 & 0 \\ \\ 0 & 1.760971 & 0 \\ \\ 0 & 0 & \sqrt2 \cdot \space \sqrt5 \end{bmatrix}$ or ${\begin{bmatrix} 3.592 & 0 & 0 \\ \\ 0 & 1.736 & 0 \\ \\ 0 & 0 & 3.162 \end{bmatrix}}$

${V = \begin{bmatrix} 0.375 & -0.243 & \frac{2 \cdot \sqrt 5}{5} \\ \\ 0.751 & -0.487 & \frac{\sqrt5}{5} \\ \\ 0.544 & 0.839 & 0 \end{bmatrix}}$ or ${\begin{bmatrix} 0.375 & -0.243 & -0.894 \\ \\ 0.751 & -0.487 & 0.447 \\ \\ 0 .544 & 0.839 & 0 \end{bmatrix}}$

Question: 3
Find the singular value decomposition SVD of given ${3 \times 3}$ Matrix
Given the matrix: $A=\begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & \frac{1}{4} \\ \\ 0 & 0 & -2 \end{bmatrix}$

Step By Step Solution :-
The Singular value decomposition (SVD) of the given matrix can be obtained as the multiple of three matrices represented by the formula.
${A = U \cdot \sigma \cdot V^T}$
Where ${A}$ is the given matrix of any order

Step-1
First, we will find the Transpose of the given matrix ${A}$ as
$A^T = \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & \frac{1}{4} \\ \\ 0 & 0 & -2 \end{bmatrix}^T = \begin{bmatrix} 3 & -1 & 0 \\ \\ 1 & 3 & 0 \\ \\ 1 & \frac{1}{4} & -2 \end{bmatrix}$

Step-2
Now we will multiply the original matrix by its transpose.
$W = A \cdot A^T = \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & \frac{1}{4} \\ \\ 0 & 0 & -2 \end{bmatrix} \cdot \begin{bmatrix} 3 & -1 & 0 \\ \\ 1 & 3 & 0 \\ \\ 1 & \frac{1}{4} & -2 \end{bmatrix} = \begin{bmatrix} 11 & \frac{1}{4} & -2 \\ \\ \frac{1}{4} & \frac{161}{16} & -\frac{1}{2} \\ \\ -2 & -\frac{1}{2} & 4 \end{bmatrix}$

Step-3
We will find the eigenvalues & eigenvectors related to matrix ${W}$ obtained in the above step.

EigenValue ${(\lambda_1) = \frac{(\sqrt{17121} 241)}{32}}$, Eigenvector ${=\begin{bmatrix} \frac{(-\sqrt{(17121)}-113)}{68} \\ \\ \frac{(-\sqrt{(17121)}-113)}{272} \\ \\ 1 \end{bmatrix}}$

EigenValue ${(\lambda_2) = \frac{(-\sqrt{17121} 241)}{32}}$, Eigenvector ${=\begin{bmatrix} \frac{(\sqrt{(17121)}-113)}{68} \\ \\ \frac{(\sqrt{(17121)}-113)}{272} \\ \\ 1 \end{bmatrix}}$

EigenValue ${(\lambda_3) = 10}$, Eigenvector ${=\begin{bmatrix} \frac{-1}{4} \\ \\ 1 \\ \\ 0 \end{bmatrix}}$

Step-4
Now we will find the square roots of the non $-$ zero eigenvalues ${(\sigma_i)}$.

${\sigma_1 = \sqrt{\frac{(\sqrt{17121} 241)}{32}} = 3.408845}$

${\sigma_2 = \sqrt{\frac{(-\sqrt{17121} 241)}{32}} = 1.855337}$

${\sigma_2 = \sqrt{10} = \sqrt{2} \sqrt{5}}$

Now we can generate the ${\Sigma}$ matrix, a zero matrix with ${\sigma_i}$ on its diagonals.
$\Sigma = \begin{bmatrix} 3.408845 & 0 & 0 \\ \\ 0 & 1.855337 & 0 \\ \\ 0 & 0 & \sqrt{2} \cdot \sqrt{5} \end{bmatrix}$

Step-5
Now we can generate the ${U}$ matrix, which is made up of the normalized unit vectors related to each eigenvector.
${U_1} = \begin{bmatrix} {\frac{-\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{68 \cdot \sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} \frac{17121}{2176}}} \\ \\ {\frac{-\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{272 \cdot \sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} \frac{17121}{2176}}} \\ \\ \frac{1}{\sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} {\frac{17121}{2176}}} \end{bmatrix}$

${U_2} = \begin{bmatrix} {\frac{\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{68 \cdot\sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} \frac{17121}{2176}}} \\ \\ {\frac{\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{272 \cdot \sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} \frac{17121}{2176}}} \\ \\ \frac{1}{\sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} {\frac{17121}{2176}}} \end{bmatrix}$

${U_3} = \begin{bmatrix} -\frac{1}{\sqrt{17}} \\ \\ \frac{4}{\sqrt{17}} \\ \\ 0 \end{bmatrix}$

${U} = \begin{bmatrix} {\frac{-\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{68 \cdot \sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} \frac{17121}{2176}}} & {\frac{\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{68 \cdot\sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} \frac{17121}{2176}}} & -\frac{1}{\sqrt{17}} \\ \\ {\frac{-\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{272 \cdot \sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} \frac{17121}{2176}}} & {\frac{\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{272 \cdot \sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} \frac{17121}{2176}}} & \frac{4}{\sqrt{17}} \\ \\ \frac{1}{\sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} {\frac{17121}{2176}}} & \frac{1}{\sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} {\frac{17121}{2176}}} & 0 \end{bmatrix}$

Step-6
Now we can generate the ${V}$ matrix by using the given formula
${V_i = \frac{1}{\sigma_i} \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & \frac{1}{4} \\ \\ 0 & 0 & -2 \end{bmatrix}^T \cdot U_i}$

${V_1 = \frac{1}{3.408845} \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & \frac{1}{4} \\ \\ 0 & 0 & -2 \end{bmatrix}^T \cdot \begin{bmatrix} {\frac{-\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{68 \cdot \sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} \frac{17121}{2176}}} \\ \\ {\frac{-\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{272 \cdot \sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} \frac{17121}{2176}}} \\ \\ \frac{1}{\sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} {\frac{17121}{2176}}} \end{bmatrix} = \begin{bmatrix} -0.755 \\ \\ -0.481 \\ \\ -0.445 \end{bmatrix}}$

${V_2 = \frac{1}{1.855337} \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & \frac{1}{4} \\ \\ 0 & 0 & -2 \end{bmatrix}^T \cdot \begin{bmatrix} {\frac{\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{68 \cdot \sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} \frac{17121}{2176}}} \\ \\ {\frac{\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{272 \cdot \sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} \frac{17121}{2176}}} \\ \\ \frac{1}{\sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} {\frac{17121}{2176}}} \end{bmatrix} = \begin{bmatrix} 0.376 \\ \\ 0.239 \\ \\ -0.895 \end{bmatrix}}$

${V_3 = \frac{1}{\sqrt{2}.\sqrt{5}} \begin{bmatrix} 3 & 1 & 1 \\ \\ -1 & 3 & \frac{1}{4} \\ \\ 0 & 0 & -2 \end{bmatrix}^T \cdot \begin{bmatrix} -\frac{1}{\sqrt{17}} \\ \\ \frac{4}{\sqrt{17}} \\ \\ 0 \end{bmatrix}= \begin{bmatrix} -\frac{7 \cdot \sqrt{170}}{170} \\ \\ \frac{11 \cdot\sqrt{170}}{170} \\ \\ 0 \end{bmatrix}}$

Therefore ${V = \begin{bmatrix} -0.755 & 0.376 & -\frac{7 \cdot \sqrt{170}}{170} \\ \\ -0.481 & 0.239 & \frac{11 \cdot \sqrt{170}}{170} \\ \\ -0.445 & -0.895 & 0 \end{bmatrix}}$

Hence, the obtained matrices ${U}$, ${\Sigma}$ and ${V}$ are such that the ${A=U. \Sigma . V^T}$

Final Answer
The SVD of the above-given matrix is

${U} = \begin{bmatrix} {\frac{-\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{68 \cdot \sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} \frac{17121}{2176}}} & {\frac{\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{68 \cdot \sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} \frac{17121}{2176}}} & -\frac{1}{\sqrt{17}} \\ \\ {\frac{-\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{272 \cdot \sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} \frac{17121}{2176}}} & {\frac{\sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}- 113}{272 \cdot \sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} \frac{17121}{2176}}} & \frac{4}{\sqrt{17}} \\ \\ \frac{1}{\sqrt\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176} {\frac{17121}{2176}}} & \frac{1}{\sqrt {-\frac{113 \cdot \sqrt{3} \cdot \sqrt{13} \cdot \sqrt{439}}{2176}} {\frac{17121}{2176}}} & 0 \end{bmatrix}$ or ${\begin{bmatrix} -0.936& 0.253 & -0.243 \\ \\ -0.234 & 0.063 & 0.97 \\ \\ 0.261 & 0.965 & 0 \end{bmatrix}}$

$\Sigma = \begin{bmatrix} 3.408845 & 0 & 0 \\ \\ 0 & 1.855337 & 0 \\ \\ 0 & 0 & \sqrt{2} \cdot \sqrt{5} \end{bmatrix}$ or ${\begin{bmatrix}3.409 & 0 & 0 \\ \\ 0 & 1.855 & 0 \\ \\ 0 & 0 & 3.162 \end{bmatrix}}$

${V = \begin{bmatrix} -0.755 & -0.376 & \frac{7 \cdot \sqrt{170}}{170} \\ \\ -0.481 & 0.239 & \frac{11 \cdot \sqrt{170}}{170} \\ \\ -0.455 & -0.895 & 0 \end{bmatrix}}$ or ${\begin{bmatrix} -0.755 & 0.376 & -0.537 \\ \\ -0.481 & 0.239 & 0.844 \\ \\ -0.445 & -0.895 & 0 \end{bmatrix}}$

8. Practice Questions on Singular Value Decomposition:

1. Given the matrix: $A = \begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix}$
   Compute the SVD.

2. For the matrix: $A = \begin{bmatrix} 2 & 3 \\ 7 & 1 \end{bmatrix}$
   Find the singular values and verify the SVD.

3. Apply SVD to a larger ${3 \times3}$ matrix and compare the results with the original matrix after reconstructing it from ${U \Sigma V^T}$.

9. Frequently Asked Questions (FAQs) on SVD:

What is Singular Value Decomposition (SVD)?

SVD is a matrix factorization technique that breaks down a matrix into three components: ${U}$, ${\Sigma}$, and ${V^T}$. It’s used to simplify complex matrix operations and analyze the structure of the matrix.

What are the applications of SVD?

SVD is widely used in image compression, data reduction, principal component analysis (PCA), and natural language processing (NLP). It helps uncover hidden patterns, compress data, and improve computational efficiency.

Is SVD applicable to all matrices?

Yes, SVD can be applied to any matrix, whether it's square or rectangular, making it highly versatile for both real and complex matrices.

How is SVD related to PCA?

SVD is the underlying mathematical technique used in Principal Component Analysis (PCA), which helps reduce the dimensionality of large datasets by identifying principal components.

What are singular values?

Singular values are the diagonal elements of matrix ${\Sigma}$ in SVD. They represent the magnitude of the transformation applied by the matrix and are always non-negative.

Why is SVD important in machine learning?

SVD helps reduce the dimensionality of data, improve algorithm performance, and enable efficient storage and computation, particularly for large datasets.

What is the stability of SVD?

SVD is known for its stability even when small perturbations or noise are present in the data, making it a reliable tool for real-world applications.

Can SVD be used for data compression?

Yes, SVD is commonly used in image and data compression by retaining only the largest singular values, which allows the matrix to be approximated with fewer dimensions while retaining essential information.

10. Real-Life Applications of SVD:

Singular Value Decomposition has several real-world applications:

• Data Science and Machine Learning: SVD is used in dimensionality reduction techniques like PCA, improving the performance of algorithms by reducing data complexity.

• Computer Vision: In image processing, SVD helps compress and reconstruct images, reducing the file size while retaining essential features.

• Recommendation Systems: SVD is used in collaborative filtering for recommendation systems, such as in Netflix or Amazon, to predict user preferences based on past behaviors.

• Natural Language Processing (NLP): In LSA, SVD is applied to text data to uncover latent relationships between words and documents, improving search engines and document categorization.

11. Conclusion:

Singular Value Decomposition (SVD) is a fundamental tool in linear algebra, widely used in various applications ranging from data science to signal processing. By decomposing a matrix into orthogonal and diagonal components, SVD simplifies complex matrix operations, making it easier to work with high-dimensional data. Its properties, stability, and versatility make it a critical technique in modern computational methods.

If you have any suggestions regarding the improvement of the content of this page, please write to me at My Official Email Address: [email protected]

$\fcolorbox{black}{lightpink}{\color{blue}{Get Assignment Help}}$
Are you Stuck on homework, assignments, projects, quizzes, labs, midterms, or exams?
To get connected to our tutors in real time. Sign up and get registered with us.

$\color{red} \bold{Related \space Pages:}$
Diagonalize Matrix Calculator
Matrix Formula Sheet
Linear Algebra Calculators
Orthogonal Complement Calculator
Gram-Schmidt Process Calculator