Gram-Schmidt Process: Orthogonalization & Orthonormalization Explained

The Gram-Schmidt Process is a method in linear algebra used to transform a set of linearly independent vectors into an orthogonal or orthonormal set. It simplifies complex operations in vector spaces by ensuring that the vectors are perpendicular to each other, which is useful in many applications like QR decomposition, signal processing, and data science. This process plays a critical role in simplifying vector space calculations.

Get Homework Help

1. Introduction to the Gram-Schmidt Process:

The Gram-Schmidt Process is a method in linear algebra used to orthogonalize a set of vectors. It is essential for constructing orthogonal or orthonormal bases from a set of linearly independent vectors. This process plays a significant role in many applications, including numerical analysis, quantum mechanics, and data science, by providing a way to simplify complex vector spaces.

2. What is Gram-Schmidt Process:

The Gram-Schmidt Process is a step-by-step algorithm that takes a set of vectors from a vector space and turns them into an orthogonal (or orthonormal) set. If the resulting vectors are normalized, meaning their length is ${1}$, they form an orthonormal set. This method is commonly used to simplify vector space problems by transforming bases into easier-to-manipulate forms.

The key idea is to take each vector in the set and project it onto the already orthogonalized vectors, subtracting the projection to remove any component that’s not orthogonal.

3. How to Find the Gram-Schmidt Process:

The Gram-Schmidt Process can be broken down into several steps. Let’s assume you are given a set of linearly independent vectors ${v_1, v_2, \cdots, v_n}$ in an inner product space.

Step-by-Step Procedure:

1. Start with the first vector ${v_1}$ and set it as the first orthogonal vector: ${u_1=v_1}$

2. For each subsequent vector, subtract the projection of the vector onto the already orthogonal vectors: ${u_k} = {v_k} - \displaystyle\sum_{i=1}^{k-1} \dfrac{\langle {v_k}, {u_i} \rangle}{\langle {u_i}, {u_i} \rangle} {u_i}$
   Where ${\langle {v_k}, {u_i} \rangle}$ is the dot product between the vectors, and ${u_i}$ is the previously computed orthogonal vector.

3. Normalize the orthogonal vectors to obtain an orthonormal set by dividing each vector by its norm: ${e_k} = \frac{{u_k}}{\|{u_k}\|}$ Where ${\|{u_k}\|}$ is the magnitude of vector ${u_k}$.

4. Rules for Gram-Schmidt Process:

Here are some important rules to follow when applying the Gram-Schmidt Process:

1. Linear Independence: The input vectors must be linearly independent. The process will fail if any vector in the set depends on the others.

2. Order Matters: The order in which input vectors affect the result. Changing the order of vectors will yield a different set of orthogonal vectors.

3. Normalization: The final step often involves normalizing the orthogonal vectors to create an orthonormal set, particularly useful in many applications.

4. Dot Products: Use dot products to calculate the projection of one vector onto another.

5. Properties of Gram-Schmidt Process:

The Gram-Schmidt Process has several key properties:

1. Orthogonality: The vectors produced by this process are mutually orthogonal. That means the dot product between any two different vectors is zero.

2. Orthonormality: After normalization, the vectors will have unit length, making them orthonormal. They are orthogonal, and each vector has a magnitude of ${1}$.

3. Stability: The process can suffer from numerical instability when applied to large or nearly linearly dependent vectors. In practice, modified versions, like the Modified Gram-Schmidt Process, are used to avoid this problem.

4. Basis Transformation: The output vectors span the same subspace as the input vectors but are easier to work with due to their orthogonality.

6. Gram-Schmidt Process Solved Examples:

Question: 1
Orthogonalizing Vectors in 3D
Given the vectors: ${v_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \space \space v_2 = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, \space \space v_3 = \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}}$

Solution:

Step 1: Set ${u_1 = v_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}}$

Step 2: Calculate ${u_2}$
Find the projection of ${v_2}$ onto ${u_1}$: $\space \space {\text{proj}_{u_1}{v_2} = \dfrac{\langle {v_2}, {u_1} \rangle}{\langle {u_1}, {u_1} \rangle} {u_1} = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}}$

Now subtract the projection from ${v_2}$: $\space \space {{u_2} = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} - \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} }$

Step 3: Calculate ${u_3}$
Find the projection of ${v_3}$ onto ${u_1}$ and ${u_2}$: $\space \space \space \text{proj}_{u_1} {v_3} = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \space \space \text{proj}_{u_2} {v_3} = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}$

Now subtract both projections: $\space \space \space {u_3} = \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix} - \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} - \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$

The orthogonal vectors are: $\space \space \space {u_1} = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \space \space \space {u_2} = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, \space \space \space {u_3} = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$

Question: 2
Orthogonalizing 2D Vectors
Given the vectors: ${v_1 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \space \space v_2 \begin{bmatrix} 1 \\ 1 \end{bmatrix}}$

Solution:

Step 1: Set ${u_1 = v_1}$ $\space \space{u_1 = v_1 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}}$

Step 2: Calculate ${u_2}$ by subtracting the projection of ${v_2}$ onto ${u_1}$ $\space \space {\text{proj}_{u_1}{v_2} = \dfrac{\langle {v_2}, {u_1} \rangle}{\langle {u_1}, {u_1} \rangle} {u_1}}$

First, calculate the dot products: $\langle {v_2}, {u_1} \rangle = 1(1) 1(0) = 1, \quad \langle {u_1}, {u_1} \rangle = 1^2 0^2 = 1$

Now, compute the projection: $\text{proj}_{u_1} {v_2} = \dfrac{1}{1} \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \end{bmatrix}$

Subtract the projection from ${v_2}$: $\space \space {u_2} = {v_2} - \text{proj}_{u_1} {v_2} = \begin{bmatrix} 1 \\ 1 \end{bmatrix} - \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \end{bmatrix}$

The orthogonal vectors are: $\space \space {u_1} = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \space \space {u_2} = \begin{bmatrix} 0 \\ 1 \end{bmatrix}$

Question: 3
Orthogonalizing 2D Vectors
Given the vectors: $\space \space {v_1} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \space \space {v_2} = \begin{bmatrix} 3 \\ 1 \end{bmatrix}$

Solution:

Step 1: Set ${u_1 = v_1}$ $\space \space \space{u_1 = v_1}\begin{bmatrix} 1 \\ 2 \end{bmatrix}$

Step 2: Calculate ${u_2}$
We subtract the projection of ${v_2}$ onto ${u_1}$: $\space \space \space \text{proj}_{u_1} {v_2} = \dfrac{\langle {v_2}, {u_1} \rangle}{\langle {u_1}, {u_1} \rangle} {u_1}$

First, compute the dot products: $\langle {v_2}, {u_1} \rangle = 3(1) 1(2) = 3 2 = 5, \space \space \space \langle {u_1}, {u_1} \rangle = 1^2 2^2 = 1 4 = 5$

Now, compute the projection: $\text{proj}_{u_1} {v_2} = \dfrac{5}{5} \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$

Subtract the projection from ${v_2}$: $\space \space \space {u_2}={v_2}- \text{proj}_{u_1} {v_2} = \begin{bmatrix} 3 \\ 1 \end{bmatrix} - \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 2 \\ -1 \end{bmatrix}$

The orthogonal vectors are: ${u_1} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \space \space {u_2} = \begin{bmatrix} 2 \\ -1 \end{bmatrix}$

Question: 4
Orthogonalizing 2D Vectors
Given the vectors: $\space \space {v_1} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}, \space \space {v_2} = \begin{bmatrix} 4 \\ 3 \end{bmatrix}$

Solution:

Step 1: Set ${u_1 = v_1}$ $\space \space \space{u_1 = v_1}\begin{bmatrix} 2 \\ 1 \end{bmatrix}$

Step 2: Calculate ${u_2}$
We subtract the projection of ${v_2}$ onto ${u_1}$: $\space \space \space \text{proj}_{u_1} {v_2} = \dfrac{\langle {v_2}, {u_1} \rangle}{\langle {u_1}, {u_1} \rangle} {u_1}$

First, compute the dot products: $\langle {v_2}, {u_1} \rangle = 4(2) 3(1) = 8 3 = 11, \space \space \space \langle {u_1}, {u_1} \rangle = 2^2 1^2 = 4 1 = 5$

Now, compute the projection: $\text{proj}_{u_1} {v_2} = \dfrac{11}{5} \begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} \frac{22}{5} \\ \frac{11}{5} \end{bmatrix}$

Subtract the projection from ${v_2}$: $\space \space \space {u_2}={v_2}- \text{proj}_{u_1} {v_2} = \begin{bmatrix} 4 \\ 3 \end{bmatrix} - \begin{bmatrix} \frac{22}{5} \\ \frac{11}{5} \end{bmatrix} = \begin{bmatrix} \frac{-2}{5} \\ \frac{4}{5} \end{bmatrix}$

The orthogonal vectors are: ${u_1} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}, \space \space {u_2} = \begin{bmatrix} \frac{-2}{5} \\ \frac{4}{5} \end{bmatrix}$

Question: 5
Orthogonalizing 3D Vectors
Given the vectors: $\space \space {v_1} = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}, \space \space {v_2} = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, \space \space {v_3}\begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix}$

Solution:

Step 1: Set ${u_1 = v_1}$ $\space \space \space{u_1 = v_1}\begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}$

Step 2: Calculate ${u_2}$
We calculate the projection of ${v_2}$ onto ${u_1}$: $\space \space \space \text{proj}_{u_1} {v_2} = \dfrac{\langle {v_2}, {u_1} \rangle}{\langle {u_1}, {u_1} \rangle} {u_1}$

First, compute the dot products:
$\langle {v_2}, {u_1} \rangle = 1(1) 1(0) 0(1) = 1, \langle {u_1}, {u_1} \rangle = 1^2 0^2 1^2 = 1 0 1=2$

Now, compute the projection: $\text{proj}_{u_1} {v_2} = \dfrac{1}{2} \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} \frac{1}{2} \\ 0 \\ \frac{1}{2} \end{bmatrix}$

Subtract the projection from ${v_2}$: $\space \space \space {u_2}={v_2}- \text{proj}_{u_1} {v_2} = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} - \begin{bmatrix} \frac{1}{2} \\ 0 \\ \frac{1}{2} \end{bmatrix} = \begin{bmatrix} \frac{1}{2} \\ 1 \\ -\frac{1}{2} \end{bmatrix}$

Step 3: Calculate ${u_3}$

Now, calculate the projection of ${v_3}$ onto ${u_1}$ and ${u_2}$: $\space \space \space \text{proj}_{u_1} {v_3} = \dfrac{\langle {v_3}, {u_1} \rangle}{\langle {u_1}, {u_1} \rangle} {u_1}$

First, compute the dot product: ${\langle {v_3}, {u_1} \rangle = 0(1) 1(0) 1(1)=1}$

Now, compute the projection: $\space \space \space \text{proj}_{u_1} {v_3} = \dfrac{1}{2} \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} \frac{1}{2} \\ 0 \\ \frac{1}{2} \end{bmatrix}$

Next, calculate the projection onto ${u_2}$ $\space \space\text{proj}_{u_2} {v_3} = \dfrac{\langle {v_3}, {u_2} \rangle}{\langle {u_2}, {u_2} \rangle} {u_2}$

First, compute the dot products: $\langle {v_3}, {u_2} \rangle = 0 \begin{bmatrix} \dfrac{1}{2} \end{bmatrix} 1(1) 1 \begin{bmatrix} \dfrac{-1}{2} \end{bmatrix} = 0 1 - \dfrac{1}{2} = \dfrac{1}{2}$

Now, compute the projection: $\text{proj}_{u_2} {v_3} = \frac{\frac{1}{2}}{\frac{3}{2}} {u_2} = \frac{1}{3} \begin{bmatrix} \frac{1}{2} \\ 1 \\ \frac{-1}{2} \end{bmatrix} = \begin{bmatrix} \frac{1}{6} \\ \frac{1}{3} \\ \frac{-1}{6} \end{bmatrix}$

Now subtract both projections from ${v_3}$: $\space \space {u_3} = {v_3} - \begin{bmatrix} \frac{1}{2} \\ 0 \\ \frac{1}{2} \end{bmatrix} - \begin{bmatrix} \frac{1}{6} \\ \frac{1}{3} \\ \frac{-1}{6} \end{bmatrix} = \begin{bmatrix} \frac{-1}{3} \\ \frac{2}{3} \\ \frac{2}{3} \end{bmatrix}$

The orthogonal vectors are: $\space {u_1} = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}, \quad {u_2} = \begin{bmatrix} \frac{1}{2} \\ 1 \\ -\frac{1}{2} \end{bmatrix}, \quad {u_3} = \begin{bmatrix} -\frac{1}{3} \\ \frac{2}{3} \\ \frac{2}{3} \end{bmatrix}$

7. Practice Questions on Gram-Schmidt Process:

Q.1 Use the Gram-Schmidt Process to orthogonalize the following set of vectors: ${v_1} = \begin{bmatrix} 1 \\ 1 \end{bmatrix}, \space {v_2} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$

Q.2 Find the orthonormal basis for the span of the vectors: ${v_1} = \begin{bmatrix} 1 \\ 9 \\ 0 \end{bmatrix}, \space {v_2} = \begin{bmatrix} 2 \\ 0 \\ 7 \end{bmatrix}, \space {v_3} = \begin{bmatrix} 1 \\ -1 \\ 1 \end{bmatrix}$

Q.3 Apply the Gram-Schmidt Process to the following vectors: ${v_1} = \begin{bmatrix} 3 \\ 6 \end{bmatrix}, \space {v_2} = \begin{bmatrix} 5 \\ -8 \end{bmatrix}$

8. FAQs on Gram-Schmidt Process:

What is the Gram-Schmidt Process used for?

The Gram-Schmidt Process is used in linear algebra to convert a set of linearly independent vectors into an orthogonal (or orthonormal) set. This process helps simplify complex problems involving vector spaces, making it easier to perform calculations such as projections and matrix decompositions.

What is the difference between orthogonal and orthonormal vectors?

Orthogonal vectors are perpendicular to each other, meaning their dot product is zero. Orthonormal vectors are orthogonal vectors that have been normalized, meaning their length (or magnitude) is ${1}$. The Gram-Schmidt Process can produce orthogonal or orthonormal vectors, depending on whether the vectors are normalized.

Can the Gram-Schmidt Process fail?

The Gram-Schmidt Process can fail if the set of input vectors is linearly dependent. In such cases, the process will attempt to generate a zero vector, making it impossible to construct an orthogonal set. The method requires a set of linearly independent vectors for successful application.

What are the applications of the Gram-Schmidt Process?

The Gram-Schmidt Process is used in various fields, including:

• QR decomposition in numerical analysis.
• Dimensionality reduction in machine learning and data science.
• Signal processing, where orthogonal vectors simplify complex signals.
• **Quantum mechanics to construct orthonormal bases for vector spaces.

How do you normalize a vector in the Gram-Schmidt Process?

To normalize a vector, divide it by its length (or magnitude). For a vector ${u}$, the normalized vector ${e}$ is given by: ${e= \frac{u}{\|{u}\|}}$
Where ${\|{u}\|}$ is the magnitude of the vector ${u}$. This ensures the vector has a unit length (magnitude of ${1}$).

What is the difference between the original and Modified Gram-Schmidt Process?

The original Gram-Schmidt Process follows a step-by-step projection and subtraction method, while the Modified Gram-Schmidt Process improves numerical stability by re-orthogonalizing vectors. To reduce numerical errors, the modified version is often preferred for high-dimensional and ill-conditioned problems.

Does the order of vectors matter in the Gram-Schmidt Process?

Yes, the order of the vectors affects the outcome of the Gram-Schmidt Process. Changing the order of input vectors will produce different orthogonal or orthonormal vectors because the projections and subtractions depend on the sequence in which the vectors are processed.

9. Real-Life Application of Gram-Schmidt Process:

The Gram-Schmidt Process is widely used in data science for dimensionality reduction, where it helps to create orthogonal features from a set of correlated data. It's also crucial in quantum mechanics for constructing orthogonal states and in numerical methods to improve the accuracy of matrix decompositions in methods like QR factorization.

10. Conclusion:

The Gram-Schmidt Process is a powerful linear algebra tool for transforming vectors into an orthogonal or orthonormal set. It simplifies many complex operations in vector spaces, such as projections and decompositions, and has a wide range of real-world applications in fields like engineering, physics, and data science. Understanding this process's rules, steps, and properties is essential for anyone dealing with vector spaces and orthogonality in mathematics.

If you have any suggestions regarding the improvement of the content of this page, please write to me at My Official Email Address: [email protected]

$\fcolorbox{black}{lightpink}{\color{blue}{Get Assignment Help}}$
Are you Stuck on homework, assignments, projects, quizzes, labs, midterms, or exams?
To get connected to our tutors in real time. Sign up and get registered with us.

$\color{red} \bold{Related \space Pages:}$
Projection of a Vector Calculator
Matrix Formula Sheet
Linear Algebra Calculators
Matrix Inverse Calculator
Magnitude of a vector Calculator