Chapter 2 section 3 covers Linear independence.

In this post, I’ll continue writing R code to accompany linear algebra equations found in Linear Algebra: Step by Step, by Kuldeep Singh. For more information on the origins of these posts, see the first companion post

Section 2.3 – Linear Independence

To detect linear dependence in rows or columns, the simplest way I’ve found is by using the qr function to check rank. Unfortunately, rank is covered in section 3.5.5.

Example 2.13 can be completed with the code below, since we compare the output to what we know of the matrix. The rank is 3, and it’s a 3×3 matrix, so we have linear independence. First, establish A as the matrix, then transpose it in the qr function so we’re checking columns for linear dependence. The book later proves that row and column ranks are equal, but for the sake of this exercise, we’ll be sure to check columns.

A <- matrix(c(-3,0,2,1,1,0,0,-1,0), nrow = 3, byrow = TRUE)
qr(t(A), tol = NULL)

Example 2.14 is performed with the code below, interpreting similarly.

A <- matrix(c(2,-4,0,3,19,0,7,-5,0), nrow = 3, byrow = TRUE)
qr(t(A), tol = NULL)

In this post, I’ll continue writing R code to accompany linear algebra equations found in Linear Algebra: Step by Step, by Kuldeep Singh. For more information on the origins of these posts, see the first companion post

Thank to you everyone who hung out throughout the brief hiatus, while life got in the way of linear algebra.

Section 2.1 – Properties of Vectors

A brief review of how dot products work, in the form of example 2.1

u <- c(-3,1,7,-5)
v <- c(9,2,-4,1)
u %*% v

To continue, we’re able to determine the norm (or length) of a vector using a simple chain of square root and sum. Example 2.4 is as follows

u <- c(-7,-2)
v <- c(8,3)
sqrt(sum(u^2))
sqrt(sum(v^2))

Example 2.5 shows more real-world applicability, by showing the distance between two points.

s1 <- c(1,2,3)
s2 <- c(7,4,3)
sqrt(sum((s1-s2)^2))

Section 2.2 – Further Properties of Vectors

Let’s simplify the distance function. From here forward, use the following function as distance:

d <- function(u) {sqrt(sum(u^2))}

To show the use of d, here is example 2.6

u <- c(1,5)
v <- c(4,1)
u %*% v
d(u)*d(v)
d(u)+d(v)
d(u+v)

For example 2.7, let’s first create an extension of the d function that helps find angles. **Remember** – R trigonometric functions use radians and not degrees!

costheta <- function(vec1,vec2) {vec1%*%vec2 / (d(vec1)*d(vec2))}
rad.to.deg <- function(rad) { 180*rad / pi }

Then we can perform example 2.7 by embedding multiple functions. Feel free to break out as appropriate for your own understanding and simplicity if using elsewhere.

u <- c(-5,-1)
v <- c(4,2)
rad.to.deg(acos(costheta(u,v)))

In the remaining sections of this chapter, we go further with matrices, finally getting into transpose and inverse, homogeneous versus non-homogeneous systems, and solutions to these systems.

A quick reminder this is the R companion series to the book Linear Algebra: Step by Step, by Kuldeep Singh. As the series progresses, I’m sure you’ll see the benefits of this particular book in its approach to building to more complex ideas. To transfer pure mathematical skills to R, it’s important to follow these same building blocks along the way. In an effort to help cover the basic costs of the website, the link above goes to the Amazon product page. As an Amazon Associate I earn from qualifying purchases.

Many of you have written encouraging notes about the small amount we’ve covered so far, which I’m extremely grateful for. Comments or questions can be sent to feedback@detroitdatalab.com as this series goes on. I’m happy to edit or update blog posts, and even add supplemental ones.

Section 1.6 – The Transpose and Inverse of a Matrix

To transpose a matrix, you can use the function t from the base R package. Example 1.29 part (i) is shown below

A <- matrix(c(-9,2,3,7,-2,9,6,-1,5), nrow = 3, byrow = TRUE)
t(A)

An identity matrix can be created with diag. We’ll use diag more later, but for now, its basic usage works great here. The I₄ matrix then is below.

A <- diag(4)
A

For the inverse of a matrix, we go back to the matlib package and use the Inverse function (note the capital “I” in the function). For example 1.33, we can show B is the inverse of A with this code snippet.

A <- matrix(c(1,2,0,2,5,-1,4,10,-1), nrow = 3, byrow = TRUE)
Inverse(A)

Section 1.7 – Types of Solutions

If a system of equations has no solution, we consider it to be inconsistent. To reproduce example 1.36, we can perform the Gaussian elimination by using matlib once more. Our result shows the same 0,0,0,5 in the last row, proving the system is inconsistent.

A <- matrix(c(1,1,2,-1,3,-5,2,-2,7), nrow = 3, byrow = TRUE)
b <- c(3,7,1)
gaussianElimination(A,b)

Let’s also explore example 1.37 for the simplest method of solving a homogeneous equation.

A <- matrix(c(-1,2,3,1,-4,-13,-3,5,4), nrow = 3, byrow = TRUE)
b <- c(0,0,0)
gaussianElimination(A,b)

The interpret the above code’s output in line with the example, it’s important to understand what the result of a Gaussian elimination means. In this example, the top row is 1,0,7 and the second row is 0,1,5. Since our variables are x,y,z this means x+7z = 0, so we can understand that x = -7z. Similarly, we can see y+5z = 0, so y = -5z.

For a non-homogeneous system, the steps are very similar. Only your vector b will change. Try solving example 1.38 using the same steps and interpretation method.

Section 1.8 – The Inverse Matrix Method

To find the inverse of a matrix, I prefer matlib to pracma, because pracma is meant to emulate matlab’s methods.

Example 1.41 companion code is below.

A <- matrix(c(1,0,2,2,-1,3,4,1,8), nrow = 3, byrow = TRUE)
Inverse(A)

See how much easier this can be with R?

Example 1.42 part A puts inverse to use along with matrix multiplication syntax we learned before.

A <- matrix(c(1,0,2,2,-1,3,4,1,8), nrow = 3, byrow = TRUE)
b <- c(5,7,10)
invA <- Inverse(A)
invA %*% b

When a matrix is non-invertible, the error message from Inverse will tell you that the matrix is numerically singular.

Chapter 1 Summary

The packages we introduced in chapter 1 were matlib, pracma, and expm, along with some base functions within R.

At this point, you should be able to understand and use R to create a matrix from a system of linear equations, perform operations such as Gaussian elimination and interpret their results, perform matrix arithmetic and algebra, determine the transpose and inverse, and use these to identify solutions to systems where they exist.

Chapter 2 will be presented in the next blog post as an entire unit, covering topics in Euclidean Space.

 

Linear Algebra: Step by Step, by Kuldeep Singh, is a tremendous resource for improving your skills in the fundamental mathematics behind machine learning. I’m authoring an R companion series to ensure that this can be translated to make sense to R programmers, and reduce the legwork for translating core principles back and forth.

This series will be light on content, making the assumption that you have the book or are looking for very quick information on producing linear algebra concepts using R. Some sections will provide tips to packages, or simple code snippets to try textbook examples yourself.

For anyone interested in catching up or following along, the book is available for purchase via Amazon by clicking here.

Section 1.1 – Systems of Linear Equations

The most beneficial information for this section is plotting. My best recommendation is to become familiar with the ggplot2 package, by Hadley Wickham.

matlib is a package that’s useful for designing and solving a variety of problems around matrices.

Section 1.2 – Gaussian Elimination

gaussianElimination is a function within the matlib package, and is useful for guiding yourself through the steps. To borrow example 1.8, here is the companion code:

A <- matrix(c(1,3,2,4,4,-3,5,1,2), nrow = 3, byrow = TRUE)
b <- c(13,3,13)
gaussianElimination(A, b, verbose = TRUE)

We can use pracma as a means for identifying the reduced row echelon form (rref) of matrices.

Example 1.9 can be double-checked using the rref function from pracma

A <- matrix(c(1,5,-3,-9,0,-13,5,37,0,0,5,-15), nrow = 3, byrow = TRUE)
rref(A)

Section 1.3 – Vector Arithmetic

The base R packages have a special syntax for dot products.

u <- c(-3,1,7)
v <- c(9,2,-4)
u %*% v

Section 1.4 – Arithmetic of Matrices

We continue using the special syntax for matrix multiplication. Example 1.18 part A is reproduced below.

A <- matrix(c(2,3,-1,5), nrow = 2, byrow = TRUE)
x <- matrix(c(2,1), nrow = 2)
A %*% x

Section 1.5 – Matrix Algebra

The zero matrix can be created very easily. A 2 by 2 zero matrix is created with the code below.

matrix(0,2,2)

We should introduce the expm package for its function %^% to perform matrix powers. Example 1.27 part A is solved below.

A <- matrix(c(-3,7,1,8), nrow = 2, byrow = TRUE)
A %^% 2

To be continued…

We will continue on with the rest of Chapter 1 in part 2. This series is ongoing, and will be grouped in sections that I’m able to pull together at a given time.

 

Note about links to some materials above: As an Amazon Associate I earn from qualifying purchases.