Suppose that we want to know the values of the variables {x,} {y,} and {z} that will make all three of these equations true simultaneously:
{ \eqalign { \hfil x &\hfil - &\hfil 5y &\hfil + &\hfil 2z &\hfil = &\hfil 1 \cr \hfil 4x &\hfil + &\hfil y &\hfil - &\hfil 6z &\hfil = &\hfil 4 \cr \hfil 2x &\hfil + &\hfil 3y &\hfil + &\hfil z &\hfil = &\hfil 19 \cr } }
This is an example of a system of linear equations. This problem can be solved using basic algebra steps by repeatedly isolating and substituting variables among the three equations. However, that process can be quite tedious, and it can sometimes be challenging to figure out exactly what steps will advance you toward a solution.
In this chapter, we’ll introduce a technique for solving these problems in a systematic way that’s usually quite faster.
This introduction is for students who have had one year of algebra education. Sections 13 and beyond are for students who are also familiar with matrix multiplication.
Let’s solve this system of two linear equations:
{2x} {+} {4y} {=} {14} {5x} {+} {3y} {=} {21}
The goal is to find values for {x} and {y} that will make both equations true.
Elementary algebra provides us with the techniques for solving this problem. The algebra steps are easy to apply, although the entire process is a bit lengthy:
First, pick an equation and isolate one of the variables on one side of the equation:
{2x} {+} {4y} {=} {14} {[ 1]} given { x} {+} {2y} {=} {7} {[ 2]} divide by {2} to help isolate {x} { x} {=} {7} {-} {2y} {[ 3]} subtract {2y} to isolate {x}
Then, substitute the isolated variable into the other equation, which will solve it:
{ 5x } {+} { 3y} {=} { 21} {} {} {[ 4]} given { 5(7 - 2y) } {+} { 3y} {=} { 21} {} {} {[ 5]} substitute {x} from {[ 3]} { 35 - 10y } {+} { 3y} {=} { 21} {} {} {[ 6]} distribute { 35 } {-} { 7y} {=} { 21} {} {} {[ 7]} combine the factors of {y} { } {-} { 7y} {=} { 21} {-} {35} {[ 8]} subtract {25} { } {-} { 7y} {=} {-14} {} {} {[ 9]} simplify { } { } { y} {=} { 2} {} {} {[10]} divide by {-7}
Finally, take the known variable
{ 2x } {+} { 4y } {=} {14} {} {} {[11]} given, from {[ 1]} { 2x } {+} { 4 \mathord{\times} 2} {=} {14} {} {} {[12]} substitute {y} from {[ 10]} { 2x } {+} { 8 } {=} {14} {} {} {[13]} simplify { 2x } { } { } {=} {14} {-} {8} {[14]} subtract {8} { 2x } { } { } {=} { 6} {} {} {[15]} simplify { \phantom{2}x } { } { } {=} { 3} {} {} {[16]} divide by {2}
And we obtain the solution:
{x} {=} {3} from {[16]} {y} {=} {2} from {[10]}
In this section, we’ll introduce another way of solving a system of linear equations using a technique called “row reduction”.
The row reduction technique has these advantages:
Over the next four sections, we’ll be applying the row reduction technique to solve this system of linear equations:
{2x} {+} {4y} {=} {14} {5x} {+} {3y} {=} {21}
The first step in this technique is to define an augmented matrix that represents the two equations. An augmented matrix is an organized, rectangular grid of all the numbers used in the equations. Here’s the augmented matrix for the above equations:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 2 &\hfil\hspace-5pt 4 &\hfil 14 \cr \hfil 5 &\hfil\hspace-5pt 3 &\hfil 21 \cr \end{array} \hspace+2pt\right] }
The matrix is organized this way:
Some equations will require additional rules:
Here are some examples of setting up a system of linear equations as an augmented matrix:
{ \eqalign { \hfil 3x &\hfil + & y = &\hfil 15 \cr \hfil 2x &\hfil + & 4 = &\hfil 12 \cr } } 🡆 { \left[\hspace+2pt \begin{array}{rr|r} \hfil 3 &\hfil\hspace-5pt 1 &\hfil 15 \cr \hfil 2 &\hfil\hspace-5pt 0 &\hfil 8 \cr \end{array} \hspace+2pt\right] }
{ \eqalign { \hfil y &\hfil = &\hfil 4x & + &\hfil 8 \cr \hfil y &\hfil = &\hfil -3x & + &\hfil 1 \cr } } 🡆 { \left[\hspace+2pt \begin{array}{rr|r} \hfil -4 &\hfil\hspace-5pt 1 &\hfil 8 \cr \hfil 3 &\hfil\hspace-5pt 1 &\hfil 1 \cr \end{array} \hspace+2pt\right] }
{ \begin{array} \hfil\hspace+8.5pt x &\hfil\hspace+2.0pt y &\hfil\hspace+3.0pt z \cr \end{array} }
{ \begin{array} \hfil\hspace+8.5pt \downarrow &\hfil\hspace+3.0pt \downarrow &\hfil\hspace+2.5pt \downarrow \cr \end{array} }{ \eqalign { \hfil 3x &\hfil - &\hfil 2y &\hfil = &\hfil 8 \cr \hfil 2x &\hfil + &\hfil 4z &\hfil = &\hfil 12 \cr \hfil 2y &\hfil - &\hfil 5z &\hfil = &\hfil -1 \cr } } 🡆 { \left[\hspace+2pt \begin{array}{rrr|r} \hfil 3 &\hfil\hspace-5pt -2 &\hfil\hspace-5pt 0 &\hfil 8 \cr \hfil 2 &\hfil\hspace-5pt 0 &\hfil\hspace-5pt 4 &\hfil 12 \cr \hfil 0 &\hfil\hspace-5pt 2 &\hfil\hspace-5pt -5 &\hfil -1 \cr \end{array} \hspace+2pt\right] }
The augmented matrix is designed to minimize the
amount of writing you need to
The row reduction technique is also frequently called the Gaussian elimination technique.
As an exercise, convert the following problem to an augmented matrix that represents a system of linear equations:
A farmer has a total of {11} animals, a mix of chickens and cows. One day he counts the legs of all of his animals and realizes there’s a total of {34} legs. How many chickens and how many cows does the farmer have? Use {x} to represent the number of chickens, and {y} to represent the number of cows.
Here’s the system of linear equations that we’ll be solving using the row reduction technique:
{2x} {+} {4y} {=} {14} {5x} {+} {3y} {=} {21}
In the previous section, we introduced the first step of the technique, which is to define an augmented matrix that represents those equations:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 2 &\hfil\hspace-5pt 4 &\hfil 14 \cr \hfil 5 &\hfil\hspace-5pt 3 &\hfil 21 \cr \end{array} \hspace+2pt\right] }
The goal of the row reduction technique is to convert this matrix to another matrix, called the reduced matrix, which directly contains the solution to the equations.
Here’s the conversion that we’ll be performing on the matrix:
the original matrix: { \left[\hspace+2pt \begin{array}{rr|r} \hfil 2 &\hfil\hspace-5pt 4 &\hfil 14 \cr \hfil 5 &\hfil\hspace-5pt 3 &\hfil 21 \cr \end{array} \hspace+2pt\right] } 🡇 the reduced matrix: { \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 0 &\hfil 3 \cr \hfil 0 &\hfil\hspace-5pt 1 &\hfil 2 \cr \end{array} \hspace+2pt\right] }
Notice that the reduced matrix represents the following system of equations:
{1x} {+} {0y} {=} {3} {0x} {+} {1y} {=} {2}
which can be simplified to:
{x} {=} {3} {y} {=} {2}
which, together, provide the solution to the original equations.
The reduced matrix uses the coefficients {1} and {0} to isolate exactly one variable per equation. To accomplish this, the entries on the left side of the vertical bar must satisfy these conditions:
Sometimes it isn’t possible to satisfy all of these conditions,
but if we can, then the
The reduced matrix is also frequently called the reduced row echelon form matrix.
Given this system of linear equations:
{2x} {+} {4y} {=} {14} {5x} {+} {3y} {=} {21}
and it’s corresponding augmented matrix:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 2 &\hfil\hspace-5pt 4 &\hfil 14 \cr \hfil 5 &\hfil\hspace-5pt 3 &\hfil 21 \cr \end{array} \hspace+2pt\right] }
our goal is to convert it to the following reduced matrix:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 0 &\hfil 3 \cr \hfil 0 &\hfil\hspace-5pt 1 &\hfil 2 \cr \end{array} \hspace+2pt\right] }
In the next section, we’ll perform this conversion one step at a time. Each step will use one of these three row operations:
Row operations:
- Interchange any two rows.
- Multiply all the entries in a row by a nonzero constant.
- Add or subtract a multiple of one row to another row.
None of these row operations will change the solution to the system of equations.
At each step, we’ll use the following strategy to choose which row operation to perform:
Row reduction strategy:
- Work column by column, doing the columns in any order that’s convenient.
- For each column, first change one entry to {1,} then change all the rest to
{0}s. - Never put two
{1}s in the same row on the left side of the vertical bar.- For extra speed, start with columns that already have a {1} or a {0.}
- For extra speed, don’t bother interchanging any rows.
Here are the steps to solve a system of linear equations using the row reduction technique:
Convert the system of equations to an augmented matrix:
{2x} {+} {4y} {=} {14} {5x} {+} {3y} {=} {21} 🡇 { \left[\hspace+2pt \begin{array}{rr|r} \hfil 2 &\hfil\hspace-5pt 4 &\hfil 14 \cr \hfil 5 &\hfil\hspace-5pt 3 &\hfil 21 \cr \end{array} \hspace+2pt\right] }
We’ll start by focusing on the first column.
We need to create a {1} in the first column, which
we can easily do by dividing the first row by
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 2 &\hfil\hspace-5pt 4 &\hfil 14 \cr \hfil 5 &\hfil\hspace-5pt 3 &\hfil 21 \cr \end{array} \hspace+2pt\right] } 🡄 divide by {2} 🡇 { \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 2 &\hfil 7 \cr \hfil 5 &\hfil\hspace-5pt 3 &\hfil 21 \cr \end{array} \hspace+2pt\right] } 🡄 changed
To finish the first column, we must change the {5} to a {0.} We can do this by taking {5} times the first row and subtracting it from the second row:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 2 &\hfil 7 \cr \hfil 5 &\hfil\hspace-5pt 3 &\hfil 21 \cr \end{array} \hspace+2pt\right] } 🡄 subtract { { \mathbf{[}\hspace+2pt \begin{array}{rr|r} \hfil 5 &\hfil\hspace-5pt 10 &\hfil 35 \cr \end{array} \hspace+2pt\mathbf{]} } } (which is {5} times the first row) 🡇 { \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 2 &\hfil 7 \cr \hfil 0 &\hfil\hspace-5pt -7 &\hfil -14 \cr \end{array} \hspace+2pt\right] } 🡄 changed
The first column is now finished.
We can now focus on the second column. We need to create a {1} in the second column, but it cannot be in the same row as the {1} in the first column. So our only choice is to change the {-7} to a {1.}
We can change the {-7} to a {1} by dividing the second row by
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 2 &\hfil 7 \cr \hfil 0 &\hfil\hspace-5pt -7 &\hfil -14 \cr \end{array} \hspace+2pt\right] } 🡄 divide by {-7} 🡇 { \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 2 &\hfil 7 \cr \hfil 0 &\hfil\hspace-5pt 1 &\hfil 2 \cr \end{array} \hspace+2pt\right] } 🡄 changed
To finish the second column, we must change the {2} to a {0.} We can do this by taking {2} times the second row and subtracting it from the first row:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 2 &\hfil 7 \cr \hfil 0 &\hfil\hspace-5pt 1 &\hfil 2 \cr \end{array} \hspace+2pt\right] } 🡄 subtract { { \mathbf{[}\hspace+2pt \begin{array}{rr|r} \hfil 0 &\hfil\hspace-5pt 2 &\hfil 4 \cr \end{array} \hspace+2pt\mathbf{]} } } (which is {2} times the second row) 🡇 { \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 0 &\hfil 3 \cr \hfil 0 &\hfil\hspace-5pt 1 &\hfil 2 \cr \end{array} \hspace+2pt\right] } 🡄 changed
The second column is now finished.
The resulting matrix is:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 0 &\hfil 3 \cr \hfil 0 &\hfil\hspace-5pt 1 &\hfil 2 \cr \end{array} \hspace+2pt\right] }
which represents this system of linear equations:
{1x} {+} {0y} {=} {3} {0x} {+} {1y} {=} {2}
which can be simplified to:
{x} {=} {3} {y} {=} {2}
which, together, provide the solution to the original equations.
Solve this problem using the row reduction technique:
A farmer has a total of {11} animals, a mix of chickens and cows. One day he counts the legs of all of his animals and realizes there’s a total of {34} legs. How many chickens and how many cows does the farmer have?
We can describe this problem using a system of linear equations like this:
{ x} {+} { y} {=} {11} {2x} {+} {4y} {=} {34} where: { \; \left\lbrace \vphantom{ \eqalign { x \cr x \cr } } \right. \space }
{x} {\;=} number of chickens {y} {\;=} number of cows
Recall the techniques used for row reduction:
Row operations:
- Interchange any two rows.
- Multiply all the entries in a row by a nonzero constant.
- Add or subtract a multiple of one row to another row.
Row reduction strategy:
- Work column by column, doing the columns in any order that’s convenient.
- For each column, first change one entry to {1,} then change all the rest to
{0}s. - Never put two
{1}s in the same row on the left side of the vertical bar.- For extra speed, start with columns that already have a {1} or a {0.}
- For extra speed, don’t bother interchanging any rows.
In this section, we’ll solve the row reduction exercise from the previous section, but in a more generalized form, by introducing the variables {A} and {L} to represent the numbers. Here’s the new version of the problem:
A farmer has a total of {A} animals, a mix of chickens and cows. One day he counts the legs of all of his animals and realizes there’s a total of {L} legs. How many chickens and how many cows does the farmer have?
Here’s the system of linear equations that describes this problem:
{ x} {+} { y} {=} {A} {2x} {+} {4y} {=} {L} where: { \; \left\lbrace \vphantom{ \eqalign { x \cr x \cr x \cr x \cr } } \right. \space }
{x} {\;=} number of chickens {y} {\;=} number of cows {A} {\;=} total number of animals {L} {\;=} total number of legs
We’ll also take this opportunity to write the augmented matrix in an abbreviated way that minimizes the amount of writing you’ll need to do. You can solve it a little faster if you write it this way:
{x \quad} {y \quad} {1 \quad} {1 \quad} { A \quad } {[ 1]} write the first equation’s row {2 \quad} {4 \quad} { L \quad } {[ 2]} write the second equation’s row {2 \quad} {2 \quad} { 2A \quad } {[ 3]} multiply {[ 1]} by {2} so subtraction can zero out the {2} {0 \quad} {2 \quad} { L - 2A \quad } {[ 4]} subtract {[ 3]} from {[ 2]} to zero out the {2} for {x} {0 \quad} {1 \quad} { {\Large{L \over 2}} - A \quad } {[ 5]} divide {[ 4]} by {2} to get a {1} for {y}
Every time you rewrite a row, be sure to cross out the old row that
it replaces. Also, cross out any temporary rows
After finishing step {[ 5],} this is our current matrix (which you don’t need to write down):
{ \left[\hspace+2pt \begin{array}{rr|c} \hfil 1 &\hfil\hspace-5pt 1 & A \cr \hfil 0 &\hfil\hspace-5pt 1 & {\Large{L \over 2}} - A \cr \end{array} \hspace+2pt\right] } (from line {[ 1]} above)
(from line {[ 5]} above)
We haven’t completely finished isolating both variables, but notice that we’ve found a solution for {y} already:
{y = {\Large{L \over 2}} - A }
At this point, we can simply back substitute {y} into to
the first equation to obtain a solution for
{x + y = A}
🡇
{x = A - y}
We’ve now got a solution for both {x} and {y,} so we’re done.
Notice that we were able to solve the problem
even though we never did isolate {x} in the matrix, and we ever
did figure out exactly how to calculate {x} directly from {A} and {L.}
In the previous section, we stopped the row reduction process early and used back substitution to find the value of {x.} That shortcut saved us some time.
If you take this shortcut to the extreme, you can significantly reduce
the number of
For example, here is the result of performing several row reduction steps on a system of four linear equations:
{ \hspace8pt w \hspace18pt x \hspace19pt y \hspace20pt z }{ \left[\hspace+2pt \begin{array}{rrrr|r} \hfil 2 &\hfil\hspace-3pt -6\phantom{0} &\hfil\hspace-3pt 12\phantom{0} &\hfil\hspace-3pt -3 \hspace+2px &\hfil\hspace-0pt -14 \cr \hfil 6 &\hfil\hspace-3pt -10\phantom{0} &\hfil\hspace-3pt 15\phantom{0} &\hfil\hspace-3pt 0 \hspace+2px &\hfil\hspace-0pt -3 \cr \hfil 1 &\hfil\hspace-3pt -2\phantom{0} &\hfil\hspace-3pt \phantom{-} 4\phantom{0} &\hfil\hspace-3pt -1 \hspace+2px &\hfil\hspace-0pt -4 \cr \hfil 5 &\hfil\hspace-3pt -8\phantom{0} &\hfil\hspace-3pt 16\phantom{0} &\hfil\hspace-3pt -1 \hspace+2px &\hfil\hspace-0pt -2 \cr \end{array} \hspace+2pt\right] }🡇 { \left[\hspace+2pt \begin{array}{rrrr|r} \hfil 2\phantom{0} &\hfil\hspace-3pt -2\phantom{0} &\hfil\hspace-3pt 4\phantom{0} &\hfil\hspace-3pt -1 \hspace+2px &\hfil\hspace+0pt -2 \cr \hfil 0\phantom{0} &\hfil\hspace-3pt 4\phantom{0} &\hfil\hspace-3pt -3\phantom{0} &\hfil\hspace-3pt 3 \hspace+2px &\hfil\hspace+0pt \phantom{-} 21 \cr \hfil 0\phantom{0} &\hfil\hspace-3pt 0\phantom{0} &\hfil\hspace-3pt 5\phantom{0} &\hfil\hspace-3pt -2 \hspace+2px &\hfil\hspace+0pt -3 \cr \hfil 0\phantom{0} &\hfil\hspace-3pt 0\phantom{0} &\hfil\hspace-3pt 0\phantom{0} &\hfil\hspace-3pt 1 \hspace+2px &\hfil\hspace+0pt 4 \cr \end{array} \hspace+2pt\right] }
Looking at the resulting matrix, we clearly didn’t achieve the goal of isolating just one variable per equation. So the values of most of the variables are not yet obvious.
Nevertheless, we can immediately see that {z = 4} by looking at the bottom row.
And now that we know {z = 4}, it’s relatively easy to solve for {y} by back substituting {z} with {4} in the third row, and solving {5y - 2z = -3} to find that {y = 1.}
The reason we found {z} and {y} so quickly is because there are numerous
Notice that those
That triangular pattern of
For example, if we look at the general triangular matrix for four variables:
{ \hspace9pt w \hspace15pt x \hspace14pt y \hspace15pt z }{ \left[\hspace+2pt \begin{array}{cccc|c} a_{\large{1}} & b_{\large{1}} & c_{\large{1}} & d_{\large{1}} & e_{\large{1}} \cr 0 & b_{\large{2}} & c_{\large{2}} & d_{\large{2}} & e_{\large{2}} \cr 0 & 0 & c_{\large{3}} & d_{\large{3}} & e_{\large{3}} \cr 0 & 0 & 0 & d_{\large{4}} & e_{\large{4}} \cr \end{array} \hspace+2pt\right] }
we can apply the following chain of back substitutions to obtain the solution:
[[ z = { {e_{\large{4}}} \over {d_{\large{4}}} } ]]
[[ y = { {e_{\large{3}} - d_{\large{3}}z } \over {c_{\large{3}}} } ]]
[[ x = { {e_{\large{2}} - d_{\large{2}}z - c_{\large{2}}y } \over {b_{\large{2}}} } ]]
[[ w = { {e_{\large{1}} - d_{\large{1}}z - c_{\large{1}}y - b_{\large{1}}x } \over {a_{\large{1}}} } ]]
However, if you don’t want to do any back substitutions, then you can continue to perform more row operations on the matrix until you finally obtain a reduced matrix, like this:
{ \hspace9pt w \hspace15pt x \hspace14pt y \hspace15pt z }{ \left[\hspace+2pt \begin{array}{cccc|c} \hspace2pt 1 &\hspace5pt 0 &\hspace5pt 0 &\hspace5pt 0 \hspace2pt &\hspace3pt ? \hspace2pt \cr \hspace2pt 0 &\hspace5pt 1 &\hspace5pt 0 &\hspace5pt 0 \hspace2pt &\hspace3pt ? \hspace2pt \cr \hspace2pt 0 &\hspace5pt 0 &\hspace5pt 1 &\hspace5pt 0 \hspace2pt &\hspace3pt ? \hspace2pt \cr \hspace2pt 0 &\hspace5pt 0 &\hspace5pt 0 &\hspace5pt 1 \hspace2pt &\hspace3pt ? \hspace2pt \cr \end{array} \hspace+2pt\right] }
You can take either approach, depending on which way is the fastest for your particular matrix.
When you apply the row reduction technique on a system of linear equations, you might encounter one of these two cases:
1. A row with all zeros.
For example, let’s perform row reduction on this augmented matrix:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 5 &\hfil 3 \cr \hfil 2 &\hfil\hspace-5pt 10 &\hfil 6 \cr \end{array} \hspace+2pt\right] }
To solve this system, we’ll need to change the {2} to a {0.} We can do this by multiplying the first row by {2} and subtracting it from the second row like this:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 5 &\hfil 3 \cr \hfil 2 &\hfil\hspace-5pt 10 &\hfil 6 \cr \end{array} \hspace+2pt\right] } 🡄 subtract { { \mathbf{[}\hspace+2pt \begin{array}{rr|r} \hfil 2 &\hfil\hspace-5pt 10 &\hfil 6 \cr \end{array} \hspace+2pt\mathbf{]} } } (which is {2} times the first row) 🡇 { \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 5 &\hfil 3 \cr \hfil 0 &\hfil\hspace-5pt 0 &\hfil 0 \cr \end{array} \hspace+2pt\right] } 🡄 changed The second row now contains all
{0}s, so it represents this linear equation:{0x + 0y = 0}
which, clearly, can be satisfied by every pair of numbers {x} and {y.}
Since the second row provides us with no additional information to identify a single solution, we must conclude that this system has an infinite number of solutions, all of them lying along the line {x + 5y = 3.}
2. A row with all zeros, except for a non-zero in the rightmost column.
For example, let’s perform row reduction on this augmented matrix:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 5 &\hfil 3 \cr \hfil 2 &\hfil\hspace-5pt 10 &\hfil 7 \cr \end{array} \hspace+2pt\right] }
To solve this system, we’ll need to change the {2} to a {0.} We can do this by multiplying the first row by {2} and subtracting it from the second row like this:
{ \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 5 &\hfil 3 \cr \hfil 2 &\hfil\hspace-5pt 10 &\hfil 7 \cr \end{array} \hspace+2pt\right] } 🡄 subtract { { \mathbf{[}\hspace+2pt \begin{array}{rr|r} \hfil 2 &\hfil\hspace-5pt 10 &\hfil 6 \cr \end{array} \hspace+2pt\mathbf{]} } } (which is {2} times the first row) 🡇 { \left[\hspace+2pt \begin{array}{rr|r} \hfil 1 &\hfil\hspace-5pt 5 &\hfil 3 \cr \hfil 0 &\hfil\hspace-5pt 0 &\hfil 1 \cr \end{array} \hspace+2pt\right] } 🡄 changed The second row represents this linear equation:
{0x + 0y = 1}
which, clearly, cannot be satisfied by any pair of numbers {x} and {y.}
If row reduction produces an always-false equation like this, it shows that the system of equations has no solution.
Cramer’s rule
Let’s look at the general system of linear equations in two variables:
{Ax} {+} {By} {=} {C} {Dx} {+} {Ey} {=} {F} It’s interesting to solve this system for {x} and {y} using the row reduction technique. When we do, we find that:
[[ x = {{CE - BF} \over {AE - BD}} ]] and [[ y = {{AF - CD} \over {AE - BD}} ]] This result is called Cramer’s rule.
Notice that all the numerators and denominators have a similar “cross multiplication” pattern over two different columns:
the denominator
{AE-BD}
has this pattern:![]()
the {x} numerator
{CE-BF}
has this pattern:![]()
the {y} numerator
{AF-CD}
has this pattern:![]()
Determinant
In the previous section on Cramer’s rule, we found that the solution to this pair of equations:
{Ax} {+} {By} {=} {C} {Dx} {+} {Ey} {=} {F} is:
[[ x = {{CE - BF} \over {AE - BD}} ]] and [[ y = {{AF - CD} \over {AE - BD}} ]]It’s interesting that these formulas have a strong recurring pattern that involves calculating the difference of two “cross multiplications”.
This pattern occurs so frequently in linear algebra that we have a special name for it: the determinant.
The determinant of the matrix { \hspace+2pt \left[\hspace+2pt \begin{array}{rr|r} \hfil a &\hfil\hspace-5pt b \cr \hfil c &\hfil\hspace-5pt d \cr \end{array} \hspace+2pt\right] \space } is { \space ad - bc. }The determinant is an operator, and it’s commonly written either like this:
{ \det \left[\hspace+2pt \begin{array}{rr|r} \hfil a &\hfil\hspace-5pt b \cr \hfil c &\hfil\hspace-5pt d \cr \end{array} \hspace+2pt\right] = ad - bc }or like this:
{ \left| \hspace+2pt \begin{array}{rr|r} \hfil a &\hfil\hspace-5pt b \cr \hfil c &\hfil\hspace-5pt d \cr \end{array} \hspace+2pt \right| = ad - bc }The determinant allows us to define Cramer’s rule in a way that better preserves the visual structure of the coefficients in the equations:
{ \left. \eqalign { \hfil Ax &\hfil + &\hfil By &\hfil = &\hfil C \cr \hfil Dx &\hfil + &\hfil Ey &\hfil = &\hfil F \cr } \space \right\rbrace } is solved by: [[ x = { { \vphantom{ \large{ \left| \begin{array} 0 & 0 \cr 0 & 0 \end{array} \right| } } { -{ \left| \hspace+2pt \begin{array}{rr|r} \hfil B &\hfil\hspace-5pt C \cr \hfil E &\hfil\hspace-5pt F \cr \end{array} \hspace+2pt \right| } } } \over { \vphantom{ \large{ \left| \begin{array} 0 & 0 \cr 0 & 0 \end{array} \right| } } { \phantom{-}{ \left| \hspace+2pt \begin{array}{rr|r} \hfil A &\hfil\hspace-5pt B \cr \hfil D &\hfil\hspace-5pt E \cr \end{array} \hspace+2pt \right| } } } } ]] and [[ y = { { \vphantom{ \large{ \left| \begin{array} 0 & 0 \cr 0 & 0 \end{array} \right| } } \left| \hspace+2pt \begin{array}{rr|r} \hfil A &\hfil\hspace-5pt C \cr \hfil D &\hfil\hspace-5pt F \cr \end{array} \hspace+2pt \right| } \over { \vphantom{ \large{ \left| \begin{array} 0 & 0 \cr 0 & 0 \end{array} \right| } } \left| \hspace+2pt \begin{array}{rr|r} \hfil A &\hfil\hspace-5pt B \cr \hfil D &\hfil\hspace-5pt E \cr \end{array} \hspace+2pt \right| } } ]]
As an exercise, solve this system of linear equations using the determinant version of Cramer’s rule:
{2x} {+} {4y} {=} {14} {5x} {+} {3y} {=} {21} Using matrix multiplication: introduction
It’s possible to use matrix multiplication to solve a system of linear equations. The next six sections will show how to do this.
First, in this section, we’ll see how a system of linear equations can be written using matrix multiplication notation.
Let’s start with a single linear equation:
{x - 5y + 2z \ = \ 1}
This equation can be written equivalently in vector dot product notation like this:
{(1, -5, 2) \cdot (x, y, z) \ = \ 1}
Matrix multiplication is also capable of performing this dot product, but it uses a different notation:
{ \mathbf{[}\hspace+2pt \begin{matrix} \hfil 1 &\hfil\hspace-5pt -5 &\hfil\hspace-5pt 2 \cr \end{matrix} \hspace+2pt\mathbf{]} \hspace-3pt \left[\hspace+2pt \begin{matrix} x \cr y \cr z \cr \end{matrix} \hspace+2pt\right] = \ \mathbf{[}\hspace+2pt 1 \hspace+2pt\mathbf{]} }
The advantage of the matrix notation is that it can be extended to multiple equations in a concise way. For example, these three linear equations:
{ \eqalign { \hfil x &\hfil - &\hfil 5y &\hfil + &\hfil 2z &\hfil \ = &\hfil \, 1 \cr \hfil 4x &\hfil + &\hfil y &\hfil - &\hfil 6z &\hfil \ = &\hfil \, 4 \cr \hfil 2x &\hfil + &\hfil 3y &\hfil + &\hfil z &\hfil \ = &\hfil \, 19 \cr } }
can be written together in matrix multiplication notation like this:
{ \left[\hspace+2pt \begin{matrix} \hfil 1 &\hfil\hspace-5pt -5 &\hfil\hspace-5pt 2 \cr \hfil 4 &\hfil\hspace-5pt 1 &\hfil\hspace-5pt -6 \cr \hfil 2 &\hfil\hspace-5pt 3 &\hfil\hspace-5pt 1 \cr \end{matrix} \hspace+2pt\right] \hspace-3pt \left[\hspace+2pt \begin{matrix} x \cr y \cr z \cr \end{matrix} \hspace+2pt\right] = \left[\hspace+2pt \begin{matrix} \hfil 1 \cr \hfil 4 \cr \hfil 19 \cr \end{matrix} \hspace+2pt\right] }
It turns out that these equations are satisfied when {x=5,} {y=2,} and {z=3,} which we can write in matrix notation like this:
{ \left[\hspace+2pt \begin{matrix} x \cr y \cr z \cr \end{matrix} \hspace+2pt\right] = \left[\hspace+2pt \begin{matrix} \hfil 5 \cr \hfil 2 \cr \hfil 3 \cr \end{matrix} \hspace+2pt\right] }
We can verify this solution by substituting the variables, performing the matrix multiplication, and observing that the equation is true:
{ \left[\hspace+2pt \begin{matrix} \hfil 1 &\hfil\hspace-5pt -5 &\hfil\hspace-5pt 2 \cr \hfil 4 &\hfil\hspace-5pt 1 &\hfil\hspace-5pt -6 \cr \hfil 2 &\hfil\hspace-5pt 3 &\hfil\hspace-5pt 1 \cr \end{matrix} \hspace+2pt\right] \hspace-3pt \left[\hspace+2pt \begin{matrix} 5 \cr 2 \cr 3 \cr \end{matrix} \hspace+2pt\right] = \left[\hspace+2pt \begin{matrix} \hfil 1 \cr \hfil 4 \cr \hfil 19 \cr \end{matrix} \hspace+2pt\right] \hspace+40pt \color{#008800} \text{Note:} \left\lbrace \hspace+3pt \eqalign { (1, -5, 2) \cdot (5, \ 2, \ 3) = & \hfil 1 \cr (4, 1, -6) \cdot (5, \ 2, \ 3) = & \hfil 4 \cr (2, \hspace+4pt 3, \hspace+4pt 1) \cdot (5, \ 2, \ 3) = & \hfil 19\cr } \right. }
Exercise: Perform the following matrix multiplication:
{ \left[\hspace+2pt \begin{matrix} \hfil 2 &\hfil\hspace-5pt -3 \cr \hfil 1 &\hfil\hspace-5pt 6 \cr \end{matrix} \hspace+2pt\right] \hspace-3pt \left[\hspace+2pt \begin{matrix} 5 \cr 4 \cr \end{matrix} \hspace+2pt\right] }Using matrix multiplication: the identity matrix
As you know, if a number {x} is multiplied by {1,} the result is unchanged:
{1 \times x = x}
This can also occur in matrix multiplication. An identity matrix is a matrix that, when multiplied by another matrix, produces a result that leaves the other matrix unchanged.
There’s a unique identity matrix for each dimension {D}:
{ D = 1\mathord{:} \hspace+13pt \mathbf{[}\hspace+2pt \begin{matrix} \hfil 1 \cr \end{matrix} \hspace+2pt\mathbf{]} \mathbf{[}\hspace+2pt \begin{matrix} x \cr \end{matrix} \hspace+2pt\mathbf{]} = \mathbf{[}\hspace+2pt \begin{matrix} x \cr \end{matrix} \hspace+2pt\mathbf{]} }
{ D = 2\mathord{:} \hspace+10pt \left[\hspace+2pt \begin{matrix} \hfil 1 &\hfil\hspace-5pt 0 \cr \hfil 0 &\hfil\hspace-5pt 1 \cr \end{matrix} \hspace+2pt\right] \hspace-3pt \left[\hspace+2pt \begin{matrix} x \cr y \cr \end{matrix} \hspace+2pt\right] = \left[\hspace+2pt \begin{matrix} x \cr y \cr \end{matrix} \hspace+2pt\right] }
{ D = 3\mathord{:} \hspace+10pt \left[\hspace+2pt \begin{matrix} \hfil 1 &\hfil\hspace-5pt 0 &\hfil\hspace-5pt 0 \cr \hfil 0 &\hfil\hspace-5pt 1 &\hfil\hspace-5pt 0 \cr \hfil 0 &\hfil\hspace-5pt 0 &\hfil\hspace-5pt 1 \cr \end{matrix} \hspace+2pt\right] \hspace-3pt \left[\hspace+2pt \begin{matrix} x \cr y \cr z \cr \end{matrix} \hspace+2pt\right] = \left[\hspace+2pt \begin{matrix} x \cr y \cr z \cr \end{matrix} \hspace+2pt\right] }
{ D = 4\mathord{:} \hspace+10pt \left[\hspace+2pt \begin{matrix} \hfil 1 &\hfil\hspace-5pt 0 &\hfil\hspace-5pt 0 &\hfil\hspace-5pt 0 \cr \hfil 0 &\hfil\hspace-5pt 1 &\hfil\hspace-5pt 0 &\hfil\hspace-5pt 0 \cr \hfil 0 &\hfil\hspace-5pt 0 &\hfil\hspace-5pt 1 &\hfil\hspace-5pt 0 \cr \hfil 0 &\hfil\hspace-5pt 0 &\hfil\hspace-5pt 0 &\hfil\hspace-5pt 1 \cr \end{matrix} \hspace+2pt\right] \hspace-3pt \left[\hspace+2pt \begin{matrix} w \cr x \cr y \cr z \cr \end{matrix} \hspace+2pt\right] = \left[\hspace+2pt \begin{matrix} w \cr x \cr y \cr z \cr \end{matrix} \hspace+2pt\right] }etc.
An identity matrix is a square {D \mathord{\times} D} matrix that contains
{1}s along the main diagonal that extends from the upper left corner to the lower right corner, and contains{0}s everywhere else.Each row vector of an identity matrix is a unit vector that points in the positive direction of one of the axes in
{D}-dimensional space. For example, on the{2}-dimensional plane, {(1, 0)} is the unit vector that points in the direction of the positive {x} axis, and {(0, 1)} is the unit vector that points in the direction of the positive {y} axis. Therefore, the two row vectors of the {2 \mathord{\times} 2} identity matrix are {\mathbf{[}\hspace+2pt 1 \ \ 0 \hspace+2pt\mathbf{]}} and {\mathbf{[}\hspace+2pt 0 \ \ 1 \hspace+2pt\mathbf{]}.}Using matrix multiplication: inverse matrix
If the product of two square matrices is the identity matrix, then the two matrices are inverses of each other.Here’s an example of two matrices that are inverses of each other:
{ \left[\hspace+2pt \begin{matrix} \hfil 2 &\hfil\hspace-2pt 1 \cr \hfil 6 &\hfil\hspace-2pt 4 \cr \end{matrix} \hspace+2pt\right] } and { \left[\hspace+2pt \begin{matrix} \hfil 2 &\hfil\hspace-5pt -{1 \over 2} \cr \hfil -3 &\hfil\hspace-5pt 1 \hspace+1.5pt \cr \end{matrix} \hspace+2pt\right] }
If we multiply them together (in either order), the result is the identity matrix:
{ \left[\hspace+2pt \begin{matrix} \hfil 2 &\hfil\hspace-2pt 1 \cr \hfil 6 &\hfil\hspace-2pt 4 \cr \end{matrix} \hspace+2pt\right] \hspace-3pt \left[\hspace+2pt \begin{matrix} \hfil 2 &\hfil\hspace-5pt -{1 \over 2} \cr \hfil -3 &\hfil\hspace-5pt 1 \hspace+1.5pt \cr \end{matrix} \hspace+2pt\right] = \left[\hspace+2pt \begin{matrix} \hfil 1 &\hfil\hspace-2pt 0 \cr \hfil 0 &\hfil\hspace-2pt 1 \cr \end{matrix} \hspace+2pt\right] }
{ \left[\hspace+2pt \begin{matrix} \hfil 2 &\hfil\hspace-5pt -{1 \over 2} \cr \hfil -3 &\hfil\hspace-5pt 1 \hspace+1.5pt \cr \end{matrix} \hspace+2pt\right] \hspace-3pt \left[\hspace+2pt \begin{matrix} \hfil 2 &\hfil\hspace-2pt 1 \cr \hfil 6 &\hfil\hspace-2pt 4 \cr \end{matrix} \hspace+2pt\right] = \left[\hspace+2pt \begin{matrix} \hfil 1 &\hfil\hspace-2pt 0 \cr \hfil 0 &\hfil\hspace-2pt 1 \cr \end{matrix} \hspace+2pt\right] }Here’s the inverse of a general {2 \mathord{\times} 2} matrix:
Given a {2 \mathord{\times} 2} matrix {M}:{ M = \left[\hspace+2pt \begin{matrix} \hfil a &\hfil\hspace-2pt b \cr \hfil c &\hfil\hspace-2pt d \cr \end{matrix} \hspace+2pt\right] }
The inverse of {M} is called {M^{\large -1},} and is defined as:[[ M^{\large -1} \ = \ { {1} \over { \det M } } \left[\hspace+2pt \begin{matrix} \hfil d &\hfil\hspace-2pt -b \cr \hfil -c &\hfil\hspace-2pt a \cr \end{matrix} \hspace+2pt\right] ]]
where:{ \det M \ = \ \det \left[\hspace+2pt \begin{array}{rr|r} \hfil a &\hfil\hspace-2pt b \cr \hfil c &\hfil\hspace-2pt d \cr \end{array} \hspace+2pt\right] \ = \ ad - bc }
If {I} is the {2 \mathord{\times} 2} identity matrix:
{ I = \left[\hspace+2pt \begin{matrix} \hfil 1 &\hfil\hspace-2pt 0 \cr \hfil 0 &\hfil\hspace-2pt 1 \cr \end{matrix} \hspace+2pt\right] }
then
{ M M^{\large -1} = I }
and{ M^{\large -1} M = I }
If the determinant of a matrix is {0,} then it does not have an inverse.
In matrix mathematics, there is no division operator that divides one matrix by another. To emulate division, we can multiply by the inverse of a matrix. For example, we can emulate {\text{“}}matrix {A} divided by matrix {B \hspace+1pt \text{”}} by calculating {A B^{\large -1}.}
Exercise: Recall the method for calculating the inverse of a {2 \mathord{\times} 2} matrix:
If { M = \left[\hspace+2pt \begin{matrix} \hfil a &\hfil\hspace-2pt b \cr \hfil c &\hfil\hspace-2pt d \cr \end{matrix} \hspace+2pt\right] } then [[ M^{\large -1} \ = \ { {1} \over { \det M } } \left[\hspace+2pt \begin{matrix} \hfil d &\hfil\hspace-2pt -b \cr \hfil -c &\hfil\hspace-2pt a \cr \end{matrix} \hspace+2pt\right] ]] where: { \ \det M = ad - bc }Calculate the inverse of this matrix:
{ \left[\hspace+2pt \begin{matrix} \hfil -3 &\hfil\hspace-2pt 2 \cr \hfil 8 &\hfil\hspace-2pt -5 \cr \end{matrix} \hspace+2pt\right] }