1.1 Systems of Linear Equations

Definitions

• Linear Equation: A linear equation in the variables $x_1,x_2,\dots ,x_n$ is an equation that can be written in the form:

$$a_1{x}_1+a_2{x}_2+\cdots +a_n{x}_n=b$$

where $b$ and the coefficients $a_1,\dots ,a_n$ are real or complex numbers ($a_1,\dots ,a_n,b\in \mathbb{C}$).

Examples:

• $4x_1-5x_2+2=x_1$

• $x_1=2(\sqrt{6}-x_1)+x_3$

• System of Linear Equations (Linear System): A collection of one or more linear equations involving the same variables $x_1,x_2,\dots ,x_n$:

$$\begin{array}{rl}{a}_{11}{x}_1+a_{12}{x}_2+\cdots +a_{1n}{x}_n& =b_1\\ a_{21}{x}_1+a_{22}{x}_2+\cdots +a_{2n}{x}_n& =b_2\\ & ⋮\\ a_{m1}{x}_1+a_{m2}{x}_2+\cdots +a_{mn}{x}_n& =b_m\end{array}$$

• Solution & Solution Set: A solution of a linear system is a list $(s_1,s_2,\dots ,s_n)$ of numbers that satisfies all equations simultaneously when $x_1=s_1,\dots ,x_n=s_n$. The set of all possible solutions is called the solution set.

• Equivalent Systems: Two linear systems are called equivalent if they have the same solution set.

---

Number of Solutions

A system of linear equations has either:

1. No solution (inconsistent system)
2. Exactly one solution (consistent system)
3. Infinitely many solutions (consistent system)

---

Matrix Notation

For a system such as:

$$\begin{array}{rl}{x}_1-2x_2+x_3& =0\\ 2x_2-8x_3& =8\\ 5x_1-5x_3& =10\end{array}$$

• Coefficient Matrix ($A$):

$$A=\left[\begin{array}{ccc}1& -2& 1\\ 0& 2& -8\\ 5& 0& -5\end{array}\right]$$

• Augmented Matrix ($\tilde{A}$):

$$\tilde{A}=\left[\begin{array}{cccc}1& -2& 1& 0\\ 0& 2& -8& 8\\ 5& 0& -5& 10\end{array}\right]$$

• Matrix Size: Expressed as $m\times n$, where $m$ is the number of rows and $n$ is the number of columns.

---

Operations to Simplify a System

To solve a system, we replace it with an equivalent, simpler system using three basic operations:

1. Replacement: Replace one equation/row by the sum of itself and a multiple of another equation/row.
2. Interchange: Interchange two equations/rows.
3. Scaling: Multiply all terms in an equation/row by a nonzero constant ($c\in \mathbb{C}\setminus \{0\}$).

Elementary Row Operations

Applied directly to matrix rows:

1. Replacement
2. Interchange
3. Scaling

---

Example 1: Solving a Linear System

Solve the system:

$$\begin{array}{rl}{x}_1-2x_2+x_3& =0\\ 2x_2-8x_3& =8\\ 5x_1-5x_3& =10\end{array}$$

Step-by-step Row Operations on Augmented Matrix:

$$\tilde{A}=\left[\begin{array}{cccc}1& -2& 1& 0\\ 0& 2& -8& 8\\ 5& 0& -5& 10\end{array}\right]$$

1. $R_3\leftarrow R_3-5R_1$:

$$\left[\begin{array}{cccc}1& -2& 1& 0\\ 0& 2& -8& 8\\ 0& 10& -10& 10\end{array}\right]$$

2. $R_2\leftarrow \frac{1}{2}{R}_2$:

$$\left[\begin{array}{cccc}1& -2& 1& 0\\ 0& 1& -4& 4\\ 0& 10& -10& 10\end{array}\right]$$

3. $R_3\leftarrow R_3-10R_2$:

$$\left[\begin{array}{cccc}1& -2& 1& 0\\ 0& 1& -4& 4\\ 0& 0& 30& -30\end{array}\right]$$

4. $R_3\leftarrow \frac{1}{30}{R}_3$:

$$\left[\begin{array}{cccc}1& -2& 1& 0\\ 0& 1& -4& 4\\ 0& 0& 1& -1\end{array}\right]$$

5. $R_2\leftarrow R_2+4R_3$ and $R_1\leftarrow R_1-R_3$:

$$\left[\begin{array}{cccc}1& -2& 0& 1\\ 0& 1& 0& 0\\ 0& 0& 1& -1\end{array}\right]$$

6. $R_1\leftarrow R_1+2R_2$:

$$\left[\begin{array}{cccc}1& 0& 0& 1\\ 0& 1& 0& 0\\ 0& 0& 1& -1\end{array}\right]$$

Solution: $x_1=1$, $x_2=0$, $x_3=-1$, or written as a point $(1,0,-1)$.

---

Geometric Explanation

• In 3-dimensional space, each linear equation in three variables represents a plane.
• The solution $(1,0,-1)$ represents the unique point where all three planes intersect.

---

Fundamental Questions: Existence and Uniqueness

1. Existence: Is the system consistent? (Does at least one solution exist?)
2. Uniqueness: If a solution exists, is it unique? (Is it the only one?)

Example 2: Inconsistent System

Determine if the following system is consistent:

$$\begin{array}{rl}{x}_2-4x_3& =8\\ 2x_1-3x_2+2x_3& =1\\ 4x_1-8x_2+12x_3& =1\end{array}$$

Row Reduction:

$$\left[\begin{array}{cccc}0& 1& -4& 8\\ 2& -3& 2& 1\\ 4& -8& 12& 1\end{array}\right]\stackrel{R_1\leftrightarrow R_2}{\to }\left[\begin{array}{cccc}2& -3& 2& 1\\ 0& 1& -4& 8\\ 4& -8& 12& 1\end{array}\right]$$ $$\stackrel{R_3\leftarrow R_3-2R_1}{\to }\left[\begin{array}{cccc}2& -3& 2& 1\\ 0& 1& -4& 8\\ 0& -2& 8& -1\end{array}\right]\stackrel{R_3\leftarrow R_3+2R_2}{\to }\left[\begin{array}{cccc}2& -3& 2& 1\\ 0& 1& -4& 8\\ 0& 0& 0& 15\end{array}\right]$$

The last row corresponds to $0x_1+0x_2+0x_3=15$, or $0=15$, which is impossible.
Thus, the system is inconsistent (has no solution). Geometrically, there is no point common to all three planes.

---

1.2 Row Reduction and Echelon Forms

Echelon Form Definitions

A rectangular matrix is in Echelon Form (or Row Echelon Form - REF) if:

1. All nonzero rows are above any rows of all zeros.
2. Each leading entry of a row is in a column to the right of the leading entry of the row above it.
3. All entries in a column below a leading entry are zeros.

A matrix is in Reduced Echelon Form (or Reduced Row Echelon Form - RREF) if it satisfies the REF conditions plus:

1. The leading entry in each nonzero row is $1$.
2. Each leading $1$ is the only nonzero entry in its column.

Key Properties

• Any nonzero matrix can be row reduced into echelon form.
• Row operations and intermediate echelon forms are not unique.
• Theorem 1 (Uniqueness of Reduced Echelon Form): Each matrix is row equivalent to one and only one reduced echelon matrix.

---

Pivot Positions and Pivot Columns

• Pivot Position: A location in a matrix $A$ that corresponds to a leading 1 in the reduced echelon form of $A$.
• Pivot Column: A column of $A$ that contains a pivot position.

Example 3: Locating Pivot Columns

Row reduce $A$ to echelon form and identify pivot columns:

$$A=\left[\begin{array}{ccccc}0& -3& -6& 4& 9\\ -1& -2& -1& 3& 1\\ -2& -3& 0& 3& -1\\ 1& 4& 5& -9& -7\end{array}\right]$$

1. Swap $R_1\leftrightarrow R_4$ to bring a nonzero entry to top left:

$$\left[\begin{array}{ccccc}1& 4& 5& -9& -7\\ -1& -2& -1& 3& 1\\ -2& -3& 0& 3& -1\\ 0& -3& -6& 4& 9\end{array}\right]$$

2. Create zeros below the first pivot ($R_2+R_1,R_3+2R_1$):

$$\left[\begin{array}{ccccc}1& 4& 5& -9& -7\\ 0& 2& 4& -6& -6\\ 0& 5& 10& -15& -15\\ 0& -3& -6& 4& 9\end{array}\right]$$

3. Clear entries below the second pivot ($R_2$):

$$\left[\begin{array}{ccccc}1& 4& 5& -9& -7\\ 0& 2& 4& -6& -6\\ 0& 0& 0& 0& 0\\ 0& 0& 0& -5& 0\end{array}\right]\stackrel{R_3\leftrightarrow R_4}{\to }\left[\begin{array}{ccccc}1& 4& 5& -9& -7\\ 0& 2& 4& -6& -6\\ 0& 0& 0& -5& 0\\ 0& 0& 0& 0& 0\end{array}\right]$$

• Pivot Positions: Row 1 Col 1, Row 2 Col 2, Row 3 Col 4.
• Pivot Columns: Columns 1, 2, and 4.

---

The Row Reduction Algorithm

1. Step 1: Begin with the leftmost nonzero column. This is a pivot column. The pivot position is at the top.
2. Step 2: Select a nonzero entry in the pivot column as a pivot. Interchange rows if necessary.
3. Step 3: Use row replacement operations to create zeros in all positions below the pivot.
4. Step 4: Apply Steps 1–3 to the submatrix that remains. Repeat until there are no more nonzero rows to modify (yields REF).
5. Step 5: Working from the rightmost pivot upward and to the left, create zeros above each pivot. Scale pivots to 1 (yields RREF).

---

Solutions of Linear Systems

• Basic Variables: Variables corresponding to pivot columns.
• Free Variables: Variables corresponding to non-pivot columns. Free variables can take any arbitrary scalar value.

Example 4: General Solution with Free Variables

Given a matrix already in reduced echelon form:

$$\left[\begin{array}{cccccc}1& 6& 0& 3& 0& 0\\ 0& 0& 1& -4& 0& 5\\ 0& 0& 0& 0& 1& 7\end{array}\right]$$

• Basic variables: $x_1,x_3,x_5$
• Free variables: $x_2,x_4$

General Solution (Parametric Description):

$$\{\begin{array}{l}{x}_1=-6x_2-3x_4\\ x_2\text{ is free}\\ x_3=5+4x_4\\ x_4\text{ is free}\\ x_5=7\end{array}$$

---

Existence and Uniqueness Theorem

Theorem 2: A linear system is consistent if and only if the rightmost column of the augmented matrix is not a pivot column — that is, if and only if an echelon form has no row of the form:

$$[0\dots 0b]\text{with }b\ne 0$$

If a linear system is consistent, the solution set contains either:

1. A unique solution (when there are no free variables).
2. Infinitely many solutions (when there is at least one free variable).

---

Systematic Procedure to Solve a System

1. Write the augmented matrix $\tilde{A}$.
2. Reduce $\tilde{A}$ to an echelon form.
3. Check for rows of the form $[0\dots 0b]$ ($b\ne 0$). If present, the system is inconsistent (STOP).
4. If consistent, continue row reduction to Reduced Echelon Form (RREF).
5. Write the system of equations corresponding to the RREF matrix.
6. Express each basic variable in terms of any free variables.

---

Example 5: Parameter Analysis

Discuss solutions to the system dependent on parameter $a$:

$$\begin{array}{rl}a{x}_1+x_2+x_3& =1\\ x_1+a{x}_2+x_3& =a\\ x_1+x_2+a{x}_3& =a^2\end{array}$$

Augmented Matrix Reduction:

$$\tilde{A}=\left[\begin{array}{cccc}a& 1& 1& 1\\ 1& a& 1& a\\ 1& 1& a& a^2\end{array}\right]\stackrel{R_1\leftrightarrow R_3}{\to }\left[\begin{array}{cccc}1& 1& a& a^2\\ 1& a& 1& a\\ a& 1& 1& 1\end{array}\right]$$ $$\stackrel{\text{Row ops}}{\to }\left[\begin{array}{cccc}1& 1& a& a^2\\ 0& a-1& 1-a& a-a^2\\ 0& 0& 2-a-a^2& 1+a-a^2-a^3\end{array}\right]$$

Analysis:

1. Case 1: $a\ne 1$ and $a\ne -2$.
   No zero rows in coefficient positions; 3 pivots for 3 variables $\to$ Unique solution.
2. Case 2: $a=1$.
   Reduces to a single equation $x_1+x_2+x_3=1$ $\to$ 2 free variables $\to$ Infinitely many solutions.
3. Case 3: $a=-2$.
   The last row becomes $[0003]$, which means $0=3$ $\to$ Inconsistent.

---

1.3 Vector Equations

Vectors in ${\mathbb{R}}^2$ and ${\mathbb{R}}^3$

• Column Vector: A matrix with only one column.
• Vectors in ${\mathbb{R}}^2$:

$$u=\left[\begin{array}{c}{u}_1\\ u_2\end{array}\right]$$

• Equality: Two vectors are equal if and only if their corresponding entries are equal.

• Operations:

  • Addition:

$$\left[\begin{array}{c}1\\ -2\end{array}\right]+\left[\begin{array}{c}2\\ 5\end{array}\right]=\left[\begin{array}{c}3\\ 3\end{array}\right]$$

• Scalar Multiplication:

$$c\cdot \left[\begin{array}{c}3\\ -1\end{array}\right]=\left[\begin{array}{c}3c\\ -c\end{array}\right]$$

Geometric Representation

• Parallelogram Rule for Addition: If $u$ and $v$ in ${\mathbb{R}}^2$ are represented as points in the plane, then $u+v$ corresponds to the fourth vertex of the parallelogram whose other vertices are $u$, $0$, and $v$.

---

Vectors in ${\mathbb{R}}^n$

${\mathbb{R}}^n$ denotes the collection of all $n\times 1$ column matrices of real (or complex) numbers:

$$u=\left[\begin{array}{c}{u}_1\\ u_2\\ ⋮\\ u_n\end{array}\right]$$

Algebraic Properties of ${\mathbb{R}}^n$

For all vectors $u,v,w\in {\mathbb{R}}^n$ and scalars $c,d$:

1. $u+v=v+u$
2. $(u+v)+w=u+(v+w)$
3. $u+0=0+u=u$
4. $u+(-u)=-u+u=0$
5. $c(u+v)=cu+cv$
6. $(c+d)u=cu+du$
7. $c(du)=(cd)u$
8. $1u=u$

---

Linear Combinations

Given vectors $v_1,v_2,\dots ,v_p\in {\mathbb{R}}^n$ and scalars $c_1,c_2,\dots ,c_p$, the vector $y$ defined by:

$$y=c_1{v}_1+c_2{v}_2+\cdots +c_p{v}_p$$

is called a linear combination of $v_1,\dots ,v_p$ with weights $c_1,\dots ,c_p$.

Fundamental Fact

A vector equation:

$$x_1{a}_1+x_2{a}_2+\cdots +x_n{a}_n=b$$

has the same solution set as the linear system whose augmented matrix is:

$$\tilde{A}=\left[\begin{array}{ccccc}{a}_1& a_2& \dots & a_n& b\end{array}\right]$$

In particular, $b$ can be generated by a linear combination of $a_1,\dots ,a_n$ if and only if the linear system corresponding to $\tilde{A}$ is consistent.

---

Span of a Set of Vectors

Definition: The subset of ${\mathbb{R}}^n$ spanned (or generated) by $v_1,\dots ,v_p$ is denoted by $\text{Span}\{v_1,\dots ,v_p\}$, defined as the collection of all vectors that can be written as:

$$c_1{v}_1+c_2{v}_2+\cdots +c_p{v}_p$$

where $c_1,\dots ,c_p$ are scalars.

Equivalent Questions

The following three questions are equivalent:

1. Is a vector $b$ in $\text{Span}\{v_1,\dots ,v_p\}$?
2. Does the vector equation $x_1{v}_1+\cdots +x_p{v}_p=b$ have a solution?
3. Does the linear system with augmented matrix $[v_1{v}_2\dots v_{p}b]$ have a solution?

Geometric Description

• $\text{Span}\{v\}$: A line through the origin in ${\mathbb{R}}^3$ (if $v\ne 0$).
• $\text{Span}\{u,v\}$: A plane through the origin in ${\mathbb{R}}^3$ (if $u,v$ are non-zero and not multiples of each other).

---

Self-Checking Questions

1. What are the three elementary row operations for a system of linear equations?

   • Replacement, Interchange, and Scaling.

2. In the expression of a linear system with $n$ variables, what does $a_{ij}$ denote? What do $i$ and $j$ represent respectively?

   • $a_{ij}$ denotes the coefficient of variable $x_j$ in equation $i$. $i$ represents the row (equation) index, and $j$ represents the column (variable) index.

3. If a matrix is denoted by $A=[a_{ij}{]}_{m\times n}$, then this matrix has ______ rows and ______ columns.

   • $m$ rows and $n$ columns.

4. What is the difference between a homogeneous system and a non-homogeneous system of linear equations?

   • A homogeneous system has all constant terms equal to zero ($b=0$), whereas a non-homogeneous system has at least one non-zero constant term ($b\ne 0$).

5. If a system contains a contradictory (or inconsistent) equation, how does it appear in the corresponding echelon form?

   • As a row of the form $[00\dots 0b]$ where $b\ne 0$.

6. If a system has infinitely many solutions, what feature does its corresponding echelon form have?

   • It is consistent and has at least one free variable (a column without a pivot position).