Chapter 1 • Theory & Derivations
Unit 1: Vectors in R^n & C^n, Systems of Linear Equations & Applications
Foundations of Euclidean and complex vector spaces, geometry of R^n and C^n, Cauchy-Schwarz and Minkowski inequalities, Gaussian elimination and Gauss-Jordan reduction to Reduced Row Echelon Form (RREF), consistency criteria, and applied network flow, electrical networks, chemical balancing, and polynomial interpolation.
§1.1Vectors in Euclidean R^n and Complex C^n Spaces
### 1. Vectors in Real Euclidean Space $\mathbb{R}^n$
Let $n$ be a positive integer. The set of all ordered $n$-tuples of real numbers is denoted by $\mathbb{R}^n$:
$$\mathbb{R}^n = \left\{ \vec{x} = \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix} : x_i \in \mathbb{R}, \; i = 1, 2, \dots, n \right\}$$
Vectors in $\mathbb{R}^n$ admit two fundamental algebraic operations:
1. **Vector Addition:** For $\vec{x}, \vec{y} \in \mathbb{R}^n$,
$$\vec{x} + \vec{y} = \begin{pmatrix} x_1 + y_1 \\ x_2 + y_2 \\ \vdots \\ x_n + y_n \end{pmatrix}$$
2. **Scalar Multiplication:** For $c \in \mathbb{R}$ and $\vec{x} \in \mathbb{R}^n$,
$$c\vec{x} = \begin{pmatrix} c x_1 \\ c x_2 \\ \vdots \\ c x_n \end{pmatrix}$$
#### The Standard Euclidean Inner Product (Dot Product)
The standard inner product of $\vec{x}, \vec{y} \in \mathbb{R}^n$ is the scalar:
$$\vec{x} \cdot \vec{y} = \langle \vec{x}, \vec{y} \rangle = \sum_{i=1}^n x_i y_i = \vec{x}^T \vec{y}$$
The **Euclidean Norm (Length)** of $\vec{x}$ is defined by:
$$\|\vec{x}\| = \sqrt{\vec{x} \cdot \vec{x}} = \sqrt{\sum_{i=1}^n x_i^2}$$
The **Euclidean Distance Metric** between two points $\vec{x}, \vec{y} \in \mathbb{R}^n$ is:
$$d(\vec{x}, \vec{y}) = \|\vec{x} - \vec{y}\| = \sqrt{\sum_{i=1}^n (x_i - y_i)^2}$$
---
### 2. Fundamental Inequalities in $\mathbb{R}^n$
#### Theorem 1.1 (The Cauchy-Schwarz Inequality):
For any vectors $\vec{x}, \vec{y} \in \mathbb{R}^n$:
$$|\vec{x} \cdot \vec{y}| \le \|\vec{x}\| \|\vec{y}\|$$
*Rigorous Proof:*
If $\vec{y} = \vec{0}$, both sides are $0$ and the equality holds trivially. Assume $\vec{y} \ne \vec{0}$. For any real scalar $t \in \mathbb{R}$, consider the squared norm:
$$0 \le \|\vec{x} - t\vec{y}\|^2 = (\vec{x} - t\vec{y}) \cdot (\vec{x} - t\vec{y}) = \|\vec{x}\|^2 - 2t(\vec{x} \cdot \vec{y}) + t^2 \|\vec{y}\|^2$$
This is a non-negative quadratic polynomial $q(t) = A t^2 + B t + C$ where $A = \|\vec{y}\|^2 > 0$, $B = -2(\vec{x} \cdot \vec{y})$, and $C = \|\vec{x}\|^2$. Since $q(t) \ge 0$ for all $t \in \mathbb{R}$, its discriminant $\Delta = B^2 - 4AC$ must be non-positive:
$$\Delta = 4(\vec{x} \cdot \vec{y})^2 - 4 \|\vec{y}\|^2 \|\vec{x}\|^2 \le 0 \implies (\vec{x} \cdot \vec{y})^2 \le \|\vec{x}\|^2 \|\vec{y}\|^2$$
Taking the positive square root of both sides yields $|\vec{x} \cdot \vec{y}| \le \|\vec{x}\| \|\vec{y}\|$. Equality holds if and only if $\vec{x}$ and $\vec{y}$ are linearly dependent. $\blacksquare$
#### Theorem 1.2 (The Triangle / Minkowski Inequality):
For any $\vec{x}, \vec{y} \in \mathbb{R}^n$:
$$\|\vec{x} + \vec{y}\| \le \|\vec{x}\| + \|\vec{y}\|$$
*Proof:*
Expanding the squared norm using the Cauchy-Schwarz inequality:
$$\|\vec{x} + \vec{y}\|^2 = (\vec{x} + \vec{y}) \cdot (\vec{x} + \vec{y}) = \|\vec{x}\|^2 + 2(\vec{x} \cdot \vec{y}) + \|\vec{y}\|^2 \le \|\vec{x}\|^2 + 2\|\vec{x}\|\|\vec{y}\| + \|\vec{y}\|^2 = (\|\vec{x}\| + \|\vec{y}\|)^2$$
Taking square roots on both sides establishes the inequality. $\blacksquare$
---
### 3. Vectors in Complex Space $\mathbb{C}^n$
The set of ordered $n$-tuples of complex numbers is denoted by $\mathbb{C}^n$:
$$\mathbb{C}^n = \left\{ \vec{z} = \begin{pmatrix} z_1 \\ z_2 \\ \vdots \\ z_n \end{pmatrix} : z_k = a_k + i b_k \in \mathbb{C}, \; a_k, b_k \in \mathbb{R} \right\}$$
For $\vec{z} \in \mathbb{C}^n$, its **complex conjugate** is $\overline{\vec{z}} = (\overline{z}_1, \dots, \overline{z}_n)^T$, and its **conjugate transpose (Hermitian adjoint)** is:
$$\vec{z}^* = \vec{z}^H = \overline{\vec{z}}^T = (\overline{z}_1, \overline{z}_2, \dots, \overline{z}_n)$$
#### Standard Inner Product in $\mathbb{C}^n$
To ensure that the norm of a non-zero complex vector is always a positive real number, the inner product in $\mathbb{C}^n$ is defined with complex conjugation:
$$\langle \vec{u}, \vec{v} \rangle = \vec{v}^* \vec{u} = \sum_{k=1}^n u_k \overline{v}_k$$
This inner product satisfies:
1. **Conjugate Symmetry:** $\langle \vec{v}, \vec{u} \rangle = \overline{\langle \vec{u}, \vec{v} \rangle}$
2. **Linearity in First Argument:** $\langle \alpha \vec{u}_1 + \beta \vec{u}_2, \vec{v} \rangle = \alpha \langle \vec{u}_1, \vec{v} \rangle + \beta \langle \vec{u}_2, \vec{v} \rangle$
3. **Conjugate Linearity in Second Argument:** $\langle \vec{u}, \alpha \vec{v}_1 + \beta \vec{v}_2 \rangle = \overline{\alpha} \langle \vec{u}, \vec{v}_1 \rangle + \overline{\beta} \langle \vec{u}, \vec{v}_2 \rangle$
4. **Positive Definiteness:** $\langle \vec{z}, \vec{z} \rangle = \sum_{k=1}^n |z_k|^2 \ge 0$, and $\langle \vec{z}, \vec{z} \rangle = 0 \iff \vec{z} = \vec{0}$
The **Complex Euclidean Norm** is $\|\vec{z}\| = \sqrt{\langle \vec{z}, \vec{z} \rangle} = \sqrt{\sum_{k=1}^n |z_k|^2}$, and the distance metric is $d(\vec{u}, \vec{v}) = \|\vec{u} - \vec{v}\| = \sqrt{\sum_{k=1}^n |u_k - v_k|^2}$.
§1.2Systems of Linear Equations, Gaussian Elimination & RREF
### 1. General System of Linear Equations
A general system of $m$ linear equations in $n$ unknowns $x_1, x_2, \dots, x_n$ over a field $F$ ($\mathbb{R}$ or $\mathbb{C}$) is formulated as:
$$\begin{aligned}
a_{11} x_1 + a_{12} x_2 + \cdots + a_{1n} x_n &= b_1 \\
a_{21} x_1 + a_{22} x_2 + \cdots + a_{2n} x_n &= b_2 \\
&\vdots \\
a_{m1} x_1 + a_{m2} x_2 + \cdots + a_{mn} x_n &= b_m
\end{aligned}$$
In compact matrix notation:
$$A\vec{x} = \vec{b}$$
where $A \in M_{m \times n}(F)$ is the **coefficient matrix**, $\vec{x} \in F^n$ is the **unknown vector**, and $\vec{b} \in F^m$ is the **constant vector**:
$$A = \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{pmatrix}, \quad \vec{x} = \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix}, \quad \vec{b} = \begin{pmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{pmatrix}$$
The **Augmented Matrix** is denoted by $[A \mid \vec{b}] \in M_{m \times (n+1)}(F)$.
- If $\vec{b} = \vec{0}$, the system $A\vec{x} = \vec{0}$ is **homogeneous**. It always possesses the trivial solution $\vec{x} = \vec{0}$.
- If $\vec{b} \ne \vec{0}$, the system is **non-homogeneous**.
---
### 2. Elementary Row Operations and Row Equivalence
Two augmented matrices are **row equivalent** if one can be transformed into the other via a finite sequence of **Elementary Row Operations (EROs)**:
1. **Row Swap ($R_i \leftrightarrow R_j$):** Interchange rows $i$ and $j$.
2. **Row Scaling ($R_i \to c R_i$):** Multiply row $i$ by a non-zero scalar $c \ne 0$.
3. **Row Addition ($R_i \to R_i + c R_j$):** Add $c$ times row $j$ to row $i$ ($i \ne j$).
Each ERO corresponds to left multiplication by an invertible **elementary matrix** $E \in M_{m \times m}(F)$. Because elementary matrices are invertible, EROs preserve the solution set of the linear system.
---
### 3. Reduced Row Echelon Form (RREF)
A matrix is in **Row Echelon Form (REF)** if:
1. All zero rows are at the bottom of the matrix.
2. The leading entry (first non-zero entry from the left, called the **pivot**) of each non-zero row is strictly to the right of the leading entry of the row above it.
3. All entries in a column below a leading entry are zero.
A matrix is in **Reduced Row Echelon Form (RREF)** if, in addition:
4. The leading entry (pivot) in each non-zero row is strictly equal to $1$.
5. Each column containing a leading $1$ has zeros in all other entries (both above and below the pivot).
#### Theorem 1.3 (Uniqueness of RREF):
Every matrix $A \in M_{m \times n}(F)$ is row-equivalent to one and only one matrix $R$ in Reduced Row Echelon Form: $R = \text{rref}(A)$.
#### Classification of Variables and Solutions:
Let $R = [R_A \mid \vec{d}] = \text{rref}([A \mid \vec{b}])$ have $r$ non-zero rows (pivots):
- **Pivot Variables (Basic Variables):** Unknowns $x_j$ corresponding to columns of $R_A$ containing a pivot $1$.
- **Free Variables:** Unknowns $x_j$ corresponding to columns without pivots. Total free variables = $n - r$.
#### Theorem 1.4 (The Rouché-Capelli Consistency Theorem):
A linear system $A\vec{x} = \vec{b}$ is **consistent** (has at least one solution) if and only if the augmented matrix $[A \mid \vec{b}]$ has the same rank as the coefficient matrix $A$:
$$\text{rank}(A) = \text{rank}([A \mid \vec{b}])$$
Equivalently, the system is consistent if and only if the last column of $\text{rref}([A \mid \vec{b}])$ does not contain a pivot row of the form $(0, 0, \dots, 0 \mid 1)$.
- **Unique Solution:** Consistent and $\text{rank}(A) = n$ (zero free variables).
- **Infinitely Many Solutions:** Consistent and $\text{rank}(A) = r < n$ ($n - r$ free variables).
- **Inconsistent (No Solution):** $\text{rank}([A \mid \vec{b}]) = \text{rank}(A) + 1$.
§1.3Applied Linear Systems: Networks, Circuits, Chemistry & Interpolation
### 1. Network Flow Analysis
A directed network consists of a set of junctions (nodes) connected by branches (edges) carrying flow rates $x_1, x_2, \dots, x_k$. The governing physical principle is **Kirchhoff's Flow Conservation Law**:
$$\sum \text{Flow In} = \sum \text{Flow Out} \quad \text{at every junction node}$$
For an urban traffic grid or pipeline network with $m$ nodes and external net inputs/outputs $b_i$:
$$\sum_{j \in \text{incoming}(i)} x_j - \sum_{k \in \text{outgoing}(i)} x_k = b_i \quad (i = 1, \dots, m)$$
This yields a linear system $A\vec{x} = \vec{b}$. Overall network feasibility requires total inflow equals total outflow: $\sum_{i=1}^m b_i = 0$.
---
### 2. Electrical Resistor Networks
Consider an electrical circuit containing resistors and DC voltage sources:
1. **Kirchhoff's Current Law (KCL):** The algebraic sum of currents at any node is zero: $\sum I_{\text{in}} = \sum I_{\text{out}}$.
2. **Kirchhoff's Voltage Law (KVL):** The algebraic sum of voltage changes around any closed loop is zero: $\sum V_{\text{sources}} = \sum I R$.
By defining loop currents $I_1, I_2, \dots, I_L$ (Mesh Current Analysis), KVL produces a symmetric, diagonally dominant linear system:
$$\begin{pmatrix} R_{11} & -R_{12} & \cdots \\ -R_{21} & R_{22} & \cdots \\ \vdots & \vdots & \ddots \end{pmatrix} \begin{pmatrix} I_1 \\ I_2 \\ \vdots \end{pmatrix} = \begin{pmatrix} \mathcal{E}_1 \\ \mathcal{E}_2 \\ \vdots \end{pmatrix}$$
Solving via Gauss-Jordan elimination determines all branch currents and power dissipations $P_k = I_k^2 R_k$.
---
### 3. Balancing Chemical Reaction Equations
A chemical reaction preserves the number of atoms of each chemical element. Consider balancing the combustion of propane:
$$x_1 \text{C}_3\text{H}_8 + x_2 \text{O}_2 \to x_3 \text{CO}_2 + x_4 \text{H}_2\text{O}$$
Equating atom counts for Carbon ($\text{C}$), Hydrogen ($\text{H}$), and Oxygen ($\text{O}$):
- $\text{C}: 3x_1 - 1x_3 = 0$
- $\text{H}: 8x_1 - 2x_4 = 0$
- $\text{O}: 2x_2 - 2x_3 - 1x_4 = 0$
This is a homogeneous linear system $A\vec{x} = \vec{0}$ with coefficient matrix:
$$A = \begin{pmatrix} 3 & 0 & -1 & 0 \\ 8 & 0 & 0 & -2 \\ 0 & 2 & -2 & -1 \end{pmatrix}$$
Row reduction to RREF yields:
$$\begin{pmatrix} 1 & 0 & 0 & -1/4 \\ 0 & 1 & 0 & -5/4 \\ 0 & 0 & 1 & -3/4 \end{pmatrix} \implies \begin{cases} x_1 = \frac{1}{4}x_4 \\ x_2 = \frac{5}{4}x_4 \\ x_3 = \frac{3}{4}x_4 \end{cases}$$
Choosing the smallest positive integer for the free variable $x_4 = 4$ gives the unique stoichiometric solution $\vec{x} = (1, 5, 3, 4)^T$:
$$\text{C}_3\text{H}_8 + 5\text{O}_2 \to 3\text{CO}_2 + 4\text{H}_2\text{O}$$
---
### 4. Polynomial Curve Interpolation and the Vandermonde Matrix
Given $n+1$ distinct data points $(t_0, y_0), (t_1, y_1), \dots, (t_n, y_n)$ with $t_i \ne t_j$ for $i \ne j$, there exists a unique polynomial of degree at most $n$:
$$p(t) = c_0 + c_1 t + c_2 t^2 + \cdots + c_n t^n$$
satisfying the interpolation conditions $p(t_i) = y_i$ for all $i = 0, \dots, n$. Setting up the linear system $V\vec{c} = \vec{y}$ reveals the **Vandermonde Matrix** $V$:
$$\begin{pmatrix} 1 & t_0 & t_0^2 & \cdots & t_0^n \\ 1 & t_1 & t_1^2 & \cdots & t_1^n \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & t_n & t_n^2 & \cdots & t_n^n \end{pmatrix} \begin{pmatrix} c_0 \\ c_1 \\ \vdots \\ c_n \end{pmatrix} = \begin{pmatrix} y_0 \\ y_1 \\ \vdots \\ y_n \end{pmatrix}$$
#### Theorem 1.5 (Vandermonde Determinant Identity):
$$\det(V) = \prod_{0 \le j < i \le n} (t_i - t_j)$$
Since the interpolation nodes $t_i$ are pairwise distinct, $t_i - t_j \ne 0$ for all $i > j$. Consequently, $\det(V) \ne 0$, proving that $V$ is invertible and the interpolating polynomial $p(t)$ exists and is strictly unique.
TIERED UNIVERSITY HONORS PROBLEMS
Step-by-Step Solved Examination Problems
Comprehensive analytical derivations, multi-tier solutions (Foundational, Intermediate Exam, and Honors/Proof Challenge) with complete line-by-line verification.
Tier 1 • Foundational
Complete Gauss-Jordan Reduction & Parametric Vector Solution
Solve the following non-homogeneous system of linear equations in four variables over $\mathbb{R}$ using elementary row operations and Gauss-Jordan elimination:
$$\begin{aligned} x_1 - 2x_2 + x_3 + 3x_4 &= 2 \\ 2x_1 - 4x_2 + 3x_3 + 8x_4 &= 7 \\ -x_1 + 2x_2 + 2x_3 + 3x_4 &= 7 \end{aligned}$$
Find the Reduced Row Echelon Form (RREF) of the augmented matrix, identify the pivot and free variables, and express the general solution in parametric vector form.
Tier 2 • Intermediate Exam
Network Flow & Resistor Loop Conservation
A traffic network has four intersections $A, B, C, D$ connected by one-way streets with traffic flows $x_1, x_2, x_3, x_4, x_5$ (measured in vehicles/hour):
- Node $A$: Inflow of $400$ enters; flows $x_1$ (to $B$) and $x_2$ (to $C$) leave.
- Node $B$: Inflow $x_1$ arrives; flows $x_3$ (to $D$) and external outflow of $250$ leave.
- Node $C$: Inflow $x_2$ arrives along with external inflow of $150$; flow $x_4$ (to $D$) and $x_5$ leave.
- Node $D$: Inflows $x_3$ and $x_4$ arrive; external outflow of $300$ leaves.
(a) Set up the linear system balancing inflow and outflow at every intersection.
(b) Solve the system to determine the general flow vector.
(c) If the road segment $CD$ is closed ($x_4 = 0$), determine the required flows on all remaining streets to avoid traffic congestion.
Tier 3 • Honors Challenge
Vandermonde Determinant & Polynomial Interpolation Uniqueness
Prove rigorously by induction on $n \ge 1$ that the determinant of the $(n+1) \times (n+1)$ Vandermonde matrix:
$$V_n = \begin{pmatrix} 1 & t_0 & t_0^2 & \cdots & t_0^n \\ 1 & t_1 & t_1^2 & \cdots & t_1^n \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & t_n & t_n^2 & \cdots & t_n^n \end{pmatrix}$$
satisfies the product formula $\det(V_n) = \prod_{0 \le j < i \le n} (t_i - t_j)$. Then, deduce that if the nodes $t_0, t_1, \dots, t_n \in F$ are distinct, there exists a unique polynomial $p(t) \in F[t]$ of degree $\le n$ passing through any prescribed points $(t_i, y_i)$. Explain the connection to the Lagrange interpolation formula.