Linear algebra

From Wikiversity

Material covered in these notes are designed to span over 12-16 weeks. Each subpage will contain about 3 hours of material to read through carefully, and additional time to properly absorb the material.

1 Introduction - linear equations
2 Solving Linear Systems Algebraically
- 2.1 Substitution
3 Thinking in terms of matrices
4 Matrices
5 Types of Matrices
6 Matrix addition
7 Multiplication by a scalar
8 Multiplication of matrices
9 Transpose of a matrix
10 Determinant of a matrix
11 Inverse of a matrix
12 Eigenvalues and eigenvectors
13 Resources
- 13.1 Wikipedia
- 13.2 Wikibooks
14 External links

[edit] Introduction - linear equations

Let us illustrate through examples what linear equations are. We will also be introducing new notation wherever appropriate.

For example:

3 x - y = 14

2 x + y = 11

If you add these two equations together, you can see that the y's cancel each other out. When this happens, you will get $5 x = 25$ , or $x = 5$ . Substituting back into the above, we find that y = 1. Note that this is the only solution to the system of equations. The above method of solving was linear combination, or elimination.

[edit] Solving Linear Systems Algebraically

One was mentioned above, but there are other ways to solve a system of linear equations without graphing.

[edit] Substitution

If you get a system of equations that looks like this:

2 x + y = 11

- 4 x + 3 y = 13

You can switch around some terms in the first to get this:

y = - 2 x + 11

Then you can substitute that into the bottom one so that it looks like this:

- 4 x + 3( - 2 x + 11) = 13

- 4 x - 6 x + 33 = 13

- 10 x + 33 = 13

- 10 x = - 20

x = 2

Then, you can substitute 2 into an x from either equation and solve for y. It's usually easier to substitute it in the one that had the single y. In this case, after substituting 2 for x, you would find that y = 7.

[edit] Thinking in terms of matrices

Much of finite elements revolves around forming matrices and solving systems of linear equations using matrices. This learning resource gives you a brief review of matrices.

[edit] Matrices

Suppose that you have a linear system of equations

$\begin{align} a_{11} x_1 + a_{12} x_2 + a_{13} x_3 + a_{14} x_4 &= b_1 \\ a_{21} x_1 + a_{22} x_2 + a_{23} x_3 + a_{24} x_4 &= b_2 \\ a_{31} x_1 + a_{32} x_2 + a_{33} x_3 + a_{34} x_4 &= b_3 \\ a_{41} x_1 + a_{42} x_2 + a_{43} x_3 + a_{44} x_4 &= b_4 ~. \end{align}$

Matrices provide a simple way of expressing these equations. Thus, we can instead write

$\begin{bmatrix} a_{11} & a_{12} & a_{13} & a_{14} \\ a_{21} & a_{22} & a_{23} & a_{24} \\ a_{31} & a_{32} & a_{33} & a_{34} \\ a_{41} & a_{42} & a_{43} & a_{44} \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \\ b_4 \end{bmatrix} ~.$

An even more compact notation is

$\left[\mathsf{A}\right] \left[\mathsf{x}\right] = \left[\mathsf{b}\right] ~~~~\text{or}~~~~ \mathbf{A} \mathbf{x} = \mathbf{b} ~.$

Here $\mathbf{A}$ is a $4\times 4$ matrix while $\mathbf{x}$ and $\mathbf{b}$ are $4\times 1$ matrices. In general, an $m \times n$ matrix $\mathbf{A}$ is a set of numbers arranged in $m$ rows and $n$ columns.

$\mathbf{A} = \begin{bmatrix} a_{11} & a_{12} & a_{13} & \dots & a_{1n} \\ a_{21} & a_{22} & a_{23} & \dots & a_{2n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & a_{m3} & \dots & a_{mn} \end{bmatrix}~.$

[edit] Types of Matrices

Common types of matrices that we encounter in finite elements are:

a row vector that has one row and $n$ columns.

$\mathbf{v} = \begin{bmatrix} v_1 & v_2 & v_3 & \dots & v_n \end{bmatrix}$

a column vector that has $n$ rows and one column.

$\mathbf{v} = \begin{bmatrix} v_1 \\ v_2 \\ v_3 \\ \vdots \\ v_n \end{bmatrix}$

a square matrix that has an equal number of rows and columns.
a diagonal matrix which is a square matrix with only the

diagonal elements ( $a i i$ ) nonzero.

$\mathbf{A} = \begin{bmatrix} a_{11} & 0 & 0 & \dots & 0 \\ 0 & a_{22} & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \dots & a_{nn} \end{bmatrix}~.$

the identity matrix ( $\mathbf{I}$ ) which is a diagonal matrix and

with each of its nonzero elements ( $a i i$ ) equal to 1.

$\mathbf{A} = \begin{bmatrix} 1 & 0 & 0 & \dots & 0 \\ 0 & 1 & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \dots & 1 \end{bmatrix}~.$

a symmetric matrix which is a square matrix with elements

such that $a i j = a j i$ .

$\mathbf{A} = \begin{bmatrix} a_{11} & a_{12} & a_{13} & \dots & a_{1n} \\ a_{12} & a_{22} & a_{23} & \dots & a_{2n} \\ a_{13} & a_{23} & a_{33} & \dots & a_{3n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{1n} & a_{2n} & a_{3n} & \dots & a_{nn} \end{bmatrix}~.$

a skew-symmetric matrix which is a square matrix with elements

such that $a i j = - a j i$ .

$\mathbf{A} = \begin{bmatrix} a_{11} & a_{12} & a_{13} & \dots & a_{1n} \\ -a_{12} & a_{22} & a_{23} & \dots & a_{2n} \\ -a_{13} & -a_{23} & a_{33} & \dots & a_{3n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ -a_{1n} & -a_{2n} & -a_{3n} & \dots & a_{nn} \end{bmatrix}~.$

Note that the diagonal elements of a skew-symmetric matrix have to be zero: $a_{ii} = -a_{ii} \Rightarrow a_{ii} = 0$ .

[edit] Matrix addition

Let $\mathbf{A}$ and $\mathbf{B}$ be two $m \times n$ matrices with components $a i j$ and $b i j$ , respectively. Then

$\mathbf{C} = \mathbf{A} + \mathbf{B} \implies c_{ij} = a_{ij} + b_{ij}$

[edit] Multiplication by a scalar

Let $\mathbf{A}$ be a $m \times n$ matrix with components $a i j$ and let $λ$ be a scalar quantity. Then,

$\mathbf{C} = \lambda\mathbf{A} \implies c_{ij} = \lambda a_{ij}$

[edit] Multiplication of matrices

Let $\mathbf{A}$ be a $m \times n$ matrix with components $a i j$ . Let $\mathbf{B}$ be a $p \times q$ matrix with components $b i j$ .

The product $\mathbf{C} = \mathbf{A} \mathbf{B}$ is defined only if $n = p$ . The matrix $\mathbf{C}$ is a $m \times q$ matrix with components $c i j$ . Thus,

$\mathbf{C} = \mathbf{A} \mathbf{B} \implies c_{ij} = \sum^n_{k=1} a_{ik} b_{kj}$

Similarly, the product $\mathbf{D} = \mathbf{B} \mathbf{A}$ is defined only if $q = m$ . The matrix $\mathbf{D}$ is a $p \times n$ matrix with components $d i j$ . We have

$\mathbf{D} = \mathbf{B} \mathbf{A} \implies d_{ij} = \sum^m_{k=1} b_{ik} a_{kj}$

Clearly, $\mathbf{C} \ne \mathbf{D}$ in general, i.e., the matrix product is not commutative.

However, matrix multiplication is distributive. That means

$\mathbf{A} (\mathbf{B} + \mathbf{C}) = \mathbf{A} \mathbf{B} + \mathbf{A} \mathbf{C} ~.$

The product is also associative. That means

$\mathbf{A} (\mathbf{B} \mathbf{C}) = (\mathbf{A} \mathbf{B}) \mathbf{C} ~.$

[edit] Transpose of a matrix

Let $\mathbf{A}$ be a $m \times n$ matrix with components $a i j$ . Then the transpose of the matrix is defined as the $n \times m$ matrix $\mathbf{B} = \mathbf{A}^T$ with components $b i j = a j i$ . That is,

$\mathbf{B} = \mathbf{A}^T = \begin{bmatrix} a_{11} & a_{12} & a_{13} & \dots & a_{1n} \\ a_{21} & a_{22} & a_{23} & \dots & a_{2n} \\ a_{31} & a_{32} & a_{33} & \dots & a_{3n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & a_{m3} & \dots & a_{mn} \end{bmatrix}^T = \begin{bmatrix} a_{11} & a_{21} & a_{31} & \dots & a_{m1} \\ a_{12} & a_{22} & a_{32} & \dots & a_{m2} \\ a_{13} & a_{23} & a_{33} & \dots & a_{m3} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{1n} & a_{2n} & a_{3n} & \dots & a_{mn} \end{bmatrix}$

A important identity involving the transpose of matrices is

${ (\mathbf{A} \mathbf{B})^T = \mathbf{B}^T \mathbf{A}^T }~.$

[edit] Determinant of a matrix

The determinant of a matrix is defined only for square matrices.

For a $2 \times 2$ matrix $\mathbf{A}$ , we have

$\mathbf{A} = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} \implies \det(\mathbf{A}) = \begin{vmatrix} a_{11} & a_{12} \\ a_{21} & a_{22}\end{vmatrix} = a_{11} a_{22} - a_{12} a_{21} ~.$

For a $n \times n$ matrix, the determinant is calculated by expanding into minors as

$\begin{align} &\det(\mathbf{A}) = \begin{vmatrix} a_{11} & a_{12} & a_{13} & \dots & a_{1n} \\ a_{21} & a_{22} & a_{23} & \dots & a_{2n} \\ a_{31} & a_{32} & a_{33} & \dots & a_{3n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & a_{n3} & \dots & a_{nn} \end{vmatrix} \\ &= a_{11} \begin{vmatrix} a_{22} & a_{23} & \dots & a_{2n} \\ a_{32} & a_{33} & \dots & a_{3n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n2} & a_{n3} & \dots & a_{nn} \end{vmatrix} - a_{12} \begin{vmatrix} a_{21} & a_{23} & \dots & a_{2n} \\ a_{31} & a_{33} & \dots & a_{3n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n3} & \dots & a_{nn} \end{vmatrix} + \dots \pm a_{1n} \begin{vmatrix} a_{21} & a_{23} & \dots & a_{2(n-1)} \\ a_{31} & a_{33} & \dots & a_{3(n-1)} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n3} & \dots & a_{n(n-1)} \end{vmatrix} \end{align}$

In short, the determinant of a matrix $\mathbf{A}$ has the value

${ \det(\mathbf{A}) = \sum^n_{i=1} (-1)^{i+j} a_{ij} M_{ij} }$

where $M i j$ is the determinant of the submatrix of $\mathbf{A}$ formed by eliminating row $i$ and column $j$ from $\mathbf{A}$ .

Some useful identities involving the determinant are given below.

If $\mathbf{A}$ is a $n \times n$ matrix, then

$\det(\mathbf{A}) = \det(\mathbf{A}^T)~.$

If $λ$ is a constant and $\mathbf{A}$ is a $n \times n$ matrix, then

$\det(\lambda\mathbf{A}) = \lambda^n\det(\mathbf{A}) \implies \det(-\mathbf{A}) = (-1)^n\det(\mathbf{A}) ~.$

If $\mathbf{A}$ and $\mathbf{B}$ are two $n \times n$ matrices, then

$\det(\mathbf{A}\mathbf{B}) = \det(\mathbf{A})\det(\mathbf{B})~.$

[edit] Inverse of a matrix

Let $\mathbf{A}$ be a $n \times n$ matrix. The inverse of $\mathbf{A}$ is denoted by $\mathbf{A}^{-1}$ and is defined such that

${ \mathbf{A} \mathbf{A}^{-1} = \mathbf{I} }$

where $\mathbf{I}$ is the $n \times n$ identity matrix.

The inverse exists only if $\det(\mathbf{A}) \ne 0$ . A singular matrix does not have an inverse.

An important identity involving the inverse is

${ (\mathbf{A}\mathbf{B})^{-1} = \mathbf{B}^{-1} \mathbf{A}^{-1}, }$

since this leads to: ${ (\mathbf{A} \mathbf{B})^{-1} (\mathbf{A} \mathbf{B}) = (\mathbf{B}^{-1} \mathbf{A}^{-1}) (\mathbf{A} \mathbf{B} ) = \mathbf{B}^{-1} \mathbf{A}^{-1} \mathbf{A} \mathbf{B} = \mathbf{B}^{-1} (\mathbf{A}^{-1} \mathbf{A}) \mathbf{B} = \mathbf{B}^{-1} \mathbf{I} \mathbf{B} = \mathbf{B}^{-1} \mathbf{B} = \mathbf{I}. }$

Some other identities involving the inverse of a matrix are given below.

The determinant of a matrix is equal to the inverse of the

determinant of its inverse.

$\det(\mathbf{A}) = \cfrac{1}{\det(\mathbf{A}^{-1})}~.$

The determinant of a similarity transformation of a matrix

is equal to the original matrix.

$\det(\mathbf{B} \mathbf{A} \mathbf{B}^{-1}) = \det(\mathbf{A}) ~.$

We usually use numerical methods such as Gaussian elimination to compute the inverse of a matrix.

[edit] Eigenvalues and eigenvectors

Let $\mathbf{A}$ be a $n \times n$ matrix, let $\mathbf{x}$ be a $n \times 1$ vector, and let $λ$ be a scalar. Consider the situation where the effect of the action of matrix $\mathbf{A}$ on vector $\mathbf{x}$ is to stretch (or shrink) the vector by an amount $λ$ . That is,

$\mathbf{A} \mathbf{x} = \lambda \mathbf{x}~.$

There are $n$ ways of achieving this transformation, each involving a non-zero vector $\mathbf{x}_i$ and a non-zero scalar $λ i$ . The vectors $\mathbf{x}_i$ are called the eigenvectors of $\mathbf{A}$ and the corresponding scalars $λ i$ are called the eigenvalues of $\mathbf{A}$ .

To compute the eigenvalues and eigenvectors of a matrix $\mathbf{A}$ , we rearrange the equation so that we have

$\text{(1)} \qquad (\mathbf{A} -\lambda \mathbf{I})\mathbf{x} = \mathbf{0}~.$

This equation has nontrivial solutions only if

$\det(\mathbf{A} -\lambda \mathbf{I}) = 0~.$

The determinant can be expressed as a polynomial of degree $n$ . The eigenvalues are the roots of this polynomial. Once we have the eigenvalues, the corresponding eigenvectors can be calculated by plugging the eigenvalues into equation (1). We usually normalize the eigenvectors so that they are unit vectors.

The eigenvalues of a real symmetric matrix are real, and its eigenvectors are orthogonal.

[edit] Resources

[edit] Wikipedia

[edit] Wikibooks

Linear Algebra

[edit] External links