In the course of the previous lecture we essentially proved the following theorem:
1) If a
matrix
has
linearly independent real or complex
eigenvectors, the
can be diagonalized.
2) If
is a matrix whose columns are eigenvectors then
is the diagonal matrix of eigenvalues.
The factorization
is called the spectral representation
of
.
We can use the spectral representation to solve a system of linear homogeneous ordinary
differential equations.
For example, we could wish to solve the system
![{\displaystyle {\cfrac {d\mathbf {u} }{dt}}=\mathbf {A} \mathbf {u} ={\begin{bmatrix}-2&1\\1&-2\end{bmatrix}}{\begin{bmatrix}u_{1}\\u_{2}\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/9cdc7f50d868059770b618a5cfdeb53117e9f4a7)
(More generally
could be a
matrix.)
Higher order ordinary differential equations can be reduced to this form. For example,
![{\displaystyle {\cfrac {d^{2}u_{1}}{dt^{2}}}+a~{\cfrac {du_{1}}{dt}}=b~u_{1}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/b6a5857a548f6ca261e25f44726239dfc9a03d6d)
Introduce
![{\displaystyle u_{2}={\cfrac {du_{1}}{dt}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/802f27236c41707d50888ebaa0bd749ba05bc5b0)
Then the system of equations is
![{\displaystyle {\begin{aligned}{\cfrac {du_{1}}{dt}}&=u_{2}\\{\cfrac {du_{2}}{dt}}&=b~u_{1}-a~u_{2}\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/8fc3a388b1ecb0645644f76bb01cf4c0d743c740)
or,
![{\displaystyle {\cfrac {d\mathbf {u} }{dt}}={\begin{bmatrix}0&1\\b&-a\end{bmatrix}}{\begin{bmatrix}u_{1}\\u_{2}\end{bmatrix}}=\mathbf {A} \mathbf {u} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/c3994d4b666ffd3de847990fd18515846fc85bee)
Returning to the original problem, let us find the eigenvalues and eigenvectors of
. The characteristic equation is
![{\displaystyle det(\mathbf {A} -\lambda ~\mathbf {I} )=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/5a5665c9b77a696a8a5cdfd3e6bf165bc8c8067d)
o we can calculate the eigenvalues as
![{\displaystyle (2+\lambda )(2+\lambda )-1=0\quad \implies \quad \lambda ^{2}+4\lambda +3=0\qquad \implies \qquad \lambda _{1}=-1,\qquad \lambda _{2}=-3}](https://wikimedia.org/api/rest_v1/media/math/render/svg/df6fa6ab8a634b4eef73a6fdab74388acb375730)
The eigenvectors are given by
![{\displaystyle (\mathbf {A} -\lambda _{1}~\mathbf {I} )\mathbf {n} _{1}=\mathbf {0} ~;~~(\mathbf {A} -\lambda _{2}~\mathbf {I} )\mathbf {n} _{2}=\mathbf {0} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/122b6afec335f9760d4a892792dcd2c2c1f22ec6)
or,
![{\displaystyle -n_{1}^{1}+n_{2}^{1}=0~;~~n_{1}^{1}-n_{2}^{1}=0~;~~n_{1}^{2}+n_{2}^{2}=0~;~~n_{1}^{2}+n_{2}^{2}=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/bda042da4a7ffcdf8fc961861c64a1eefe143b76)
Possible choices of
and
are
![{\displaystyle \mathbf {n} _{1}={\begin{bmatrix}1\\1\end{bmatrix}}~;~~\mathbf {n} _{2}={\begin{bmatrix}1\\-1\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/162591a08c225dafae310bb4e0cffa5702609ea4)
The matrix
is one whose columd are the eigenvectors of
, i.e.,
![{\displaystyle \mathbf {T} ={\begin{bmatrix}1&1\\1&-1\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/4813344de043de8978e9ea5b0b73d45a4b3a60ad)
and
![{\displaystyle {\boldsymbol {\Lambda }}=\mathbf {T} ^{-1}\mathbf {A} \mathbf {T} ={\begin{bmatrix}-1&0\\0&-3\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/610d8afbe78fabd8545fdb231024c391382f42d2)
If
the system of equations becomes
![{\displaystyle {\cfrac {d\mathbf {u} ^{'}}{dt}}=\mathbf {T} ^{-1}\mathbf {A} \mathbf {T} \mathbf {u} ^{'}={\boldsymbol {\Lambda }}~\mathbf {u} ^{'}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/79b81bcb379517e0b9b8c874b3e99f920afec25c)
Expanded out
![{\displaystyle {\cfrac {du_{1}^{'}}{dt}}=-u_{1}^{'}~;~~{\cfrac {du_{2}^{'}}{dt}}=-3~u_{2}^{'}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d8435f26a0bf91d4ded79e87fe2bf43b22cbbf27)
The solutions of these equations are
![{\displaystyle u_{1}^{'}=C_{1}~e^{-t}~;~~u_{2}^{'}=C_{2}~e^{-3t}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/868831ba43c09059f44253e9b2d9d868814d0f88)
Therefore,
![{\displaystyle \mathbf {u} =\mathbf {T} ~\mathbf {u} ^{'}={\begin{bmatrix}C_{1}~e^{-t}+C_{2}~e^{-3t}\\C_{1}~e^{-t}-C_{2}~e^{-3t}\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c3803a998305dec9f535242af873a4588649594e)
This is the solution of the system of ODEs that we seek.
Most "generic" matrices have linearly independent eigenvectors. Generally a matrix will
have
distinct eigenvalues unless there are symmetries that lead to repeated values.
If
has
distinct eigenvalues then it has
linearly independent eigenvectors.
Proof:
We prove this by induction.
Let
be the eigenvector corresponding to the eigenvalue
. Suppose
are linearly independent (note that this is true for
= 2). The question then becomes: Do there exist
not all zero such that the linear combination
![{\displaystyle \alpha _{1}~\mathbf {n} _{1}+\alpha _{2}~\mathbf {n} _{2}+\dots +\alpha _{k}~\mathbf {n} _{k}=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/6c19f20d85fe2c3ab2b88bda44015d411a85ff31)
Let us multiply the above by
. Then, since
, we have
![{\displaystyle \alpha _{1}~(\lambda _{1}-\lambda _{k})~\mathbf {n} _{1}+\alpha _{2}~(\lambda _{2}-\lambda _{k})~\mathbf {n} _{2}+\dots +\alpha _{k-1}~(\lambda _{k-1}-\lambda _{k})~\mathbf {n} _{k-1}+\alpha _{k}~(\lambda _{k}-\lambda _{k})~\mathbf {n} _{k}=\mathbf {0} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/327b8474c866fc69ee18eb2ac3b4ebe4ce814559)
Since
is arbitrary, the above is true only when
![{\displaystyle \alpha _{1}=\alpha _{2}=\dots =\alpha _{k-1}=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/294efb855a2d05de7006ac45ac5f4610f269d8d6)
In thast case we must have
![{\displaystyle \alpha _{k}~\mathbf {n} _{k}=\mathbf {0} \quad \implies \quad \alpha _{k}=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/3e8b9ebf9225ecd87c14902e91693b473ef750cb)
This leads to a contradiction.
Therefore
are linearly independent.
Another important class of matrices which are diagonalizable are those which are self-adjoint.
If
is self-adjoint the following statements are true
is real for all
.
- All eigenvalues are real.
- Eigenvectors of distinct eigenvalues are orthogonal.
- There is an orthonormal basis formed by the eigenvectors.
- The matrix
can be diagonalized (this is a consequence of the previous statement.)
Proof
1) Because the matrix is self-adjoint we have
![{\displaystyle \langle {\boldsymbol {A}}\mathbf {x} ,\mathbf {x} \rangle =\langle \mathbf {x} ,{\boldsymbol {A}}\mathbf {x} \rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/7035c8f119e4247547346293c0e7fe7be209d41e)
From the property of the inner product we have
![{\displaystyle \langle \mathbf {x} ,{\boldsymbol {A}}\mathbf {x} \rangle ={\overline {\langle {\boldsymbol {A}}\mathbf {x} ,\mathbf {x} \rangle }}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/eae16cb8cd0283c63076c92f80936d01ac9f9b10)
Therefore,
![{\displaystyle \langle {\boldsymbol {A}}\mathbf {x} ,\mathbf {x} \rangle ={\overline {\langle {\boldsymbol {A}}\mathbf {x} ,\mathbf {x} \rangle }}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/63a6c32dfbdb1aacdc3f36fb759773a0cbad5ff0)
which implies that
is real.
2) Since
is real,
is real.
Also, from the eiegnevalue problem, we have
![{\displaystyle \langle {\boldsymbol {A}}\mathbf {x} ,\mathbf {x} \rangle =\lambda ~\langle \mathbf {x} ,\mathbf {x} \rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/dc38f2773d374b1dd09f6d276156fc1f2b4956d8)
Therefore,
is real.
3) If
and
are two eigenpairs then
![{\displaystyle \lambda ~\langle \mathbf {x} ,\mathbf {y} \rangle =\langle {\boldsymbol {A}}\mathbf {x} ,\mathbf {y} \rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/27d554a4d6951edabf6fac305564488a85b38348)
Since the matrix is self-adjoint, we have
![{\displaystyle \lambda ~\langle \mathbf {x} ,\mathbf {y} \rangle =\langle \mathbf {x} ,{\boldsymbol {A}}\mathbf {y} \rangle =\mu ~\langle \mathbf {x} ,\mathbf {y} \rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/8696008cee71fd509137042328c76f0a7873559c)
Therefore, if
, we must have
![{\displaystyle \langle \mathbf {x} ,\mathbf {y} \rangle =0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/91c94233efe3dbc09548f4716e5c44438c7ea8a0)
Hence the eigenvectors are orthogonal.
4) This part is a bit more involved. We need to define a manifold first.
A linear manifold (or vector subspace)
is a subset of
which is
closed under scalar multiplication and vector addition.
Examples are a line through the origin of
-dimensional space, a plane through the origin,
the whole space, the zero vector, etc.
An invariant manifold
for the matrix
is the linear manifold for which
implies
.
Examples are the null space and range of a matrix
. For the case of a rotation
about an axis through the origin in
-space, invaraiant manifolds are the origin, the
plane perpendicular to the axis, the whole space, and the axis itself.
Therefore, if
are a basis for
and
are a basis for
(the perpendicular component of
) then in this basis
has the representation
![{\displaystyle {\boldsymbol {A}}={\begin{bmatrix}x&x&|&x&x\\x&x&|&x&x\\-&-&-&-&-\\0&0&|&x&x\\0&0&|&x&x\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7d4a3769f062140afdf038613460675958069305)
We need a matrix of this form for it to be in an invariant manifold for
.
Note that if
is an invariant manifold of
it does not follow that
is also an invariant manifold.
Now, if
is self adjoint then the entries in the off-diagonal spots must be zero too.
In that case,
is block diagonal in this basis.
Getting back to part (4), we know that there exists at least one eigenpair (
)
(this is true for any matrix). We now use induction. Suppose that we have found (
)
mutually orthogonal eigenvectors
with
and
are real,
. Note that the
s are invariant manifolds of
as
is the space spanned by the
s and so is the manifold perpendicular to these vectors).
We form the linear manifold
![{\displaystyle {\mathcal {M}}_{k}=\{\mathbf {x} |\langle \mathbf {x} ,\mathbf {x} _{j}\rangle =0~~j=1,2,\dots ,k-1\}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/003ecb5970fa1d5ef3dbb31e1fffa2dc2232333a)
This is the orthogonal component of the
eigenvectors
If
then
![{\displaystyle \langle \mathbf {x} ,\mathbf {x} _{j}\rangle =0\quad {\text{and}}\quad \langle {\boldsymbol {A}}\mathbf {x} ,\mathbf {x} _{j}\rangle =\langle \mathbf {x} ,{\boldsymbol {A}}\mathbf {x} _{j}\rangle =\lambda _{j}\langle \mathbf {x} ,\mathbf {x} _{j}\rangle =0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/49ed79f6c680ec91cb702328b2c542bff1a56f8b)
Therefore
which means that
is invariant.
Hence
contains at least one eigenvector
with real eigenvalue
.
We can repeat the procedure to get a diagonal matrix in the lower block of the block
diagonal representation of
. We then get
distinct eigenvectors and so
can
be diagonalized. This implies that the eigenvectors form an orthonormal basis.
5) This follows from the previous result because each eigenvector can be normalized so
that
.
We will explore some more of these ideas in the next lecture.