In the last lecture we talked about norms in inner product spaces. The
induced norm was defined as
![{\displaystyle \lVert \mathbf {x} \rVert ={\sqrt {\langle \mathbf {x} ,\mathbf {x} \rangle }}=\lVert \mathbf {x} \rVert _{2}~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/8839f89687409468d30ae6c3ffc916b255343165)
We also talked about orthonomal bases and biorthonormal bases. The biorthonormal
bases may be thought of as dual bases in the sense that covariant and
contravariant vector bases are dual.
The last thing we talked about was the idea of a linear operator. Recall that
![{\displaystyle {\boldsymbol {A}}~{\boldsymbol {\varphi }}_{j}=\sum _{i}A_{ij}~\varphi _{i}\equiv A_{ij}~\varphi _{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d24c3150f40a45d6e1c54a5f1b8e93a6b503c5fb)
where the summation is on the first index.
In this lecture we will learn about adjoint operators, Jacobi tridiagonalization,
and a bit about the spectral theory of matrices.
Assume that we have a vector space with an orthonormal basis. Then
![{\displaystyle \langle {\boldsymbol {A}}~{\boldsymbol {\varphi }}_{j},{\boldsymbol {\varphi }}_{i}\rangle =A_{kj}~\langle {\boldsymbol {\varphi }}_{k},{\boldsymbol {\varphi }}_{i}\rangle =A_{kj}~\delta _{ki}=A_{ij}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/6b00b1cec2e4f7fffa5eaf52a0379ee1606550b8)
One specific matrix connected with
is the Hermitian conjugate matrix.
This matrix is defined as
![{\displaystyle A_{ij}^{*}={\overline {A}}_{ji}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/fc87f9f9fb5a100fe304504f9dfa4cac47681687)
The linear operator
connected with the Hermitian matrix is called the
adjoint operator and is defined as
![{\displaystyle {\boldsymbol {A}}^{*}={\overline {\boldsymbol {A}}}^{T}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/75c3c4a3fe8dbf70752a9f1c3c4899e33b69e98e)
Therefore,
![{\displaystyle \langle {\boldsymbol {A}}^{*}~{\boldsymbol {\varphi }}_{j},{\boldsymbol {\varphi }}_{i}\rangle ={\overline {A}}_{ji}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/77c1c7333346d27b3276854e9f66615d1c330d48)
and
![{\displaystyle \langle {\boldsymbol {\varphi }}_{i},{\boldsymbol {A}}^{*}~{\boldsymbol {\varphi }}_{j}\rangle =A_{ji}=\langle {\boldsymbol {A}}~{\boldsymbol {\varphi }}_{i},{\boldsymbol {\varphi }}_{j}\rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/15a336e717b6de5ae1436907cf989a71412a7700)
More generally, if
![{\displaystyle \mathbf {f} =\sum _{i}\alpha _{i}~{\boldsymbol {\varphi }}_{i}\qquad {\text{and}}\qquad \mathbf {g} =\sum _{j}\beta _{j}~{\boldsymbol {\varphi }}_{j}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/0c9b7a7b4e8f78e4b910266f488152b7e9dd6238)
then
![{\displaystyle \langle \mathbf {f} ,{\boldsymbol {A}}^{*}~\mathbf {g} \rangle =\sum _{i,j}\alpha _{i}~{\overline {\beta }}_{j}\langle {\boldsymbol {\varphi }}_{i},{\boldsymbol {A}}^{*}~{\boldsymbol {\varphi }}_{j}\rangle =\sum _{i,j}\alpha _{i}~{\overline {\beta }}_{j}\langle {\boldsymbol {A}}~{\boldsymbol {\varphi }}_{i},{\boldsymbol {\varphi }}_{j}\rangle =\langle {\boldsymbol {A}}~\mathbf {f} ,\mathbf {g} \rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/5ba2724e7661cb339f3f1ec5cc04ef308243d292)
Since the above relation does not involve the basis we see that the adjoint operator is also basis independent.
If
we say that
is self-adjoint, i.e.,
in any orthonomal basis, and the matrix
is said to be Hermitian.
A matrix
is anti-Hermitian if
![{\displaystyle B_{ij}=-{\overline {B}}_{ji}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/58862cc69f43341925c02df59ab6fb9a203c0cb0)
There is a close connection between Hermitian and anti-Hermitian matrices.
Consider a matrix
. Then
![{\displaystyle A_{ij}=i~B_{ij}=-i~{\overline {B}}_{ji}={\overline {A}}_{ji}~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/fb0a0704140a68e9faaafb57de17809e703d1566)
Let
be self-adjoint and suppose that we want to solve
![{\displaystyle ({\boldsymbol {I}}-\lambda ~{\boldsymbol {A}})~\mathbf {y} =\mathbf {b} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/63293251bdadb96cdc35ef3785bbbc4054bd8c74)
where
is constant. We expect that
![{\displaystyle \mathbf {y} =({\boldsymbol {I}}-\lambda ~{\boldsymbol {A}})^{-1}~\mathbf {b} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/a22eff90bdad0f5065de6746f9e01e9115046cad)
If
is "sufficiently" small, then
![{\displaystyle \mathbf {y} =({\boldsymbol {I}}+\lambda ~{\boldsymbol {A}}+\lambda ^{2}~{\boldsymbol {A}}^{2}+\lambda ^{3}~{\boldsymbol {A}}^{3}+\dots )~\mathbf {b} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/a9e0e9778589fe159b0d28538a0cf91f77a450d5)
This suggest that the solution should be in the subspace spanned by
.
Let us apply the Gram-Schmidt orthogonalization procedure where
![{\displaystyle \mathbf {x} _{1}=\mathbf {b} ,~\mathbf {x} _{2}={\boldsymbol {A}}~\mathbf {b} ,~\mathbf {x} _{3}={\boldsymbol {A}}^{2}~\mathbf {b} ,~\dots }](https://wikimedia.org/api/rest_v1/media/math/render/svg/4d5e5a9513215502d2c1cc524be111a6411258ab)
Then we have
![{\displaystyle {\boldsymbol {\varphi }}_{n}={\boldsymbol {A}}^{n}~\mathbf {b} -\sum _{j=1}^{n-1}{\cfrac {\langle {\boldsymbol {A}}^{n}~\mathbf {b} ,{\boldsymbol {\varphi }}_{j}\rangle }{\lVert {\boldsymbol {\varphi }}_{j}\rVert ^{2}}}~{\boldsymbol {\varphi }}_{j}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/2030bae8f5faeffb22ce3e3983dc183a4b06a541)
This is clearly a linear combination of
.
Therefore,
is a linear combination of
. This
is the same as saying that
is a linear combination of
.
Therefore,
![{\displaystyle \langle {\boldsymbol {A}}~{\boldsymbol {\varphi }}_{n},{\boldsymbol {\varphi }}_{k}\rangle =0~~{\text{if}}~k>n+1}](https://wikimedia.org/api/rest_v1/media/math/render/svg/531122130d00fab3f32fd07173251cf36757a3b6)
Now,
![{\displaystyle A_{kn}=\langle {\boldsymbol {A}}~{\boldsymbol {\varphi }}_{n},{\boldsymbol {\varphi }}_{k}\rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/91948d259c46f25e5b217fcaadb0d68b863197ef)
But the self-adjointeness of
implies that
![{\displaystyle A_{nk}={\overline {A_{kn}}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/ff6e7baa4f9930b1b36d17b251af74d72a670303)
So
is
or
. This is equivalent to expressing
the operator
as a tridiagonal matrix
which has the form
![{\displaystyle \mathbf {A} ={\begin{bmatrix}x&x&0&\dots &\dots &\dots &0\\x&x&x&\ddots &&&\vdots \\0&x&x&\ddots &\ddots &&\vdots \\\vdots &\ddots &\ddots &\ddots &\ddots &\ddots &\vdots \\\vdots &&\ddots &\ddots &x&x&0\\\vdots &&&\ddots &x&x&x\\0&\dots &\dots &\dots &0&x&x\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/913716832ccb46e7d3ade9493bb816f723ac11a6)
In general, the matrix can be represented in block tridiagonal form.
Another consequence of the Gram-Schmidt orthogonalization is that
Lemma:
Every finite dimensional inner-product space has an orthonormal basis.
Proof:
The proof is trivial. Just use Gram-Schmidt on any basis for that space and
normalize.
A corollary of this is the following theorem.
Theorem:
Every finite dimensional inner product space is complete.
Recall that a space is complete is the limit of any Cauchy sequence from a
subspace of that space must lie within that subspace.
Proof:
Let
be a Cauchy sequence of elements in the subspace
with
. Also let
be an
orthonormal basis for the subspace
.
Then
![{\displaystyle \mathbf {u} _{k}=\sum _{j=1}^{n}\alpha _{kj}~\mathbf {e} _{j}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/af665ccdb56bccb9f1295c766073a146d1c7eaef)
where
![{\displaystyle \alpha _{ki}=\langle \mathbf {u} _{k},\mathbf {e} _{i}\rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/0a0d317caec9757687c7e730078f27ac9a6c91f0)
By the Schwarz inequality
![{\displaystyle |\alpha _{ki}-\alpha _{pi}|=|\langle \mathbf {u} _{k},\mathbf {e} _{i}\rangle -\langle \mathbf {u} _{p},\mathbf {e} _{i}\rangle |\leq \lVert \mathbf {u} _{k}-\mathbf {u} _{p}\rVert \rightarrow 0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/eed93b47f07387c685c8b4d0fd249e962a77156c)
Therefore,
![{\displaystyle \alpha _{ki}-\alpha _{pi}\rightarrow 0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/5dc1632a8f2af3b8ba811fe64f2d35778d4da949)
But the
~s are just numbers. So, for fixed
,
is
a Cauchy sequence in
(or
) and so converges to
a number
as
, i.e.,
![{\displaystyle \lim _{k\rightarrow \infty }\mathbf {u} _{k}=\sum _{i=1}^{n}\alpha _{i}~\mathbf {e} _{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/66037f3ddbec97ca7fbec2bcb44420feab1880cc)
which is is the subspace
.
Suppose
is expressed in coordinates relative to some basis
, i.e.,
![{\displaystyle {\boldsymbol {A}}~{\boldsymbol {\varphi }}_{j}=\sum _{i}A_{ij}~{\boldsymbol {\varphi }}_{i}~;~~\mathbf {x} =\sum _{i}x_{i}~{\boldsymbol {\varphi }}_{i}~;~~\mathbf {b} =\sum _{i}b_{i}~{\boldsymbol {\varphi }}_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/adcf763f649925745dd46bbb69bbee813cfcb688)
Then
![{\displaystyle {\boldsymbol {A}}~\mathbf {x} ={\boldsymbol {A}}~\sum _{j}x_{j}~{\boldsymbol {\varphi }}_{j}=\sum _{j}x_{j}~(A~{\boldsymbol {\varphi }}_{j})=\sum _{i,j}x_{j}~A_{ij}~{\boldsymbol {\varphi }}_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/401d927543deb199f543e8ce5260ebad84fc1801)
So
implies that
![{\displaystyle \sum _{i,j}A_{ij}~x_{j}=b_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/b75893b6325dab64134e4b876d71fe797a069aa8)
Now let us try to see the effect of a change to basis to a new basis
with
![{\displaystyle {\boldsymbol {\varphi }}_{i}^{'}=\sum _{j=1}^{n}C_{ji}~{\boldsymbol {\varphi }}_{j}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7d1401d1348a705174d8433a44b3551949c2f203)
For the new basis to be linearly independent,
should be invertible so
that
![{\displaystyle {\boldsymbol {\varphi }}_{i}=\sum _{j=1}^{n}C_{mj}^{-1}~{\boldsymbol {\varphi }}_{m}^{'}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/06ef5c3dce5579bf08124b98a443d4b79883b838)
Now,
![{\displaystyle \mathbf {x} =\sum _{j}x_{j}~{\boldsymbol {\varphi }}_{j}=\sum _{i,j}x_{j}~C_{ij}^{-1}~{\boldsymbol {\varphi }}_{i}^{'}=\sum _{i}x_{i}^{'}~{\boldsymbol {\varphi }}_{i}^{'}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/5321230bb7291ddb622c1dfd3ef8a5922b9e0dee)
Hence
![{\displaystyle x_{i}^{'}=C_{ij}^{-1}~x_{j}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/62c289457b0892b40e3b86e9c7c4ace1d3b2968e)
Similarly,
![{\displaystyle b_{i}^{'}=C_{ij}^{-1}~b_{j}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7b58e1f0d4457724354d1b454d00b6b3154de477)
Therefore
![{\displaystyle {\boldsymbol {A}}~{\boldsymbol {\varphi }}^{'}=\sum _{j=1}^{n}C_{ji}~{\boldsymbol {A}}~{\boldsymbol {\varphi }}_{j}=\sum _{j,k}C_{ji}~A_{kj}~{\boldsymbol {\varphi }}_{k}=\sum _{j,k,m}C_{ji}~A_{kj}~C_{mk}^{-1}~{\boldsymbol {\varphi }}_{m}^{'}=\sum _{m}A_{mi}^{'}~{\boldsymbol {\varphi }}_{m}^{'}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/1f469489c76f590929400fa75d36430f9ec0c892)
So we have
![{\displaystyle A_{mi}^{'}=\sum _{j,k}C_{mk}^{-1}~A_{kj}~C_{ji}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/6ad265c92c4b88f7e87ee421d831d9b12126e7de)
In matrix form,
![{\displaystyle \mathbf {x} ^{'}=\mathbf {C} ^{-1}~\mathbf {x} ~;~~\mathbf {b} ^{'}=\mathbf {C} ^{-1}~\mathbf {b} ~;~~\mathbf {A} ^{'}=\mathbf {C} ^{-1}~\mathbf {A} ~\mathbf {C} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/3d4c9d269f66d18d9abcc67d51eb63aff24eecd2)
where the objects here are not operators or vectors but rather the matrices and
vectors representing them. They are therefore basis dependent.
In other words, the matrix equation
The transformation
![{\displaystyle \mathbf {A} ^{'}=\mathbf {C} ^{-1}~\mathbf {A} ~\mathbf {C} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/ac3b274376f180c032254377d488d4f9e10d9dcf)
is called a similarity transformation. Two matrices are equivalent
or similar is there is a similarity transformation between them.
Suppose we want to find a similarity transformation which makes
diagonal,
i.e.,
![{\displaystyle \mathbf {A} ^{'}={\begin{bmatrix}\lambda _{1}&0&\dots &\dots &0\\0&\lambda _{2}&0&\dots &0\\\vdots &\vdots &\ddots &\vdots &\vdots \\0&\dots &\dots &\dots &\lambda _{n}\end{bmatrix}}={\boldsymbol {\Lambda }}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/58aced1ec70c2119807f21db4f8041a4494c9615)
Then,
![{\displaystyle \mathbf {A} ~\mathbf {C} =\mathbf {C} ~\mathbf {A} ^{'}=\mathbf {C} ~{\boldsymbol {\Lambda }}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/9b5f91f1df401706a2663f558229329f5f24ee70)
Let us write
(which is a
matrix) in terms of its columns
![{\displaystyle \mathbf {C} ={\begin{bmatrix}\mathbf {x} _{1}&\mathbf {x} _{2}&\dots &\mathbf {x} _{n}\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d7e2592c21245a4fcb0c4bf3ab8d8bf90107a666)
Then,
![{\displaystyle \mathbf {A} ~{\begin{bmatrix}\mathbf {x} _{1}&\mathbf {x} _{2}&\dots &\mathbf {x} _{n}\end{bmatrix}}={\begin{bmatrix}\mathbf {x} _{1}&\mathbf {x} _{2}&\dots &\mathbf {x} _{n}\end{bmatrix}}~{\boldsymbol {\Lambda }}={\begin{bmatrix}\lambda _{1}~\mathbf {x} _{1}&\lambda _{2}~\mathbf {x} _{2}&\dots &\lambda _{n}~\mathbf {x} _{n}\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7571f452510d133733e802fa9627fb7f1acf76a6)
i.e.,
![{\displaystyle \mathbf {A} ~\mathbf {x} _{i}=\lambda _{i}~\mathbf {x} _{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f42a9c74ed3692c54a9117593f55d991fbda5d4a)
The pair
is said to be an eigenvalue pair if
where
is an eigenvector and
is an eigenvalue.
Since
this means that
is an
eigenvalue if and only if
![{\displaystyle \det(\mathbf {A} -\lambda ~\mathbf {I} )=\mathbf {0} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/cad8bd8ea96242f9ac4cab3be10ac542cbeed34d)
The quantity on the left hand side is called the characteristic polynomial
and has
roots (counting multiplicities).
In
there is always one root. For that root
is singular, i.e., there always exists at least one eigenvector.
We will delve a bit more into the spectral theory of matrices in the next
lecture.