Applied linear operators and spectral methods/Lecture 2
Norms in inner product spaces[edit | edit source]
Inner product spaces have norms which are defined as
When , we get the norm
When , we get the norm
In the limit as we get the norm or the sup norm
The adjacent figure shows a geometric interpretation of the three norms.
If a vector space has an inner product then the norm
is called the induced norm. Clearly, the induced norm is nonnegative and zero only if . It is also linear under multiplication by a positive vector. You can think of the induced norm as a measure of length for the vector space.
So useful results that follow from the definition of the norm are discussed below.
Schwarz inequality[edit | edit source]
In an inner product space
This statement is true if .
If we have
Let us choose such that it minimizes the left hand side above. This value is clearly
which gives us
Triangle inequality[edit | edit source]
The triangle inequality states that
From the Schwarz inequality
Angle between two vectors[edit | edit source]
In or we have
So it makes sense to define in this way for any real vector space.
We then have
Orthogonality[edit | edit source]
In particular, if we have an analog of the Pythagoras theorem.
In that case the vectors are said to be orthogonal.
If then the vectors are said to be orthogonal even in a complex vector space.
Orthogonal vectors have a lot of nice properties.
Linear independence of orthogonal vectors[edit | edit source]
- A set of nonzero orthogonal vectors is linearly independent.
If the vectors are linearly dependent
and the are orthogonal, then taking an inner product with gives
Therefore the only nontrivial case is that the vectors are linearly independent.
Expressing a vector in terms of an orthogonal basis[edit | edit source]
If we have a basis and wish to express a vector in terms of it we have
The problem is to find the s.
If we take the inner product with respect to , we get
In matrix form,
where and .
Generally, getting the s involves inverting the matrix , which is an identity matrix , because , where is the Kronecker delta.
Provided that the s are orthogonal then we have
and the quantity
is called the projection of onto .
Therefore the sum
says that is just a sum of its projections onto the orthogonal basis.
Let us check whether is actually a projection. Let
Therefore and are indeed orthogonal.
Note that we can normalize by defining
Then the basis is called an orthonormal basis.
It follows from the equation for that
You can think of the vectors as orthogonal unit vectors in an -dimensional space.
Biorthogonal basis[edit | edit source]
However, using an orthogonal basis is not the only way to do things. An alternative that is useful (for instance when using wavelets) is the biorthonormal basis.
The problem in this case is converted into one where, given any basis , we want to find another set of vectors such that
In that case, if
it follows that
So the coefficients can easily be recovered. You can see a schematic of the two sets of vectors in the adjacent figure.
Gram-Schmidt orthogonalization[edit | edit source]
One technique for getting an orthogonal baisis is to use the process of Gram-Schmidt orthogonalization.
The goal is to produce an orthogonal set of vectors given a linearly independent set .
We start of by assuming that . Then is given by subtracting the projection of onto from , i.e.,
Thus is clearly orthogonal to . For we use
If you want an orthonormal set then you can do that by normalizing the orthogonal set of vectors.
We can check that the vectors are indeed orthogonal by induction. Assume that all are orthogonal for some . Pick . Then
Now unless . However, at , because the two remaining terms cancel out. Hence the vectors are orthogonal.
Note that you have to be careful while numerically computing an orthogonal basis using the Gram-Schmidt technique because the errors add up in the terms under the sum.
Linear operators[edit | edit source]
The object is a linear operator from onto if
A linear operator satisfies the properties
Note that is independent of basis. However, the action of on a basis determines completely since
Since we can write
where is the matrix representing the operator in the basis .
Note the location of the indices here which is not the same as what we get in matrix multiplication. For example, in , we have
We will get into more details in the next lecture.