Linear Methods of Applied Mathematics

Evans M. Harrell II and James V. Herod*

version of 21 February 1999

The printed version of these notes is an appendix to the rest. In this Web version, it consists of links bringing out the analogies between integral operators and matrices.

The common setting for the two subjects is an
*inner product space*, which was defined abstractly in the
chapter entitled
The Geometry of Functions.
Since the idea of
an inner product, or dot product, arises in such a variety of problems, we
should recall exactly what are the properties that define an inner product and
what are some of the consequences of these properties.

**Appendix A1** **ARITHMETIC AND GEOMETRY IN R**^{n}

If E is the linear space on which the inner product is defined and x and y are in E, then < x, y > denotes the (perhaps, complex) number which is the inner product of x and y. Moreover

(a) <x,y> = <y,x>* for all x,y in ExE (Note that * is the complex conjugate),

(b) < [[alpha]]x+y, z > = [[alpha]]< x, z > + < y, z > for all numbers [[alpha]] and all x and y in E,

(c) < x, x > > 0 if x !=0, with < x, x > = 0 if x = 0.

One consequence of these defining properties is the Cauchy-Schwartz inequality: if x and y are in E then

|< x, y >|^{2} <= < x, x > < y, y >.

A norm is defined in terms of the inner product:

||x||^{2} = < x , y >.

This norm has those properties which characterize any norm:

||x|| > 0 unless x = 0,

||ax|| = |a| ||x||, and

||x+y|| <= ||x|| + ||y||.

**EXAMPLE**: One can think of a* weighted *dot product on
R^{n} by choosing a positive number sequence {ap}O(p=1,^{n})
and defining < x, y > to be given by

The *usual* inner product on R^{n} is the one obtained in the
above example by choosing ap = 1 for p= 1,2,...n. In this setting, we can
think of the structure of R^{n} in the language of Euclidean geometry:
the distance between x and y is ||x-y|| and the angle between x and y is
[[theta]] where 0 <= [[theta]] <= [[pi]] and

provided neither x nor y is zero. For example, points x and y are
perpendicular, or orthogonal, if < x, y > = 0. Also, the concept of
distance provides a notion of points being "close together". It is natural to
say that the sequence {vp} of points in R^{n} has limit the point y in
R^{n} provided

limp || vp - y || = 0.

It is of value, even at this elementary level, to realize that there are
several ways to think of the idea "limp vp = y". In R^{n}, the
following three are equivalent:

The sequence {vp} in R^{n} converges to y *component wise* if for
each integer i, 1 <= i <= n,

limp(vp)i = yi.

The sequence {vp} in R^{n }converges to y *uniformly *if

limp( maxi | xp(i) -y(i) | )= 0.

The sequence {vp} in R^{n} converges to y in *norm* if

limp||vp - y|| =0.

It is not difficult to establish that these three notions are equivalent
in R^{n}. The value of thinking of them separately here is that the
three methods of convergence have analogues in situations which we will
encounter in later sections. In those situations, the three methods of
convergence may be not equivalent.

Fundamental in the development of Green's Functions will be
the
Riesz
Representation Theorem: If L is a linear function from R^{n} to R then
there is a vector **v** in R^{n} such that L(u) = < u, **v
**> for all u in R^{n}. Closely related to the Riesz
Representation Theorem is the fact that every linear function from
R^{n} to R^{n} has a matrix representation. These ideas are
familiar.

**Appendix A2:
THE ADJOINT A***

Linear functions and matrices arise in many ways. Here is one that
you might not have considered. Choose an nxn matrix A. Consider all points x
and y in R^{n} related by

< Au, x > = < u, y >

for all u in R^{n}. To be sure that, given x, there is such a point y,
consider the following: pick x; then L(u) = <Au, x > is a linear
function of u from R^{n} to R. By the Riesz Representation Theorem,
there is a point y in R^{n} such that L(u) = < u, y > for all u
in R^{n}. Define this y as B(x). It can be established that B is,
itself, a linear function from R^{n} to R^{n}. Hence, it has a
matrix representation. This linear function B is called the adjoint of A and is
denoted as A*. For all x and u in R^{n},

< Au, x > = < u, A*x >.

The concern of this course, stated in the context of R^{n}, is the
following problem: given a matrix A and a vector f = {f1 ,f2 , ...fn}, can a
vector u be found such that Au = f? There are matrices A and vectors f such
that the equation Au=f has exactly one solution, or no solution, or an infinity
of solutions.

**Appendix A3** **THE
FREDHOLM
ALTERNATIVE THEOREMS**

The following ideas will persist in each segment of the course. These fundamental results are known as the Fredholm Alternative Theorems. For matrices, the alternatives hinge on whether or not the determinant of A is zero or not.

I. Exactly one of the following two alternatives holds:

(a)(**First Alternative**) if f is in R^{n}, then Au = f has
one and only one solution.

(b)(**Second Alternative**) Au= 0 has a nontrivial solution.

II. (a) If the first alternative holds for A, then it also holds for A*.

(b) In either alternative, the equations Au= 0 and A* u = 0 have the same number of linearly independent solutions.

III. Suppose the second alternative holds. Then Au=f has a solution if and only if < f, z > = 0 for each z such that A*z=0.

A matrix problem is *well- posed* if

(a) for each f, the equation Au=f has a solution,

(b) the solution is unique, and

(c) the solution depends continuously on the data-- in the sense that if f is close to g and u and v satisfy Au=f and Av = g, then u is close to v.

**Appendix A4 SOLVING EQUATIONS**

Just as there are many ways to conceive of solving Au=f in case A is a matrix, so there are many ways for finding solutions for problems which are introduced in the next chapters. We concentrate on the method of constructing a Green's function. From what has come before, it may be clear that in some sense, we are finding an inverse for the linear operation A.

**THE FIRST ALTERNATIVE**

We discuss the solution of the linear equation

Au = f,

where we suppose we are given a matrix A and know f. Don't be to quick to
dismiss the solution as u = A^{-1} f. While that is correct, we want
perspective here, not just results.

Here is an adaptation of the methods which we will use to find Green's
functions in two chapters from here: Let delta_{i} be the vector which has the
property that

< delta_{i}, u > = u(i)

for all u in R^{n}. One can write the components for delta_{i}

delta_{i} = {0,0,...0. 1, 0, ... 0}

where the 1 is in the i-th component. Note that

u = [[Sigma]]i delta_{i} u(i).

Let A be an nxn matrix and f be a vector. In order to solve Au=f, we seek G(i,j) such that u defined by

< G(j,.) , f(.) > = u(j) (*)

might provide the solution for Au=f.

Look again at equation (*). Writing the dot product as a sum changes that equation to

or, in the notation of matrix multiplication,

Gf = u.

In what follows, perhaps you will see that writing the equation as (*) or (**) provides unifying ideas.

Here is a proposal for how humans find G. Find G such that

A(G( ,i))= delta_{i}.

That is, A(G( ,i))(m) = delta_{i}(m).

Having such a G, define u by

u(j) = [[Sigma]] i G(j,i)f(i).

Then

[A(u)](m) = [[Sigma]]j A(m,j) u(j) = [[Sigma]] j A(m,j) [[Sigma]] i G(j,i)f(i)

= [[Sigma]] i [[Sigma]] j A(m,j)G(j,i)f(i)

= [[Sigma]] idelta_{i}(m) f(i) = f(m).

This is what was desired: u solves the equation

Au = f.

Here is an alternate approach to the problem: Consider these relations:

<delta_{i},u> = u(i) = <G(i,.),f(.)> = <G(i,.),Au> =
<A*G(i,.),u>.

Thus, we might seek G such that delta_{i} = A*G(i,.). Go back and re-do the above
exercise this way to see if you get the same answer. Here's a proof that you
should fill in with details.

**THEOREM.** Suppose that A and G are matrices. These are equivalent:

(a) A(G( ,i)) = delta_{i} and (b) A*(G(i, )) = delta_{i}.

Suggestion for a proof. Let G1 be defined by the first equation:

A(G1( ,i)) = delta_{i}

and G2 be defined by the second equation:

A*(G2(i, )) = delta_{i}.

Then G1(j,i) = < dj , G1( ,i) > = < A*(G2(j, )) , G1( ,i)) >

= < G2(j, ) , A(G1( ,i)) > = < G2(j, ) , delta_{i}
> = G2(j,i).

In case the determinant of A is zero and we are in the second alternative, we can still conceive of the possibility of constructing G such that if f is perpendicular to the null space of the adjoint of A, then

u(j) = < G(j, ), f >

provides a solution to the equation Au = f. The methods developed above will
not work for, given i, we cannot find a vector G(i, ) that solves A(G(i, )) =
delta_{i}. To see this, recall the third part of the Fredholm Alternative Theorem and
then note that

< delta_{i},v > != 0

for all vectors v in the nullspace of A*. Thus, in this second alternative, we must modify the method.

Let
be a maximal orthonormal sequence in the nullspace of A*. We know there is G(
,p) in R^{n} such that

for < dp- [[Sigma]]i=1vi( ) vi(p) , w > = 0

for all w in the nullspace of A*. We will show that u(i) defined by

< G(i, ), f >

satisfies the equation Au = f. In fact,

(b) Suppose that L: R^{3} -> R and is defined by L(u)= u2.
Find **v** such that L(u) = < u, **v **>.

(c) Give the matrix representation of L: R^{3} -> R^{3}
if L(u) = {3u1+2u3, -u1+u2, u1-u2+u3}.

**A.2**.
(a)

(b) For the following matrices A and vectors f, determing whether the equation Au = f has exactly one solution, no solution or an infinity of solutions. If there are solutions, find them.

**A.3**.
(a) For each of the matrices listed below determine what
is the dimension of the null space of A and of the null space of A*? Give a
basis for each. Find one f such that < f, z > = 0 for each z in the null
space of A*. Solve Au = f. Find g such that < g, z > != 0 for each z
in the null space of A*. Show that one can not solve Au=g for this g.

(b) Let

Show that the problem A_{1} u = f is well posed and the problem
A_{2} u = f is not.

**A.4**.
(a) For the following matrices A, show that det(A) = 0, Find an orthonormal
basis for the null space of A*. Make up G. For the given f show that is it is
perpendicular to the null space of A*. Show that u as defined by equation (*)
in the discussion of the First Alternative solves Au=f.

(2) Take

Find G such that the equation

< G(i .), f(.) > = u(i)

provides a solution for Au = f for any vector f. ( Be aware that high school
students know how to do this without ever thinking of the vector delta_{i}!)

Back to Chapter XVIII

Return to Table of Contents

Return to Evans Harrell's home page