S. D. V. Stephens

notes / Studies in Algebra and Group Theory / chapter 4: Categories, Bilinear Forms and Inner Products / From my book

B9.2 Orthonormal Bases

9.2.1 The Gram–Schmidt Process

In an inner product space, we can ask for bases that respect the geometry: bases whose vectors are mutually orthogonal and have unit length.

Definition 9.2.1 (Orthonormal basis). Let be an inner product space. A list of vectors in is orthonormal if Equivalently, for all , and for . An orthonormal basis is an orthonormal list that is a basis.

Note. An orthonormal list is linearly independent: if , then for each . So in an -dimensional space, every orthonormal list of length is an orthonormal basis.

In an orthonormal basis , coordinates are computed by inner products: if , then , so The inner product also takes a simple form: if , then For real spaces this is ; for complex spaces it is .

Theorem 9.2.2 (Existence of orthonormal bases). Every finite-dimensional inner product space has an orthonormal basis.

We give two proofs: one by induction, one constructive (Gram–Schmidt).

Proof 1 (Induction on dimension). Let . If , the empty list is an orthonormal basis. For :

Pick any nonzero and set , so . Let . By the orthogonal decomposition theorem, , and .

The restriction of to is still an inner product (the defining properties are inherited by a subspace). By induction, has an orthonormal basis .

Then is orthonormal, because for (as ); being of length , it is an orthonormal basis of . ∎

Proof 2 (Gram–Schmidt). Start with any basis of . We inductively construct orthonormal vectors such that for each ,

Step 1: Set .

Step : Suppose are orthonormal and span the same subspace as . Define This subtracts from its components along each : for , Since (linear independence), we have . Set

By construction, for , and . The span condition holds since and . ∎

Corollary 9.2.3 (Isomorphism with standard space). Every -dimensional real inner product space is isomorphic to with the standard inner product, as an inner product space (that is, by a linear isomorphism that preserves inner products). Similarly, every -dimensional complex inner product space is isomorphic to with the standard Hermitian inner product.

Proof. Let be an orthonormal basis. The coordinate map sending to is a linear isomorphism. Moreover, for , which is the standard inner product of the coordinate vectors. ∎

Note. This says that all -dimensional inner product spaces over the same field are “the same.” The choice of orthonormal basis corresponds to the choice of isomorphism with the standard space. Different orthonormal bases give different coordinate systems, all equally valid.

Example 9.2.4 (Gram–Schmidt in ). Apply Gram–Schmidt to the basis , , of with the standard inner product.

Step 1: , so .

Step 2: , so , so .

Step 3: and , so , so .

One checks directly that and .

9.2.2 Orthogonal Projections

The orthogonal decomposition gives a canonical way to project onto . It holds whenever is finite-dimensional, even if is not, and an orthonormal basis of gives an explicit formula.

Proposition 9.2.5 (Projection onto a finite-dimensional subspace). Let be an inner product space (not necessarily finite-dimensional) and a finite-dimensional subspace with orthonormal basis . Then , and the decomposition of is

Proof. Let . For each , Hence for all , and since the span , . So , and by positive-definiteness. ∎

Definition 9.2.6 (Orthogonal projection). Let be an inner product space and a subspace with (for instance, finite-dimensional). The orthogonal projection onto is the map defined by It is linear, by uniqueness of the decomposition.

Proposition 9.2.7 (Properties of orthogonal projections). Let be the orthogonal projection onto .

  1. and .
  2. (i.e., is idempotent).
  3. for all .
  4. If is finite-dimensional, then .

Proof. (i) follows from the definition. (ii) If , then has decomposition , so .

(iii) Write and with . Then

(iv) We have with and (Corollary 9.1.32). This is the decomposition of in , so . ∎

Note. In the language of the next section, (iii) says that is self-adjoint.

Proposition 9.2.8 (Best approximation). For and a finite-dimensional subspace, the projection is the unique vector in minimizing over .

Proof. For any , write Since and , these two vectors are orthogonal, and by the Pythagorean theorem with equality iff . ∎

Example 9.2.9 (Fourier series). Let be the space of infinitely differentiable functions with period (functions on the circle ), with This is a Hermitian inner product ( for a continuous ). The functions , , are orthonormal: They are not a basis of : their span is the subspace of trigonometric polynomials. But for each , Proposition 9.2.5 applies to : The numbers are the Fourier coefficients of , so the partial sums of the Fourier series of are orthogonal projections, and by best approximation is the trigonometric polynomial of degree at most closest to . Making sense of the full infinite series requires analysis (Hilbert spaces), which we do not pursue.