Spectral Theorem II: compact operators via the variational principle

Part I built the spectral theorem in finite dimensions, where the characteristic polynomial supplies an eigenvalue and Gram–Schmidt induction does the rest. Part III handles general bounded operators on a Hilbert space, where an operator may have no eigenvectors at all and the eigenbasis is replaced by a projection-valued measure. This Part II is the case between them: compact operators on a Hilbert space, where eigenvectors still exist, the nonzero spectrum is still discrete, and the spectral theorem still delivers an orthonormal basis.

The post moves between two settings. A Banach space is a complete normed vector space. A Hilbert space is a Banach space whose norm comes from an inner product (equivalently, whose norm satisfies the parallelogram law). §1 and §2 run at the Banach level: compactness is a norm/topology notion, the finite-dimensional eigenspace fact (Riesz–Schauder) is Banach, and Schauder’s adjoint theorem holds between any two Banach spaces. From §3 onward $H$ is a Hilbert space. What a Hilbert space adds is orthogonality: the variational principle uses $\langle Th, h\rangle$, self-adjointness is an inner-product condition, and the form the spectral theorem takes here (an orthonormal eigenbasis with orthogonal eigenspace projections $P_n$ in the operator-norm sum $T = \sum_n \lambda_n P_n$) is intrinsically Hilbert.

Two replacements take the place of finite-dimensional structure. In §3 the variational principle for $\|T\|$ supplies an eigenvalue where the characteristic polynomial used to. In §4 compactness itself forces the nonzero eigenvalues into a countable sequence with $|\lambda_n| \to 0$ where dimension induction used to terminate. With those in hand §5 assembles the self-adjoint spectral theorem and §6 extends to compact normal operators on a complex Hilbert space. Construction follows Conway (2007, Ch. II); Lax (2002) is the parallel reference.

Compact operators: closure of finite-rank, ideal property

A bounded operator $T \in \mathcal{B}(X, Y)$ between Banach spaces is compact if $T(\overline{B_X})$ is relatively compact in $Y$. Relatively compact means the norm-closure $\overline{T(\overline{B_X})}$ is compact in $Y$; equivalently (by metric compactness), every bounded sequence $\{x_n\} \subseteq X$ has $\{T x_n\}$ admitting a norm-convergent subsequence. We write $\mathcal{B}_0(X, Y)$ for the compact operators.

The intuition driving the definition is “essentially finite-dimensional, up to a controlled tail”. Every finite-rank operator is compact: its image of $\overline{B_X}$ sits in a bounded subset of a finite-dimensional subspace, compact by Heine–Borel. On a Hilbert space the converse holds, with the Hilbert-space shortcut in §2 supplying the proof: every compact operator is the operator-norm limit of finite-rank operators. This is the structure the spectral theorem below recovers: isolated finite-multiplicity nonzero eigenvalues, with the operator approximable in norm by truncating its eigenexpansion.

Closure. $\mathcal{B}_0(X, Y)$ is a closed subspace of $\mathcal{B}(X, Y)$ in operator norm. If $T_n \to T$ in operator norm with each $T_n$ compact, pick $N$ with $\|T_N - T\| < \varepsilon/2$ and an $\varepsilon/2$-net $\{y_1, \ldots, y_M\}$ for $T_N(\overline{B_X})$. For any $x \in \overline{B_X}$ some $y_i$ satisfies

$\|T x - y_i\| \le \|T x - T_N x\| + \|T_N x - y_i\| < \varepsilon,$

so $T(\overline{B_X})$ is totally bounded, hence relatively compact.

Compact operators form a two-sided ideal

If $K: X \to Y$ is compact and $S: Y \to Z$ is bounded, then $S K: X \to Z$ is compact. The image $K(\overline{B_X})$ is precompact in $Y$ by assumption, and the continuous map $S$ carries precompact sets to precompact sets, so $S K(\overline{B_X})$ is precompact in $Z$. The symmetric argument, that $K T$ is compact for any bounded $T: W \to X$, is identical. Specialized to $\mathcal{B}(H) = \mathcal{B}(H, H)$, this is the two-sided ideal property: composition of a compact operator with a bounded operator on either side stays inside the compact operators (Conway, 2007, Ch. II).

Finite-dimensional eigenspaces. For $T \in \mathcal{B}_0(X)$ and $\lambda \ne 0$, the eigenspace $M := \ker(T - \lambda I)$ is finite-dimensional. The restriction $T|_M$ is compact (its image of $\overline{B_M}$ sits inside $T(\overline{B_X})$, which is relatively compact). But $T|_M = \lambda I_M$, so $\lambda \overline{B_M} = T|_M(\overline{B_M})$ is relatively compact, hence $\overline{B_M}$ is relatively compact (using $\lambda \ne 0$), forcing $\dim M < \infty$ by Riesz’s lemma.

Riesz's lemma

In an infinite-dimensional normed space, the closed unit ball is not (pre)compact. The standard form: for any proper closed subspace $M \subset X$ and any $\varepsilon \in (0, 1)$ there exists a unit vector $x \in X$ with $d(x, M) \ge 1 - \varepsilon$. Iterating gives a bounded sequence of unit vectors pairwise distant by at least $1 - \varepsilon$, hence with no Cauchy subsequence, hence with no convergent subsequence. So a (pre)compact closed unit ball forces finite dimension (Conway, 2007, Ch. III).

Schauder: compactness passes to the adjoint

Schauder’s theorem says compactness transfers to the adjoint between any two Banach spaces (no inner product required). The Hilbert adjoint $T^*: H \to H$ characterized by $\langle T x, y\rangle = \langle x, T^* y\rangle$ needs one, so we first need the broader Banach adjoint, available for any bounded $T: X \to Y$ between Banach spaces:

$T^*: Y^* \to X^*, \qquad (T^* y^*)(x) := y^*(T x).$

Equivalently, $T^* y^*$ is the linear functional $x \mapsto y^*(T x)$ on $X$. The operator $T^*$ satisfies $\|T^*\| = \|T\|$ (Conway, 2007, Ch. VI). For Hilbert spaces the Banach adjoint pulls back through the Riesz identification $H \cong H^*$ to the familiar inner-product form (see Part III §2).

Theorem (Schauder). Let $T: X \to Y$ be bounded between Banach spaces. Then $T$ is compact iff $T^*: Y^* \to X^*$ is compact.

Two routes deliver this. The general Banach proof goes through Arzelà–Ascoli on $C(K)$. The Hilbert-space version is much shorter via finite-rank approximation.

Banach proof via Arzelà–Ascoli

Forward direction. Compactness of $T^*$ means $T^*(\overline{B_{Y^*}})$ is relatively compact in $X^*$, where $\overline{B_{Y^*}}$ is the closed unit ball of $Y^*$.

Let $K := \overline{T(\overline{B_X})} \subseteq Y$. Compactness of $T$ is exactly the statement that $K$ is compact in $Y$. Each $y^* \in \overline{B_{Y^*}}$ restricts to a continuous function $y^*|_K \in C(K)$: uniformly bounded by $|y^*(y)| \le \|y\|$ on $K$, and $1$-Lipschitz by $|y^*(y) - y^*(z)| \le \|y - z\|$. The family

$\mathcal{F} := \{\, y^*|_K : y^* \in \overline{B_{Y^*}} \,\} \subseteq C(K)$

is therefore uniformly bounded and equicontinuous on the compact metric space $K$. By the Arzelà–Ascoli theorem $\mathcal{F}$ is relatively compact in $C(K)$ with the sup norm.

The link from $\mathcal{F}$ back to $T^*(\overline{B_{Y^*}})$ is an isometry. For $y_1^*, y_2^* \in \overline{B_{Y^*}}$,

$\|T^* y_1^* - T^* y_2^*\|_{X^*} \;=\; \sup_{x \in \overline{B_X}} | (y_1^* - y_2^*)(T x) | \;=\; \sup_{y \in K} |(y_1^* - y_2^*)(y)| \;=\; \| y_1^*|_K - y_2^*|_K \|_{C(K)},$

using continuity of $y_1^* - y_2^*$ to extend the supremum from $T(\overline{B_X})$ to its closure $K$. So $T^*(\overline{B_{Y^*}})$ sits inside $X^*$ as an isometric copy of $\mathcal{F}$; relative compactness transfers from $C(K)$ across the isometry to $X^*$.

The converse direction applies the forward argument to $T^*$ to conclude that $T^{**}: X^{**} \to Y^{**}$ is compact, then uses the canonical isometric embedding $X \hookrightarrow X^{**}$ to recover compactness of $T$ itself (Conway, 2007, Ch. VI).

Hilbert shortcut via finite-rank approximation

On a Hilbert space the Schauder argument shortens by using the approximation property that Hilbert spaces always possess: every compact operator on $H$ is the operator-norm limit of finite-rank operators. The shortcut runs in four lines.

1. Compact operators on $H$ are norm-closure of finite-rank. Compactness of $T$ makes $T(\overline{B_H})$ precompact, so for any $\varepsilon > 0$ there is a finite $\varepsilon/2$-net $\{y_1, \ldots, y_N\} \subseteq T(\overline{B_H})$. Let $P_N$ be the orthogonal projection onto $\mathrm{span}(y_1, \ldots, y_N)$. For each $x \in \overline{B_H}$, with $y_i$ the nearest net point to $T x$,

   $\|T x - P_N T x\| \;\le\; \|T x - y_i\| + \|P_N(y_i - T x)\| \;\le\; 2\|T x - y_i\| \;\le\; \varepsilon,$

   so $\|T - P_N T\| \le \varepsilon$ in operator norm, with $P_N T$ of rank $\le N$.

2. The adjoint is an isometric involution on $\mathcal{B}(H)$, $\|S^*\| = \|S\|$. Norm convergence is preserved: $P_N T \to T$ implies $(P_N T)^* = T^* P_N \to T^*$ in norm.

3. $T^* P_N$ is finite-rank of rank $\le N$, since its range sits in $T^*(\mathrm{range}\, P_N)$, a finite-dimensional subspace. Finite-rank operators are compact.

4. Compact operators are norm-closed in $\mathcal{B}(H)$ (the closure result of the previous section). So $T^*$, a norm limit of compact operators, is compact.

This route bypasses Arzelà–Ascoli, $C(K)$, and the double dual entirely, at the cost of being Hilbert-specific. The Banach argument cannot be shortened the same way: there exist separable Banach spaces without the approximation property (Enflo, 1973), and on such a space some compact operator is not a norm limit of finite-rank operators. Hilbert spaces always have the approximation property because orthonormal bases supply the approximating projections.

The variational eigenvalue lemma

From here $H$ is a Hilbert space, $T^*$ denotes the Hilbert adjoint $T^*: H \to H$ (the previous section’s Banach adjoint pulled back through the Riesz identification $H \cong H^*$), and self-adjoint / normal are taken relative to this inner product. The first replacement for the characteristic polynomial is variational.

What the finite-dimensional setting hid

Part I could assume an inner product almost for free: in finite dimensions all norms are equivalent, and any space admits one (pick a basis, declare it orthonormal). None of that holds here. Norms on an infinite-dimensional space are not all equivalent; not every Banach space is a Hilbert space (a norm is Hilbertian only under the parallelogram law); and the inner product is now part of the given Hilbert structure, not something conjured from a basis. The dependence is essential rather than pedantic: compactness itself is a norm/topology property (whether $T(\overline{B_H})$ is relatively compact), while the adjoint and self-adjoint/normal are inner-product properties, and the spectral theorem below is a statement about an operator on a fixed Hilbert space. Change the inner product and the adjoint, the normality, and the orthonormal eigenbasis all change with it.

Lemma (variational eigenvalue). If $T \in \mathcal{B}_0(H)$ is self-adjoint and $T \ne 0$, then $\|T\|$ or $-\|T\|$ is an eigenvalue of $T$.

For any self-adjoint operator, the variational formula $\|T\| = \sup_{\|h\|=1} |\langle T h, h\rangle|$ holds and gives a maximizing sequence $\{h_n\}$ on the unit sphere. In infinite dimensions the sequence itself need not converge: the unit sphere is not compact. Compactness of $T$ is the structural input that fixes that. The image $\{T h_n\}$ inherits a convergent subsequence from $T$‘s compactness, and a short calculation forces $h_n$ itself to converge along that subsequence, with the limit an eigenvector for $\pm\|T\|$.

Proof

The variational identity $\|T\| = \sup_{\|h\| = 1} |\langle T h, h\rangle|$ for self-adjoint $T$ is standard. Choose unit vectors $h_n$ with $|\langle T h_n, h_n\rangle| \to \|T\|$. Self-adjointness makes $\langle T h_n, h_n\rangle$ real, so a subsequence has $\langle T h_n, h_n\rangle \to \lambda$ with $\lambda \in \{\|T\|, -\|T\|\}$.

Compactness of $T$ applied to the bounded sequence $\{h_n\}$ gives a further subsequence with $T h_n \to k$ in norm. From $\|T h_n\| \ge |\langle T h_n, h_n\rangle| \to \|T\|$ we get $\|k\| \ge \|T\| > 0$, so $k \ne 0$ and $\lambda \ne 0$. Then

$\|T h_n - \lambda h_n\|^2 = \|T h_n\|^2 - 2\lambda\, \mathrm{Re}\,\langle T h_n, h_n\rangle + \lambda^2 \le 2\|T\|^2 - 2\lambda \langle T h_n, h_n\rangle \to 2\|T\|^2 - 2\lambda^2 = 0,$

using $\lambda^2 = \|T\|^2$ in the last step. Combined with $T h_n \to k$, this gives $\lambda h_n \to k$, so $h_n \to k/\lambda$. Continuity of $T$ yields $T(k/\lambda) = \lim_n T h_n = k$, i.e., $T k = \lambda k$. $\square$

Countability and decay of the nonzero spectrum

The second replacement for finite-dimensional structure: compactness alone forces the nonzero eigenvalues to be a countable set decaying to zero, provided eigenvectors of distinct eigenvalues are orthogonal. The orthogonality is automatic for self-adjoint or normal $T$ (Part I §2).

Lemma (countability and decay). Suppose $T \in \mathcal{B}_0(H)$ and eigenvectors of distinct nonzero eigenvalues are orthogonal. Then the set of nonzero eigenvalues is at most countable; in the countable case, any enumeration $|\lambda_1| \ge |\lambda_2| \ge \cdots$ satisfies $|\lambda_n| \to 0$.

Proof. Let $\{\lambda_i\}_{i \in I}$ enumerate the distinct nonzero eigenvalues with unit eigenvectors $v_i$. Fix $\varepsilon > 0$ and set $F_\varepsilon := \{i \in I : |\lambda_i| \ge \varepsilon\}$. For $i \ne j$ in $F_\varepsilon$, orthogonality of $v_i$ and $v_j$ gives

$\|T v_i - T v_j\|^2 = \|\lambda_i v_i - \lambda_j v_j\|^2 = |\lambda_i|^2 + |\lambda_j|^2 \ge 2 \varepsilon^2.$

If $F_\varepsilon$ were infinite, $\{T v_i\}_{i \in F_\varepsilon}$ would have no Cauchy subsequence, contradicting compactness of $T$ applied to the bounded sequence $\{v_i\}$. So $F_\varepsilon$ is finite for every $\varepsilon > 0$, yielding countability and the decay. $\square$

Spectral theorem (compact self-adjoint)

Theorem. Let $T \in \mathcal{B}_0(H)$ be self-adjoint. Then:

1. $H$ has an orthonormal basis of eigenvectors of $T$.
2. The nonzero eigenvalues form a finite or countable real sequence $\lambda_1, \lambda_2, \ldots$ with $|\lambda_1| \ge |\lambda_2| \ge \cdots$ and $|\lambda_n| \to 0$ in the countable case; each $\ker(T - \lambda_n)$ is finite-dimensional.
3. With $P_n$ the orthogonal projection onto $\ker(T - \lambda_n)$,

$T = \sum_n \lambda_n P_n \quad \text{in operator norm}.$

The construction is iterative peel-off, finite-dimensional in spirit. The variational lemma supplies $\lambda_1 := \pm\|T\|$ as the dominant eigenvalue; its finite-dimensional eigenspace $M_1 = \ker(T - \lambda_1)$ splits off as an orthogonal direct summand, and the restriction $T_2 := T|_{M_1^\perp}$ is again compact self-adjoint with strictly smaller norm. Iterate, generating $\lambda_n = \pm\|T_n\|$ and finite-dimensional $M_n$ at each step. Two outcomes are possible: either the process terminates with $T_{N+1} = 0$ (so $T$ was finite-rank to begin with, modulo $\ker T$), or it continues, producing the infinite sequence $|\lambda_n| \to 0$ guaranteed by the countability lemma. The remaining vectors live in $\ker T$, where $T$ acts as zero; any orthonormal basis of $\ker T$ completes the eigenbasis. Operator-norm convergence of $\sum_{i \le n} \lambda_i P_i$ to $T$ falls out of the bound $\|T_{n+1}\| = |\lambda_{n+1}| \to 0$ for the leftover restriction.

Proof

If $T = 0$, take any ONB of $H$; (1)–(3) hold trivially. Assume $T \ne 0$.

Iteration. By the variational lemma, $\lambda_1 := \pm \|T\| \ne 0$ is an eigenvalue. Set $M_1 := \ker(T - \lambda_1)$, finite-dimensional by §1. The orthogonal complement $H_2 := M_1^\perp$ reduces $T$ (self-adjoint operators take orthogonal complements of invariant subspaces to invariant subspaces), so $T_2 := T|_{H_2}$ is compact self-adjoint on $H_2$ with $\|T_2\| \le \|T\|$.

At step $n$ we have $\lambda_n := \pm \|T_n\|$, $M_n := \ker(T_n - \lambda_n) \subseteq H_n$ finite-dimensional, and $H_{n+1} := (M_1 \oplus \cdots \oplus M_n)^\perp$. Set $T_{n+1} := T|_{H_{n+1}}$. The sequence $\|T_n\| = |\lambda_n|$ is nonincreasing.

Two cases.

(a) Finite termination. If $T_{N+1} = 0$ for some $N$, then $H_{N+1} \subseteq \ker T$. Taking ONBs of each $M_n$ ($n \le N$) and of $\ker T$ and unioning them gives an ONB of $H = M_1 \oplus \cdots \oplus M_N \oplus \ker T$ consisting of eigenvectors; $T = \sum_{n \le N} \lambda_n P_n$ exactly.

(b) Countable case. $T_{n+1} \ne 0$ for every $n$, producing infinitely many $\lambda_n$. By the countability lemma, $|\lambda_n| \to 0$.

Operator-norm convergence. For $h \in M_1 \oplus \cdots \oplus M_n$, the partial sum $\sum_{i \le n} \lambda_i P_i$ acts as $T$, so $\bigl(T - \sum_{i \le n} \lambda_i P_i\bigr) h = 0$. For $h \in H_{n+1}$,

$\Bigl( T - \sum_{i \le n} \lambda_i P_i \Bigr) h = T h = T_{n+1} h, \qquad \|T_{n+1} h\| \le \|T_{n+1}\|\, \|h\| = |\lambda_{n+1}|\, \|h\|.$

A general $h$ decomposes orthogonally as $h = h' + h''$ with $h' \in M_1 \oplus \cdots \oplus M_n$ and $h'' \in H_{n+1}$, giving

$\Bigl\| T - \sum_{i \le n} \lambda_i P_i \Bigr\|_{\mathrm{op}} \le |\lambda_{n+1}| \to 0.$

ONB property. The union of ONBs of each $M_n$ with an ONB of $\ker T$ is orthonormal (distinct eigenspaces are orthogonal). To show totality, suppose $v$ is orthogonal to every element of the union. Then $P_n v = 0$ for every $n$, so by operator-norm convergence $T v = \lim_n \sum_{i \le n} \lambda_i P_i v = 0$, hence $v \in \ker T$. But $v$ is orthogonal to an ONB of $\ker T$, so $v = 0$. $\square$

Spectral theorem (compact normal, complex case)

The normal case reduces to two applications of the self-adjoint case via the Cartesian decomposition. For $T \in \mathcal{B}_0(H)$ on a complex Hilbert space, define

$A := \tfrac{1}{2}(T + T^*), \qquad B := \tfrac{1}{2i}(T - T^*),$

so $T = A + iB$. Both $A$ and $B$ are compact (ideal property) and self-adjoint by construction. Normality $T^* T = T T^*$ is equivalent to $A B = B A$ (the operator analogue of the trivially commuting decomposition $z = \mathrm{Re}\, z + i\, \mathrm{Im}\, z$ for complex scalars; see Part I §2).

Theorem. Let $T \in \mathcal{B}_0(H)$ be normal on a complex Hilbert space. Then conclusions (1)–(3) of the self-adjoint theorem hold for $T$, with complex eigenvalues $\lambda_n$.

Proof. Apply the self-adjoint theorem to $A$: $H = \bigl(\bigoplus_n \ker(A - a_n)\bigr) \oplus \ker A$, with distinct real eigenvalues $a_n$, each $\ker(A - a_n)$ finite-dimensional, $|a_n| \to 0$.

The key step is that $A B = B A$ implies each $\ker(A - a_n)$ and $\ker A$ is invariant under $B$: for $v \in \ker(A - a_n)$, $A(B v) = B(A v) = a_n B v$, so $B v \in \ker(A - a_n)$. Self-adjointness of $B$ makes these invariant subspaces reducing.

On each finite-dimensional $\ker(A - a_n)$, $B$ restricts to a self-adjoint operator on a finite-dimensional space, and the finite-dimensional spectral theorem (Part I) gives an ONB $\beta_n$ of $B$-eigenvectors. On $\ker A$ (possibly infinite-dimensional), $B$ restricts to compact self-adjoint, and the previous section gives an ONB $\gamma$ of $B$-eigenvectors.

The union $\mathcal{E} := \bigl(\bigcup_n \beta_n\bigr) \cup \gamma$ is an ONB of $H$ (orthonormality from the orthogonal decomposition above; totality from the same). Each element of $\mathcal{E}$ is a joint eigenvector of $(A, B)$, hence of $T = A + iB$. For $v \in \beta_n$ with $A v = a_n v$ and $B v = b\, v$, $T v = (a_n + i b)\, v$; for $v \in \gamma$ with $A v = 0$ and $B v = b\, v$, $T v = i b\, v$.

Countability and decay of the eigenvalues of $T$ follow from the countability lemma applied to compact $T$ (normality gives orthogonality of eigenvectors for distinct eigenvalues; see Part I §2). Operator-norm convergence is a Parseval calculation on the ONB $\mathcal{E}$, identical in form to the self-adjoint case (Conway, 2007, Ch. II). $\square$

What it amounts to

In finite dimensions (Part I), the characteristic polynomial and Gram–Schmidt finish the construction in finitely many steps. Here the characteristic polynomial is unavailable: the variational principle supplies an eigenvalue at $\pm\|T\|$, and compactness forces the nonzero spectrum into a sequence with $|\lambda_n| \to 0$. In the general bounded case (Part III), neither route works and the spectral theorem becomes an integral against a projection-valued measure. Compactness is what keeps the discrete-sum form valid outside finite dimensions; the SVD consequence is collected in the Operator SVD capstone.

References

1. Enflo, P. (1973). A counterexample to the approximation problem in Banach spaces. Acta Mathematica 130, 309–317.
2. Lax, P. D. (2002). Functional Analysis. Wiley-Interscience.
3. Conway, J. B. (2007). A Course in Functional Analysis (2nd ed.). Springer.