Spectral Theorem I: adjoint, normality, and the finite-dimensional case

The spectral theorem is the structural fact that organizes linear algebra on inner-product spaces: every normal operator admits an orthonormal eigenbasis. This post builds the result in finite dimensions: what the adjoint of a linear operator is independently of a chosen basis, what normality really means and what its two parts say about a Markov dynamics, and how Gram–Schmidt induction on the eigenvectors delivers the spectral theorem itself.

Three companion posts continue and apply the result. Spectral Theorem II covers compact operators on a Hilbert space, where eigenvectors still exist but dimension induction does not. Spectral Theorem III covers the general bounded case, where eigenvectors need not exist at all and a projection-valued measure replaces the eigenbasis. The Operator SVD capstone collects the three spectral-theorem results and assembles them, via the polar reduction $L \to L^* L$ and the polar form $L = U|L|$, into the singular value decomposition at each level.

The adjoint and its relation to the transpose

When we write $A^\top$ for the transpose of a matrix $A \in \mathbb{R}^{n \times n}$, we are performing a purely algebraic operation: $(A^\top)_{ij} = A_{ji}$. The textbook identity $\langle A\mathbf{x}, \mathbf{y}\rangle = \langle \mathbf{x}, A^\top \mathbf{y}\rangle$, with $\langle \cdot, \cdot\rangle$ the dot product on $\mathbb{R}^n$, looks like it identifies the transpose with the adjoint of the operator defined by $A$. It does, but only because both sides of the equation rest on the same implicit pair of conventions: the standard basis $\{\mathbf{e}_1, \ldots, \mathbf{e}_n\}$ and the dot product as the inner product.

Drop either alignment and the two notions diverge. Let $V = \mathbb{R}^n$ with some non-standard basis $\{\mathbf{b}_1, \ldots, \mathbf{b}_n\}$, and let the inner product on $V$ be specified by its Gram matrix $G_{ij} = \langle \mathbf{b}_i, \mathbf{b}_j\rangle$. For a linear operator $L \colon V \to V$ with coordinate matrix $M$ in this basis, the geometric adjoint $L^*$ defined by the inner-product identity $\langle L\mathbf{v}, \mathbf{w}\rangle = \langle \mathbf{v}, L^* \mathbf{w}\rangle$ has coordinate matrix

$M^* = G^{-1} M^\top G.$

The transpose still appears, but flanked on both sides by basis-change matrices. The simple identity “adjoint equals transpose” holds only when $G = I$, i.e., when the chosen basis is orthonormal with respect to the chosen inner product.

The basis-free way to say this is that an inner product on $V$ gives a canonical isomorphism

$\Phi_V \colon V \to V^*, \quad \mathbf{v} \mapsto \langle \mathbf{v}, \cdot \rangle,$

between $V$ and its dual $V^* = \mathrm{Hom}(V, \mathbb{C})$. (In the complex case $\Phi_V$ is conjugate-linear; the spirit is the same.) For any linear map $L \colon V \to W$ between inner-product spaces, the algebraic dual $L^\vee \colon W^* \to V^*$ is defined without any inner product by $(L^\vee \varphi)(\mathbf{v}) = \varphi(L\mathbf{v})$. The geometric adjoint $L^* \colon W \to V$ is then the algebraic dual pulled back through the inner-product identifications on the two sides (Halmos, 1958):

$L^* := \Phi_V^{-1} \circ L^\vee \circ \Phi_W.$

This is the object that the matrix formula $M^* = G^{-1} M^\top G$ is computing in coordinates. The construction depends on the inner products on $V$ and $W$ but not on any choice of basis: change basis, and the matrix of $L$ changes by similarity while $G$ changes correspondingly; the formula delivers the same operator $L^*$ in the new coordinates.

Why the adjoint is a Hilbert-space notion

In Banach spaces (no inner product), the algebraic dual map $L^\vee$ still exists, but the geometric adjoint $L^*$ does not. Without an inner-product isomorphism $V \cong V^*$ there is no way to land back in $V$. The adjoint is intrinsically a Hilbert-space notion, which is why this post stays in inner-product spaces throughout.

So transpose is an operation on coordinate matrices that depends on a choice of basis. Adjoint is an operation on operators that depends on a choice of inner product but not on a choice of basis. The two coincide whenever the basis is orthonormal in the inner product, which is exactly the default we were using without naming it.

Normality: $LL^* = L^*L$ and what it really says

A linear operator $L \in \mathrm{End}(V)$ on a finite-dimensional complex inner-product space is normal if $LL^* = L^*L$. The condition admits several equivalent reformulations, and the one to anchor on says what normality is.

Split $L$ into self-adjoint and anti-self-adjoint parts,

$L = H + iK, \quad H := \frac{L + L^*}{2}, \quad K := \frac{L - L^*}{2i},$

both Hermitian by construction. A direct computation gives

$LL^* - L^*L = (H + iK)(H - iK) - (H - iK)(H + iK) = -2i\,[H, K],$

where $[H, K] := HK - KH$ is the commutator. So

$L \text{ is normal} \;\iff\; [H, K] = 0.$

The right-hand side is the operator analogue of the trivially commuting decomposition $z = \mathrm{Re}\,z + i\,\mathrm{Im}\,z$ for scalars. For complex numbers, real and imaginary parts commute automatically because scalars commute. For operators, the real and imaginary parts are Hermitian operators that need not commute in general, and normality is the condition that they do. A normal operator is one that “acts like a scalar” in the sense that its self-adjoint and anti-self-adjoint pieces do not interfere with each other.

A second equivalent formulation will do load-bearing work in the spectral-theorem proof below: $L$ is normal iff $\|L\mathbf{v}\| = \|L^* \mathbf{v}\|$ for every $\mathbf{v} \in V$. The forward and adjoint orbits of any vector have the same length. As an immediate consequence, if $L$ is normal, then $L$ and $L^*$ share eigenvectors with conjugate eigenvalues: $L\mathbf{v} = \lambda \mathbf{v}$ implies $L^* \mathbf{v} = \bar\lambda \mathbf{v}$. Apply the norm equality to $L - \lambda I$ (which inherits normality from $L$):

$\|(L - \lambda I)\mathbf{v}\|^2 = \|(L - \lambda I)^* \mathbf{v}\|^2 = \|(L^* - \bar\lambda I)\mathbf{v}\|^2.$

The left side vanishing forces the right side to vanish. This eigenvector-sharing fact is the engine of the spectral theorem, proved two sections below.

Is "normal" basis-independent, and is it the same as a normal matrix?

Two layers of dependence have to be separated. The inner product itself is chosen structure, not read off a norm: every finite-dimensional space admits one (pick a basis, declare it orthonormal), but a given norm induces an inner product only when it obeys the parallelogram law, and topological equivalence of norms supplies no inner product at all. Fix that choice and the rest is clean.

Basis-independent, inner-product-relative. With the inner product fixed, $L^* = \Phi_V^{-1} L^\vee \Phi_V$ is basis-free, so $LL^* = L^*L$ and $L = L^*$ are operator equations that hold or fail independently of any basis. They still depend on the inner product: change it and $L^*$ changes, so one operator can be self-adjoint under one inner product and not another.

Not the same as matrix normality, except in an orthonormal basis. In a basis with Gram matrix $G$, the matrix of $L^*$ is $G^{-1} M^\dagger G$. So $L$ is self-adjoint iff $GM = M^\dagger G$ (equivalently $GM$ is Hermitian), and normal iff $M\,G^{-1}M^\dagger G = G^{-1}M^\dagger G\,M$. Only when $G = I$ do these collapse to the familiar $M = M^\dagger$ and $MM^\dagger = M^\dagger M$. A Hermitian matrix in a skewed basis need not represent a self-adjoint operator, and the converse fails too.

The invariant content. Matrix normality is preserved by unitary change of basis but not by a general one, so it is not a similarity invariant. General-similarity invariants are the vector-space data (eigenvalues, diagonalizability, Jordan form, no inner product needed); unitary-similarity invariants are the inner-product data (normality, self-adjointness, singular values). Operator normality is exactly the unitary-similarity-invariant content, and $MM^\dagger = M^\dagger M$ is its shadow in orthonormal coordinates. This is why the spectral theorem can state “normal iff an orthonormal eigenbasis exists” without naming a basis.

The reversible and irreversible parts of a Markov dynamics

The $L = H + iK$ split is not just an algebraic device. For the Koopman operator of a Markov process it separates the reversible and irreversible parts of the dynamics, and a small example puts both parts in closed form.

Take a Markov chain on a finite state space with transition matrix $P$, $P_{ij} = p(j \mid i)$. Its Koopman operator acts on observables by $(\mathcal{K} g)(i) = \sum_j P_{ij}\, g(j) = \mathbb{E}[g(X_{t+1}) \mid X_t = i]$. The inner product that defines the adjoint is the one weighted by the stationary distribution $\pi$ (the row vector with $\pi P = \pi$): $\langle f, g\rangle_\pi = \sum_i \pi_i f(i) g(i)$. As the opening section stressed, the adjoint depends on this inner product, and a short computation gives

$\mathcal{K}^* = \Pi^{-1} P^\top \Pi, \qquad \Pi = \mathrm{diag}(\pi),$

the $\pi$-weighted transpose. The two Hermitian parts $H = (\mathcal{K} + \mathcal{K}^*)/2$ and $K = (\mathcal{K} - \mathcal{K}^*)/(2i)$ then have a dynamical reading. $H$ is self-adjoint on $L^2(\pi)$, so it is the Koopman operator of a reversible chain. $K$ measures the failure of $\mathcal{K}$ to be self-adjoint, and $\mathcal{K}^* = \mathcal{K}$ holds exactly when $\pi_i P_{ij} = \pi_j P_{ji}$ for all $i, j$, the detailed-balance condition, equivalent to time-reversibility (Norris, 1997). So $K = 0$ iff the chain is reversible; when $K \ne 0$ it encodes the net probability current circulating through the state space.

A worked example: the random walk on a 3-cycle. Put three states on a ring. From each state, step clockwise with probability $a$, counterclockwise with probability $b$, and stay with probability $c = 1 - a - b$. The transition matrix is the circulant

$P = \begin{pmatrix} c & a & b \\ b & c & a \\ a & b & c \end{pmatrix}.$

The rotational symmetry forces the stationary distribution to be uniform, $\pi = (\tfrac13, \tfrac13, \tfrac13)$, regardless of $a$ and $b$: the clockwise/counterclockwise asymmetry changes the current, not the occupancy. Because $\pi$ is uniform, $\Pi = \tfrac13 I$ and $\mathcal{K}^* = P^\top$ is the plain transpose, so

$H = \frac{P + P^\top}{2}, \qquad K = \frac{P - P^\top}{2i}.$

Both are circulant, so both are diagonalized by the discrete Fourier basis $\mathbf{u}_k = \tfrac{1}{\sqrt3}(1, \omega^k, \omega^{2k})$, $k = 0, 1, 2$, with $\omega = e^{2\pi i/3}$, and the eigenvalues come out in closed form. The eigenvalues of $\mathcal{K} = P$ are

$\lambda_k = c + a\,\omega^k + b\,\omega^{2k}, \qquad \lambda_0 = 1, \qquad \lambda_{1,2} = 1 - \tfrac{3(a+b)}{2} \pm i\,\tfrac{\sqrt{3}}{2}(a - b).$

Since $H$ and $K$ are the Hermitian and anti-Hermitian parts of $\mathcal{K}$ and share its eigenbasis, their eigenvalues are the real and imaginary parts of the $\lambda_k$:

$H \colon \bigl\{\, 1,\; 1 - \tfrac{3(a+b)}{2},\; 1 - \tfrac{3(a+b)}{2} \,\bigr\}, \qquad K \colon \bigl\{\, 0,\; \tfrac{\sqrt{3}}{2}(a-b),\; -\tfrac{\sqrt{3}}{2}(a-b) \,\bigr\}.$

Each component has a clean meaning. The mode $\mathbf{u}_0 = \tfrac{1}{\sqrt3}(1,1,1)$ is the constant observable, fixed by $\mathcal{K}$ (eigenvalue $1$) and contributing nothing to $K$. On the two non-trivial Fourier modes, $H$ acts with the single real eigenvalue $1 - \tfrac{3(a+b)}{2}$: this is the decay rate, and it is the same for both modes because the reversible part has no preferred sense of rotation. $K$ acts with the equal-and-opposite eigenvalues $\pm\tfrac{\sqrt3}{2}(a - b)$: this is the circulation rate, the irreversible part, and it is precisely the clockwise/counterclockwise asymmetry $a - b$. A complex eigenvalue of the Koopman operator is therefore a damped oscillation, with the decay supplied by the reversible part $H$ and the oscillation supplied by the irreversible part $K$. When $a = b$ the eigenvalues are real, $K = 0$, and the chain is reversible; when $a \ne b$ the eigenvalues are genuinely complex and the observable rotates as it relaxes.

Because $P$ is circulant, $H$ and $K$ are simultaneously diagonalized by the Fourier basis, so $[H, K] = 0$ and $\mathcal{K}$ is normal even when $a \ne b$. Normality is therefore strictly weaker than reversibility: the 3-cycle with unequal rates is irreversible (it carries a current) yet still normal. To break normality one has to break the circulant symmetry: a chain with state-dependent hop rates gives $[H, K] \ne 0$, and then no orthonormal eigenbasis exists. The companion post on the Koopman operator takes up the genuinely non-normal case in the stochastic, infinite-state setting.

The spectral theorem via Gram–Schmidt induction

The spectral theorem for normal operators in finite dimensions is now an induction-on-dimension corollary of the eigenvector-sharing fact:

1. (Base.) Pick any eigenvalue $\lambda$ of $L$ (which exists by the fundamental theorem of algebra applied to the characteristic polynomial); let $\mathbf{v}_1$ be a unit eigenvector.
2. (Step.) The orthogonal complement $\mathbf{v}_1^\perp$ is $L$-invariant. To see this, use $L^* \mathbf{v}_1 = \bar\lambda \mathbf{v}_1$: for $\mathbf{w} \perp \mathbf{v}_1$, $\langle L\mathbf{w}, \mathbf{v}_1\rangle = \langle \mathbf{w}, L^* \mathbf{v}_1\rangle = \langle \mathbf{w}, \bar\lambda \mathbf{v}_1\rangle = \lambda \langle \mathbf{w}, \mathbf{v}_1\rangle = 0,$ so $L\mathbf{w} \in \mathbf{v}_1^\perp$. The restriction $L|_{\mathbf{v}_1^\perp}$ is normal on a space of dimension $\dim V - 1$.
3. (Reassembly.) Iterating yields an orthonormal basis $\{\mathbf{v}_1, \ldots, \mathbf{v}_n\}$ of eigenvectors. When eigenvalues are degenerate, the eigenspace itself has higher dimension; one applies Gram–Schmidt orthogonalization within each eigenspace to obtain orthonormal eigenvectors.

Normality versus Schur triangularization

Without normality, Schur’s triangularization theorem still gives a form $L = U T U^*$ with $U$ unitary and $T$ upper-triangular; this is the triangularization available for any operator. Normality is precisely the additional condition under which the off-diagonal entries of $T$ vanish, leaving a diagonal matrix. Mechanically, the vanishing happens because of the eigenvector-sharing fact: with $L^* \mathbf{v}_1 = \bar\lambda \mathbf{v}_1$, the orthogonal complement of $\mathbf{v}_1$ is exactly $L$-invariant rather than approximately so, and the inductive peel-off produces no off-diagonal residue. Without normality the off-diagonal entries record the failure and survive the triangularization (Axler, 2015).

In operator form this is the resolution of identity. Grouping the orthonormal eigenbasis by eigenvalue gives mutually orthogonal eigenspaces $V_\lambda$ that span $V$; writing $P_\lambda$ for the orthogonal projection onto $V_\lambda$,

$I = \sum_{\lambda \in \sigma(L)} P_\lambda, \qquad L = \sum_{\lambda \in \sigma(L)} \lambda\, P_\lambda,$

since $L$ acts as $\lambda \cdot I$ on each $V_\lambda$. The decomposition of the identity into orthogonal projections is what the name resolution of identity refers to; the second equation is the spectral theorem in operator form. In infinite dimensions the sum becomes an integral against a projection-valued measure, which is the subject of Spectral Theorem III.

What it amounts to

The adjoint is the basis-free shadow of the transpose, fixed by the inner product. Normality, $[H, K] = 0$, is the condition that the self-adjoint and anti-self-adjoint parts of an operator commute; for the Koopman operator, that the reversible and irreversible parts of a dynamics do not interfere. The spectral theorem turns normality into an orthonormal eigenbasis, written either as a list of eigenvectors or as the resolution of identity above.

All of this is finite-dimensional. The arguments lean on two facts that do not survive the passage to infinite dimensions: the characteristic polynomial has a root, and Gram–Schmidt induction terminates after finitely many steps. Spectral Theorem II rebuilds the result for compact operators on a Hilbert space; eigenvectors still exist, with the variational principle replacing the characteristic polynomial, but the dimension induction has to be replaced by compactness-driven countability and decay. Spectral Theorem III handles the general bounded case, where an operator may have no eigenvectors at all and the functional calculus and projection-valued measure stand in for the eigenbasis entirely. The Operator SVD capstone applies all three results to the polar reduction $L \to L^* L$ and reads off the singular value decomposition at each level, with the Eckart–Young best-rank-$k$ statement as the standard quantitative consequence.

References

1. Halmos, P. R. (1958). Finite-Dimensional Vector Spaces (2nd ed.). Van Nostrand.
2. Norris, J. R. (1997). Markov Chains. Cambridge University Press.
3. Axler, S. (2015). Linear Algebra Done Right (3rd ed.). Springer.