40. Orthogonal Decomposition

Dated: 18-06-2025

Orthogonal Projection

\(\vec y \in \mathbb W \in \mathbb R^n \ \exists \ \vec y^\prime \in \mathbb W\) such that

\(\vec y^\prime\) is closest to \(\vec y\)
\((\vec y - \vec y^\prime) \perp \mathbb W\)

Best Approximation Theorem

Let \(\mathbb W\) be a finite dimensional subspace¹ of an inner product space¹ \(\mathbb V\) and \(\vec y \in \mathbb V\).
The best approximation to \(\vec y\) from \(\mathbb W\) is \(\text{Proj}_\vec w^ \vec y\) that is, for every \(\vec w \notin \text{Proj}_\vec w^ \vec y \text{ but } \in \mathbb W\)

\[||\vec y - \text{Proj}_\vec w^\vec y|| < ||\vec y - \vec w||\]

Theorem

If \(\{\vec u_1, \vec u_2, \ldots, \vec u_p\}\) is an orthonormal basis² for a subspace³ \(\mathbb W \in \mathbb R^n\) then

\[\text{Proj}_\vec w ^\vec y = (y \cdot u_1) u_1 + (y \cdot u_2) u_2 + \cdots + (y \cdot u_p) u_p\]

if \(U = \begin{bmatrix}u_1 & u_2 & \cdots & u_p\end{bmatrix}\) then

\[\text{Proj}_\vec w ^\vec y = UU^T \vec y \quad \forall \ \vec y \in \mathbb R^n\]

40. Orthogonal Decomposition

Orthogonal Projection

Best Approximation Theorem

Theorem

References