43. Inner Product Space

Dated: 21-06-2025

Inner Product Space

The inner product¹ is helpful in introducing rigorous intuitive geometrical notions such as length, angle, orthogonality² between 2 vectors.³ Now consider a mathematical structure which associates the inner product¹ with a pair of vectors³ and now imagine this structure contained within a vector space⁴ called the inner product space.
This also generalizes the Euclidean spaces into vector spaces,⁴ possibly of infinite dimensions and are also studied in functional analysis.

Definition

An inner product¹ on a vector space⁴ \(\mathbb V\) is a function⁵ that assigned a real number \(\langle \vec u, \vec v \rangle\) to a pair of vectors³ \(\vec u\) and \(\vec v\) and satisfies following axioms

For all \(\vec u, \vec v, \vec w \in \mathbb V\) and all scalars \(c\).

\[\langle\vec u, \vec v\rangle = \langle\vec u, \vec v\rangle\]
\[\langle\vec u + \vec v, \vec w \rangle = \langle\vec u, \vec w\rangle + \langle \vec v, \vec w\rangle\]
\[\langle c\vec u, \vec v\rangle = c\langle\vec u, \vec v\rangle\]
\[\langle \vec v, \vec v\rangle \ge 0 \text{ and } \langle \vec v, \vec v \rangle = 0 \iff \vec v = 0\]

A vector space⁴ with an inner product is called inner product space.

Norm of a `Vector`³

Let \(\mathbb V\) be an inner product space with inner product³ denoted by \(\langle \vec u, \vec v \rangle\) just as in \(\mathbb R^n\), we define the norm of length of a vector³ to be the scalar

\[||\vec v|| = \sqrt{\langle\vec u, \vec v\rangle}\]

A unit vector³ whose length is \(1\).
The distance between \(\vec u\) and \(\vec v\) is \(||\vec u - \vec v||\) and they are orthogonal² if \(\langle \vec u, \vec v\rangle = 0\).

Best Approximation in Inner Produce Spaces

Imagine a vector space⁴ \(\mathbb{V}\) whose elements are functions.⁵ The task is to approximate a function⁵ \(f \in \mathbb{V}\) by another function⁵ \(g \in \mathbb{W} \subset \mathbb{V}\). How we measure the quality of this approximation depends on how we define the "distance" between \(f\) and \(g\), which in this case is derived from an inner product.¹ Under this setup, the best approximation of \(f\) by elements of \(\mathbb{W}\) is given by the orthogonal projection² of \(f\) onto \(\mathbb{W}\).

Cauchy – Schwarz Inequality

\[|\langle \vec u, \vec v \rangle| \le ||\vec u|| \ ||\vec v|| \quad \forall \vec u, \vec v \in \mathbb V\]

Triangle Inequality

\[|\langle \vec u, \vec v \rangle| \le ||\vec u|| + ||\vec v|| \quad \forall \vec u, \vec v \in \mathbb V\]

Inner Product for \(C[a, b]\)

\(C[a, b]\) is a vector space⁴ of continuous functions⁶ over the interval⁷ \(a \le t \le b\).
For \(f\) and \(g\) in \(C[a, b]\)

\[\langle f, g \rangle = \int_b^a f(t) g(t) dt\]