Skip to content

38. Inner Product

Dated: 18-06-2025

Let \(\vec u\) and \(\vec v\) be vectors1 in \(\mathbb R^n\) represented by \(n \times 1\) matrices.2 The transpose2 \(u^T\) is a \(1 \times n\) matrix2 and their product \(u^T \cdot v\) is a \(1 \times 1\) matrix2 called the inner product, which we can write as a simple scalar value.

The inner product is also referred to as dot product.1

\[ \vec u = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \\ \end{bmatrix} \text{ and } \vec v = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \\ \end{bmatrix} \]
\[ \vec u^T \cdot \vec v = \begin{bmatrix} u_1 & u_2 & \cdots & u_n \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \\ \end{bmatrix} = u_1v_1 + u_2v_2 + \cdots + u_nv_n \]

Theorem

Let \(\vec u, \vec v, \vec w \in \mathbb R^n\) and \(c\) be a scalar then

  • \(\vec u \cdot \vec v = \vec v \cdot \vec u\)
  • \((\vec u + \vec v) \cdot \vec w = \vec u \cdot \vec w + \vec v \cdot \vec w\)
  • \((c \vec u) \cdot \vec v = c(\vec u \cdot \vec v) = \vec u (c \vec v)\)
  • \(\vec u \cdot \vec u \ge 0\) and \(\vec u \cdot \vec u = 0 \iff \vec u = 0\)

Length or Norm

\[||\vec v|| = \sqrt{\vec v \cdot \vec v} = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}\]
\[||v||^2 = \vec v \cdot \vec v\]

\(||c \vec v|| = |c| \ ||\vec v||\)

Normalization

The process of producing a unit vector1 \(u\) from a vector1 \(\vec v\) is called normalization.

Distance

Let \(\vec u, \vec v \in \mathbb R^n\) and distance between \(\vec u\) and \(\vec v\) is

\[\text{dist}(\vec u, \vec v) = ||\vec u - \vec v||\]

Orthogonal Complements

The set of all vectors1 \(\vec z\) that are orthogonal to \(\vec w \in \mathbb W\) is called the orthogonal complement of \(\mathbb W\) and is denoted by \(\mathbb W^{\perp}\)

Remarks

  • \[\vec x \in \mathbb W^\perp \iff \vec x \perp \forall \vec v \in \text{Span}(\mathbb W)\]

  • \(\mathbb W^\perp\) is a subspace3 of \(\mathbb R^n\)

Theorem

Let \(A\) be \(m \times n\) matrix2 then the orthogonal complement of the row space3 of \(A\) is the null space4 of \(A\). Similarly, the orthogonal complement of the column space3 of \(A\) is the null space4 of \(A^T\).

\[(\text{Row } A)^\perp = \text{Nul } A\]
\[(\text{Col } A)^\perp = \text{Nul } A^T\]

References

Read more about notations and symbols.

  1. Read more about vectors. ↩↩↩↩↩

  2. Read more about matrices. ↩↩↩↩↩

  3. Read more about vector spaces. ↩↩↩

  4. Read more about null spaces. ↩↩