Skip to content

44. Application of Inner Product Spaces1

Dated: 22-06-2025

Least Squares Lines

Experiments often produce results in form of pairs \((x, y)\) and when graphed, seem like a line.2 The equation for the line2 can be

\[y = \beta_0 + \beta_1 x\]

We want to determine the parameters \(\beta_0\) and \(\beta_1\) such that those points are as close to the line2 as possible. If the data points are on the line2 then these parameters will satisfy the following equations.

\[ \begin{array}{ccc} \text{Predicted value} & &\text{Observed Value} \\ \beta_0 + \beta_1 x_1 &= &y_1 \\ \beta_0 + \beta_1 x_2 &= &y_2 \\ &\vdots \\ \beta_0 + \beta_1 x_n &= &y_n \\ \end{array} \]

This can be written as

\[X \beta = y\]

Where

\[ X = \begin{bmatrix} 1 & x_1 \\ 1 & x_2 \\ \vdots & \vdots \\ 1 & x_n \\ \end{bmatrix}, \ \beta = \begin{bmatrix} \beta_0 \\ \beta_1 \\ \end{bmatrix}, \ y = \begin{bmatrix} y_0 \\ y_1 \\ \vdots \\ y_n \end{bmatrix} \]

Computing the least squares solution3 of \(X\beta = y\) is equivalent to finding the \(\beta\) that determines the least squares line.

Weighted Least Squares

Imagine a vector4 of \(n\) observations \(y_1, \ldots, y_n\) and suppose we want to approximate it by a vector4 \(y^\prime\) which belongs to some subspace5 of \(\mathbb R^n\), which will to be constructed by the columns of \(A\). Then we find \(x^\prime\) that makes \(A x^\prime = y^\prime\) as close to \(y\) as possible. So that measure of closeness is the weighted error

\[||Wy - Wy^\prime||^2 = ||Wy - WAy^\prime||^2\]

Where

\[ W = \begin{bmatrix} w_1 & 0 & 0 & 0\\ 0 & w_2 & 0 & 0\\ 0 & 0 & \ddots & \vdots\\ 0 & 0 & \ldots & w_n\\ \end{bmatrix} \]

Thus \(x^\prime\) is the ordinary least squares solution3 of the equation

\[WAx = Wy\]

The normal equation for the weighted least squares solution is

\[(WA)^T WAx = (WA)^T Wy\]

What Does Trend Analysis Mean?

An aspect of technical analysis that tries to predict the future movement of a stock based on past data. Trend analysis is based on the idea that what has happened in the past gives traders an idea of what will happen in the future.

Linear Trend

The simplest and most common use of trend analysis occurs when the points \(t_0, t_1, \ldots, t_n\) can be adjusted so that they are evenly spaced and sum to zero.

Example

Fit a quadratic trend function to the data \((-2,3), (-1,5), (0,5), (1,4)\) and \((2,3)\)

\[ \begin{array}{cccc} p_0 & p_1 & p_2 & \text{data}:g\\ \downarrow & \downarrow & \downarrow & \downarrow \\ \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \\ 1 \\ \end{bmatrix} & \begin{bmatrix} -2 \\ -1 \\ 0 \\ 1 \\ 2 \\ \end{bmatrix} & \begin{bmatrix} 2 \\ -1 \\ -2 \\ -1 \\ 2 \\ \end{bmatrix} & \begin{bmatrix} 3 \\ 5 \\ 5 \\ 4 \\ 3 \\ \end{bmatrix} \end{array} \]
\[ p^\prime = \frac{\langle g, p_0 \rangle}{\langle p_0, p_0 \rangle} p_0 + \frac{\langle g, p_1 \rangle}{\langle p_1, p_1 \rangle} p_1 + \frac{\langle g, p_2 \rangle}{\langle p_2, p_2 \rangle} p_2 \]
\[ = \frac {20} 5 p_0 - \frac 1 {10}p_1 - \frac 7 {14} p_2 \]
\[p^\prime(t) = 4 - 0.1t - 0.5(t^2 - 2)\]

Fourier Series

If \(f\) is a \(2 \pi\) periodic function6 then

\[ f(t) = \frac {a_0} 2 + \sum_{m = 1}^\infty (a_m \cos (mt) + b_m \sin(mt)) \]

is called the Fourier series of \(f\) where

\[ a_m = \frac 1 \pi \int_0^{2 \pi} f(t) \cos (mt) dt \]
\[ b_m = \frac 1 \pi \int_0^{2 \pi} f(t) \sin (mt) dt \]

References

Read more about notations and symbols.

  1. Read more about inner product spaces. ↩

  2. Read more about lines. ↩↩↩↩

  3. Read more about least squares solution ↩↩

  4. Read more about vectors. ↩↩

  5. Read more about vector spaces. ↩

  6. Read more about functions. ↩