Proton: Prediction Computation Drill

Goal: Given a model and data, compute $P_nX_{n+h}$, MSE, and prediction interval. Covers both the shortcut (AR) and full linear system approaches.

Problem 1: AR(1) One-Step

$X_t = 0.7X_{t-1} + Z_t$, $\sigma^2 = 9$. Observed: $X_{10} = 5.2$. Mean $\mu = 0$.

Compute $P_{10}X_{11}$ and 95% prediction interval.

Answer 1

Shortcut: $P_{10}X_{11} = 0.7 \times 5.2 = 3.64$.

MSE = $\sigma^2 = 9$.

95% PI: $3.64 \pm 1.96\sqrt{9} = 3.64 \pm 5.88 = (-2.24,\; 9.52)$.

Problem 2: AR(1) Two-Step

Same model. Compute $P_{10}X_{12}$.

Answer 2

$X_{12} = 0.7X_{11} + Z_{12}$.

$P_{10}X_{12} = 0.7 \cdot P_{10}X_{11} + P_{10}Z_{12}$.

$P_{10}Z_{12} = E(Z_{12}) = 0$ (future noise, uncorrelated with past).

$P_{10}X_{12} = 0.7 \times 3.64 = 2.548$.

More generally, $P_n X_{n+h} = \phi^h X_n$ for causal AR(1) with zero mean.

MSE for $h$-step AR(1): $\text{MSE}_h = \sigma^2 \sum_{j=0}^{h-1}\phi^{2j} = \sigma^2 \cdot \frac{1-\phi^{2h}}{1-\phi^2}$.

For $h = 2$: $\text{MSE}_2 = 9 \cdot \frac{1 - 0.49}{1 - 0.49} = 9 \cdot \frac{0.51}{0.51}$…

Wait, let’s be precise: $\text{MSE}_2 = 9(1 + 0.49) = 9 \times 1.49 = 13.41$.

95% PI: $2.548 \pm 1.96\sqrt{13.41} = 2.548 \pm 7.18 = (-4.63,\; 9.73)$.

Note: prediction interval widens as forecast horizon increases — this is because MSE grows.

Problem 3: Full Linear System

$\{X_t\}$ is stationary with $\mu = 0$, $\gamma(0) = 4$, $\gamma(1) = 2$, $\gamma(2) = 1$, $\gamma(h) = 0$ for $h \geq 3$.

You observe $X_1 = 1, X_2 = 3$. Compute $P_2 X_3$.

Answer 3

Set up $\Gamma_2 \mathbf{a} = \boldsymbol{\gamma}_2(1)$:

$$\begin{pmatrix} 4 & 2 \\ 2 & 4 \end{pmatrix}\begin{pmatrix}a_1 \\ a_2\end{pmatrix} = \begin{pmatrix}\gamma(1) \\ \gamma(2)\end{pmatrix} = \begin{pmatrix}2 \\ 1\end{pmatrix}$$

Note on ordering: $a_1$ multiplies $(X_2 - \mu) = X_2$ and $a_2$ multiplies $(X_1 - \mu) = X_1$. The covariance vector $\boldsymbol{\gamma}_2(1) = (\gamma(1), \gamma(2))'$ — entry $i$ is $\text{Cov}(X_3, X_{3-i}) = \gamma(i)$.

Solve: $4a_1 + 2a_2 = 2$ and $2a_1 + 4a_2 = 1$.

From eq. 1: $a_1 = (2 - 2a_2)/4 = 0.5 - 0.5a_2$.

Substitute into eq. 2: $2(0.5 - 0.5a_2) + 4a_2 = 1$ → $1 - a_2 + 4a_2 = 1$ → $3a_2 = 0$ → $a_2 = 0$.

Then $a_1 = 0.5$.

$$P_2 X_3 = 0.5 \cdot X_2 + 0 \cdot X_1 = 0.5 \times 3 = 1.5$$

MSE: $\gamma(0) - \mathbf{a}'\boldsymbol{\gamma}(1) = 4 - (0.5 \times 2 + 0 \times 1) = 4 - 1 = 3$.

95% PI: $1.5 \pm 1.96\sqrt{3} = 1.5 \pm 3.39 = (-1.89,\; 4.89)$.

Observation: $a_2 = 0$ means $X_1$ contributes nothing once $X_2$ is known. This ACVF (cutoff at lag 2) corresponds to an MA(2) process — and for MA(2), observations more than $q = 2$ lags ago are indeed irrelevant.

Key Takeaways

  1. For causal AR(p): use the shortcut $P_nX_{n+1} = \sum \phi_i X_{n+1-i}$. One-step MSE $= \sigma^2$.
  2. For general stationary process: set up $\Gamma_n\mathbf{a} = \boldsymbol{\gamma}_n(h)$, solve, compute MSE $= \gamma(0) - \mathbf{a}'\boldsymbol{\gamma}_n(h)$.
  3. PI widens with forecast horizon (MSE increases).
  4. The ordering convention of $\Gamma_n$ and $\boldsymbol{\gamma}_n(h)$: first row/entry corresponds to the most recent observation $X_n$.