AR(2) Model & Yule–Walker Equations Explained Simply
This guide explains autoregressive models and Yule–Walker equations in a simple, step-by-step way. You'll learn how to estimate parameters and understand autocorrelation terms like \(r_1\) and \(r_2\).
1. What is an AR(2) Model?
The AR(2) model predicts a value using its past two values:
\[ x_t = \phi_1 x_{t-1} + \phi_2 x_{t-2} + \varepsilon_t \]
Goal: Estimate \( \phi_1 \) and \( \phi_2 \).
2. Matrix Form (Linear Regression)
We rewrite the model as:
\[ x = A\phi \]
- A: matrix of past values
- \(\phi\): parameters
- x: observed values
Solution:
\[ \hat{\phi} = (A^T A)^{-1} A^T x \]
3. Autocorrelation Terms
We define:
- \( r_0 = E[x_t^2] \) (variance)
- \( r_1 = E[x_t x_{t-1}] \)
- \( r_2 = E[x_t x_{t-2}] \)
4. Yule–Walker Matrix (Toeplitz Structure)
The system becomes:
\[ \begin{bmatrix} r_0 & r_1 \\ r_1 & r_0 \end{bmatrix} \begin{bmatrix} \phi_1 \\ \phi_2 \end{bmatrix} = \begin{bmatrix} r_1 \\ r_2 \end{bmatrix} \]
This matrix is Toeplitz because values depend only on lag.
5. Deriving r₁
Multiply the model by \(x_{t-1}\):
\[ r_1 = E[x_t x_{t-1}] \]
Result:
\[ r_1 = \phi_1 r_0 + \phi_2 r_1 + \phi_3 r_2 + \cdots + \phi_p r_{p-1} \]
The noise term disappears because it is independent of past values.
6. Deriving r₂
Multiply the model by \(x_{t-2}\):
\[ r_2 = E[x_t x_{t-2}] \]
Result:
\[ r_2 = \phi_1 r_1 + \phi_2 r_0 + \phi_3 r_1 + \phi_4 r_2 + \cdots + \phi_p r_{p-2} \]
7. General Yule–Walker Equation
\[ r_k = \phi_1 r_{k-1} + \phi_2 r_{k-2} + \cdots + \phi_p r_{k-p} \]
8. Intuition
Each autocorrelation value is built from past correlations weighted by model coefficients.
- \(r_1\): depends on immediate past
- \(r_2\): depends on deeper past
This creates a structured system that allows solving for model parameters.
9. Why Toeplitz Matrix?
Because the process is stationary:
\[ r_k = r_{-k} \]
So matrix entries depend only on distance \(|i - j|\).
10. Final Takeaway
Yule–Walker equations transform a time series problem into a linear algebra problem.
- Multiply by lagged values
- Take expectations
- Use autocorrelations
- Solve linear system
This is the foundation of many time series estimation methods.