AR(2) Model Explained: Step-by-Step Time Series Estimation

This guide explains how to estimate parameters in a second-order autoregressive model (AR(2)) using simple math and intuition.

1. The Model

The AR(2) model predicts a value using its past two values:

x_t = φ₁x_t-1 + φ₂x_t-2 + ε_t

Goal: Estimate φ₁ and φ₂.

2. Matrix Form

We rewrite the model like a linear regression problem:

x = Aφ

• A: matrix of past values
• φ: parameters (φ₁, φ₂)
• x: current values

3. Least Squares Solution

The best estimate is:

φ̂ = (A^TA)^-1 A^Tx

4. Understanding A^TA

This matrix contains sums of products (autocorrelations):

A^TA =
[ c₀    c₁ ]
[ c₁    c₀ ]

• c₀ = Σ x_t²
• c₁ = Σ x_tx_t-1

5. Understanding A^Tx

A^Tx =
[ c₁ ]
[ c₂ ]

• c₂ = Σ x_tx_t-2

6. Solve the System

We solve:

[ c₀    c₁ ] [ φ₁ ] = [ c₁ ]
[ c₁    c₀ ] [ φ₂ ] [ c₂ ]

7. Final Formulas

φ₁ = (c₁(c₀ − c₂)) / (c₀² − c₁²)

φ₂ = (c₂c₀ − c₁²) / (c₀² − c₁²)

You are predicting the current value using the last two values. The coefficients depend on how strongly the data correlates with its past.

8. Why It Gets Harder for Higher Order Models

For AR(3) and beyond, the matrices become larger and harder to invert. That’s why methods like Yule-Walker equations are used in practice.

Summary

The AR(2) model finds the best weights to predict current values from past values using correlations in the data.