The Yule-Walker equations are fundamentally derived using the properties of a Wide-Sense Stationary (WSS) random process. Without the WSS assumption, the classical Yule-Walker equations cannot be obtained in their standard form.
What is a Wide-Sense Stationary (WSS) Process?
A random process \(X(t)\) is said to be Wide-Sense Stationary (WSS) if it satisfies three important conditions.
1. Constant Mean
$$ E[X(t)] = \mu $$ where the mean \(\mu\) does not depend on time.2. Constant Variance
$$ Var(X(t))=\sigma^2 $$ The variance remains unchanged over time.3. Autocovariance Depends Only on Lag
Instead of depending on two different time instants, $$ C_X(t_1,t_2), $$ the covariance depends only on their difference, $$ C_X(\tau) = C_X(t_1-t_2). $$ Similarly, the autocorrelation function becomes $$ R_X(\tau) = E[X(t)X(t+\tau)]. $$ This property is the key assumption used in deriving the Yule-Walker equations.What are the Yule-Walker Equations?
The Yule-Walker equations describe the relationship between the autocorrelation function of a stationary process and the coefficients of an Autoregressive (AR) model.
Consider an AR(p) process: $$ X_t = \phi_1X_{t-1} + \phi_2X_{t-2} + \cdots + \phi_pX_{t-p} + \varepsilon_t $$ where- \(\phi_1,\phi_2,\ldots,\phi_p\) are AR coefficients.
- \(\varepsilon_t\) is white noise with
Derivation Using the WSS Assumption
Multiply both sides of the AR equation by \(X_{t-k}\) and take expectations.
$$ E[X_tX_{t-k}] = \sum_{i=1}^{p} \phi_i E[X_{t-i}X_{t-k}] $$ Because the process is WSS, $$ E[X_tX_{t-k}] = \gamma(k), $$ where $$ \gamma(k) = Cov(X_t,X_{t-k}) $$ depends only on the lag \(k\). Therefore, $$ \boxed{ \gamma(k) = \sum_{i=1}^{p} \phi_i \gamma(k-i) } $$ for $$ k\ge1. $$ These are known as the Yule-Walker equations.Matrix Form of the Yule-Walker Equations
For an AR(p) process, $$ \begin{bmatrix} \gamma(0) & \gamma(1) & \cdots & \gamma(p-1)\\ \gamma(1) & \gamma(0) & \cdots & \gamma(p-2)\\ \vdots & \vdots & \ddots & \vdots\\ \gamma(p-1) & \gamma(p-2) & \cdots & \gamma(0) \end{bmatrix} \begin{bmatrix} \phi_1\\ \phi_2\\ \vdots\\ \phi_p \end{bmatrix} = \begin{bmatrix} \gamma(1)\\ \gamma(2)\\ \vdots\\ \gamma(p) \end{bmatrix} $$ The covariance matrix is a Toeplitz matrix, which occurs only because the covariance depends solely on lag—a direct consequence of WSS.Why is WSS Essential?
If a process is not stationary, then
$$ \gamma(t_1,t_2) \neq \gamma(t_1-t_2). $$Instead, covariance depends on both time indices independently. As a result:
- The covariance matrix is no longer Toeplitz.
- The Yule-Walker equations lose their standard form.
- Estimating AR coefficients becomes significantly more difficult.
Relationship Between WSS and the Yule-Walker Equations
In summary:
- Wide-Sense Stationarity (WSS) is the fundamental assumption.
- It ensures that autocovariance depends only on lag.
- This property enables the derivation of the Yule-Walker equations.
- The equations are specifically used for estimating the coefficients of stationary AR models.
- Although every stationary AR process is WSS (under standard stability conditions), not every WSS process is autoregressive.
Frequently Asked Questions (FAQs)
Is stationarity required for the Yule-Walker equations?
Yes. The derivation relies on the covariance function depending only on lag, which is a defining property of wide-sense stationary processes.
Can Yule-Walker estimate MA model parameters?
No. The classical Yule-Walker equations are designed for autoregressive (AR) models. Other estimation techniques are generally used for MA and ARMA models.
Why is the covariance matrix Toeplitz?
Because, under WSS, each covariance entry depends only on the lag between two observations rather than their absolute time indices.