Relationship Between WSS, LTI, AR, MA and ARMA Models

The Unified Relationship Between WSS, LTI, AR, MA, and ARMA Processes

An in-depth exploration of how stochastic processes and linear systems interact in digital signal processing and time-series analysis.

In modern signal processing, the interaction between Wide-Sense Stationary (WSS) random processes and Linear Time-Invariant (LTI) systems forms the backbone of statistical modeling. Models like AR (Autoregressive), MA (Moving Average), and ARMA are not just statistical tools—they are specific realizations of white noise filtered through LTI systems.

---

1. Defining the Wide-Sense Stationary (WSS) Process

A random process \(x[n]\) is categorized as Wide-Sense Stationary if its first and second-order moments are invariant to time shifts:

1. Constant Mean:
   $$E[x[n]] = \mu$$
2. Time-Dependent Autocorrelation: The correlation between two points depends only on their lag \(k\):
   $$R_x(n_1, n_2) = R_x(n_1 - n_2) = R_x[k]$$ $$R_x[k] = E[x[n]x[n-k]]$$

The Power Spectral Density (PSD), defined by the Wiener-Khinchin theorem, is the Fourier Transform of the autocorrelation:

$$S_x(\omega) = \sum_{k=-\infty}^{\infty} R_x[k]e^{-j\omega k}$$

2. Characterizing the Linear Time-Invariant (LTI) System

An LTI system is defined by its impulse response \(h[n]\). Its behavior in the frequency domain is described by the transfer function:

$$H(z) = \sum_{n=-\infty}^{\infty} h[n]z^{-n}$$

For an input \(x[n]\), the output \(y[n]\) is the convolution:

$$y[n] = x[n] * h[n] \iff Y(z) = H(z)X(z)$$

3. The Fundamental WSS-LTI Relationship

When a WSS process passes through a stable LTI system, the output remains WSS. The transformation is governed by these key properties:

Mean and Autocorrelation

$$\mu_y = \mu_x \sum_{n=-\infty}^{\infty} h[n]$$ $$R_y[k] = R_x[k] * h[k] * h[-k]$$

Spectral Shaping Equation

Perhaps the most critical result in stochastic filtering is how the system "shapes" the input spectrum:

$$S_y(\omega) = |H(e^{j\omega})|^2 S_x(\omega)$$

4. White Noise: The Stochastic Foundation

Most parametric models are built by filtering White Noise \(w[n]\), which has zero mean and a constant PSD:

$$R_w[k] = \sigma^2 \delta[k] \implies S_w(\omega) = \sigma^2$$

5. Moving Average (MA) Model: The FIR Perspective

An MA(q) process is a finite linear combination of current and past white noise terms:

$$x[n] = w[n] + b_1w[n-1] + \dots + b_qw[n-q]$$

Interpretation: White Noise → FIR Filter → MA Process

The PSD is shaped by the zeros of the system \(B(z)\):

$$S_x(\omega) = \sigma^2 |B(e^{j\omega})|^2$$

6. Autoregressive (AR) Model: The All-Pole Perspective

An AR(p) process expresses the current value as a linear combination of its own past, plus white noise:

$$x[n] = \sum_{i=1}^{p} a_ix[n-i] + w[n]$$

Interpretation: White Noise → IIR (All-Pole) Filter → AR Process

The PSD is inversely proportional to the magnitude of the feedback coefficients:

$$S_x(\omega) = \frac{\sigma^2}{|A(e^{j\omega})|^2}$$

7. ARMA Model: The Pole-Zero Perspective

The ARMA(p,q) model is the most general parametric form, utilizing both autoregressive and moving average components:

$$A(z)X(z) = B(z)W(z) \implies H(z) = \frac{B(z)}{A(z)}$$

Interpretation: White Noise → General LTI (Pole-Zero) Filter → ARMA Process

8. Unified Comparison Table

Model	LTI Filter Type	Transfer Function \(H(z)\)	Power Spectral Density \(S_x(\omega)\)
White Noise	All-Pass (None)	\(1\)	\(\sigma^2\)
MA(q)	FIR (Zeros only)	\(B(z)\)	\(\sigma^2 |B(e^{j\omega})|^2\)
AR(p)	IIR (Poles only)	\(\frac{1}{A(z)}\)	\(\frac{\sigma^2}{|A(e^{j\omega})|^2}\)
ARMA(p,q)	General Pole-Zero	\(\frac{B(z)}{A(z)}\)	\(\sigma^2 \frac{|B(e^{j\omega})|^2}{|A(e^{j\omega})|^2}\)

9. Summary

Theorem of Representation:

If \(w[n]\) is white noise (WSS) and \(H(z)\) is a stable LTI system, the output \(x[n] = H(z)w[n]\) is always WSS. This establishes that:

• FIR Filters produce MA processes.
• All-pole IIR Filters produce AR processes.
• Pole-zero Filters produce ARMA processes.

Thus, AR, MA, and ARMA models are simply classes of WSS processes generated by specific LTI filter architectures.

Return to Signal Processing Main Page →