ARMA Process and Wide-Sense Stationarity (WSS): A Comprehensive Guide ARMA Process and Wide-Sense Stationarity (WSS) Core Concept: An ARMA (AutoRegressive Moving Average) process is Wide-Sense Stationary (WSS) if and only if the system's autoregressive part is stable and the input is a stationary white noise process. 1. The ARMA(p, q) Model Equation A stochastic process {X t } follows an ARMA( p, q ) model if it satisfies the following linear difference equation: X t = φ 1 X t-1 + φ 2 X t-2 + ... + φ p X t-p + a t + θ 1 a t-1 + θ 2 a t-2 + ... + θ q a t-q Where: φ 1 , ..., φ p : Autoregressive (AR) parameters. θ 1 , ..., θ q : Moving Average (MA) parameters. a t : A white noise process (the innovation). ...

Relationship Between WSS, LTI, AR, MA and ARMA Models | Signal Processing Guide The Unified Relationship Between WSS, LTI, AR, MA, and ARMA Processes An in-depth exploration of how stochastic processes and linear systems interact in digital signal processing and time-series analysis. In modern signal processing, the interaction between Wide-Sense Stationary (WSS) random processes and Linear Time-Invariant (LTI) systems forms the backbone of statistical modeling. Models like AR (Autoregressive) , MA (Moving Average) , and ARMA are not just statistical tools—they are specific realizations of white noise filtered through LTI systems. 1. Defining the Wide-Sense Stationary (WSS) Process A random process \(x[n]\) is categorized as Wide-Sense Stationary if its first and second-order moments are invariant to time shifts: Constant Mean: $$E[x[n]] = \mu$$ Time-Dependent Autocorrelation: The cor...

  The Yule-Walker equations are fundamentally derived using the properties of a Wide-Sense Stationary (WSS) random process. Without the WSS assumption, the classical Yule-Walker equations cannot be obtained in their standard form. What is a Wide-Sense Stationary (WSS) Process? A random process \(X(t)\) is said to be Wide-Sense Stationary (WSS) if it satisfies three important conditions. 1. Constant Mean $$ E[X(t)] = \mu $$ where the mean \(\mu\) does not depend on time. 2. Constant Variance $$ Var(X(t))=\sigma^2 $$ The variance remains unchanged over time. 3. Autocovariance Depends Only on Lag Instead of depending on two different time instants, $$ C_X(t_1,t_2), $$ the covariance depends only on their difference, $$ C_X(\tau) = C_X(t_1-t_2). $$ Similarly, the autocorrelation function becomes $$ R_X(\tau) = E[X(t)X(t+\tau)]. $$ This property is the key assumption used in deriving the Yule-Walker equations. What are the Yule-Walker Equa...

1. Random Variables and Stochastic Processes A Random Variable is a deterministic function that maps each possible outcome of a random experiment to a unique real number. A stochastic process is a sequence of these random variables indexed by time, denoted as {X t : t ∈ T}. In time series applications, T is restricted to a set of discrete integers. 2. Stationarity For a stochastic process to be compatible with standard linear modeling, it must exhibit stationarity , ensuring its statistical properties are invariant to time shifts. Strict-Sense Stationarity (SSS) A process {X t } is strictly stationary if the joint probability distribution of the vector (X t 1 , ..., X t m ) is identical to that of (X t 1 +τ , ..., X t m +τ ) for all possible time shifts τ and all m . Wide-Sense Stationarity (WSS) A process {X t } is Wide-Sense Stationary (or covariance stationary) if it possess...

Can a Sinusoid Be Wide-Sense Stationary (WSS)? A noisy sinusoidal is not itself a Wide-sense stationary (WSS) signal. Sinusoidal signal + Random phase = WSS In the code below it demonstrates that how sinusoidal signal can be turned into wide-sense stationary signal where it captures the property like constant mean and same auto-correlation value over time. MATLAB Code % Generate Noisy Sinusoid (Single Script) clc; clear; close all; %% Parameters N = 1000; % Number of samples freq = 5; % Frequency in Hz sampleRate = 100; % Sampling rate in Hz meanValue = 2; % Desired mean variance = 0.5; % Desired variance %% Amplitude from variance amp = sqrt(2 * variance); %% Random phase theta = 2 * pi * rand(); %% Time vector n = 0:N-1; t = n / sampleRate; %% Generate sinusoidal signal signal = meanValue + amp * cos(2 * pi * freq * t + theta); %% Plot signal figure; plo...

A strict-sense stationary process is one whose complete statistical structure does not change with time — all joint distributions remain identical under time shifts. For SSS, the following all must be time-invariant: Mean Variance Autocorrelation All higher-order moments All joint PDFs All marginal PDFs Everything . That’s why SSS is a very strong condition. A random process X ( t ) X(t) X ( t ) is strict-sense stationary if for every : Positive integer n n n Any set of time instants t 1 , t 2 , … , t n t_1, t_2, \dots, t_n t 1 ​ , t 2 ​ , … , t n ​ Any time shift τ \tau τ the joint distribution satisfies: F X ( t 1 ) , … , X ( t n ) ( x 1 , … , x n ) = F X ( t 1 + τ ) , … , X ( t n + τ ) ( x 1 , … , x n ) A Large Strictly Stationary Example Let’s define a random process where: X(t) = 0 with probability 1/2 X(t) = 1 with probability 1/2 Each time point is independent. This is like an infinite sequence of fair coin flips. t:...

Impulse Response h[n] of an ARMA System Step-by-Step Solution 1. Start with the Transfer Function: Given: H(z) = (1 + 0.3z⁻¹) / (1 - 0.75z⁻¹ + 0.5z⁻²) This is an ARMA(2,1) system where: MA (numerator) coefficients: [1, 0.3] AR (denominator) coefficients: [1, -0.75, 0.5] 2. Find the Impulse Response h[n]: We want the inverse Z-transform of H(z). Instead of doing partial fraction decomposition, we use the system's difference equation. 3. Recursive Computation Using the Difference Equation: From the system equation: y[n] + 0.75y[n−1] − 0.5y[n−2] = x[n] + 0.3x[n−1] Assume x[n] = δ[n] (unit impulse): x[0] = 1 , others are 0. Then y[n] = h[n] 4. Compute h[n] values: n = 0: y[0] + 0 = 1 → h[0] = 1 n = 1: y[1] + 0.75*1 = 0.3 → h[1] = -0.45 n = 2: y[2] - 0.3375 - 0.5 = 0 → h[2] = 0.8375 n = 3: y[3] + 0.628125 + 0.225 = 0 → h[3] = -0.853125 5. Table of Fi...

Q & A and Summary Stationary and Wide Sense Stationary Process A stochastic process {…, X t-1 , X t , X t+1 , X t+2 , …} consisting of random variables indexed by time index t is a time series. The stochastic behavior of {X t } is determined by specifying the probability density or mass functions (pdf’s): p(x t1 , x t2 , x t3 , …, x tm ) for all finite collections of time indexes {(t 1 , t 2 , …, t m ), m < ∞} i.e., all finite-dimensional distributions of {X t }. A time series {X t } is strictly stationary if p(t 1 + τ, t 2 + τ, …, t m + τ) = p(t 1 , t 2 , …, t m ) , ∀τ, ∀m, ∀(t 1 , t 2 , …, t m ) . Where p(t 1 + τ, t 2 + τ, …, t m + τ) represents the cumulative distribution function of the unconditional (i.e., with no reference to any particular starting value) joint distribution. A process {X t } is said to be strictly stationary or strict-sense stationary if τ doesn’t affect the function p. Thus, p is not a function of time. A time series {X t } ...

  Autocorrelation function Autocorrelation function: For a signal x(t), the autocorrelation is defined as R xx (τ) = E[x(t)x(t+τ)] for a random process, or R xx (τ) = ∫ x(t)x(t+τ) dt for an energy signal. The auto-correlation of a periodic signal preserves the periodicity. For example, we are transmitting a signal x(t) through the wireless medium, and we receive the signal y(t) at the receiver. y(t) = x(t) + n(t) where n(t) is the additive white Gaussian noise (AWGN) . You can find that the periodicity of the autocorrelation of y(t) will be the same as the periodicity of x(t) . In other words, we can say that the autocorrelation of the noisy signal is equal to the autocorrelation of the original periodic signal, except at zero lag (τ = 0), where the noise contributes. To find the spectral density (also known as the power spectral density, or PSD) from the autocorrelation function mathematically, you can use the Wiener–Khinchin theorem. This theorem states th...