1. Random Variables and Stochastic Processes
A Random Variable is a deterministic function that maps each possible outcome of a random experiment to a unique real number. A stochastic process is a sequence of these random variables indexed by time, denoted as {Xt : t ∈ T}. In time series applications, T is restricted to a set of discrete integers.
2. Stationarity
For a stochastic process to be compatible with standard linear modeling, it must exhibit stationarity, ensuring its statistical properties are invariant to time shifts.
Strict-Sense Stationarity (SSS)
A process {Xt} is strictly stationary if the joint probability distribution of the vector (Xt1, ..., Xtm) is identical to that of (Xt1+τ, ..., Xtm+τ) for all possible time shifts τ and all m.
Wide-Sense Stationarity (WSS)
A process {Xt} is Wide-Sense Stationary (or covariance stationary) if it possesses a finite second moment (E[Xt2] < ∞) and satisfies two conditions:
- Constant Mean: E[Xt] = μ for all t.
- Time-Invariant Covariance: The autocovariance between any two observations depends solely on the lag τ:
Cov(Xt, Xt+τ) = γ(τ).
(This implies a constant variance, where Var(Xt) = γ(0)).
White Noise (WN)
A process {εt} is defined as white noise if it is a WSS process with E[εt] = 0, constant variance σ2, and Cov(εt, εt+τ) = 0 for all τ ≠ 0. If the variables are also independent and identically distributed (I.I.D.), it is termed "Independent White Noise."