Shannon’s Theorem vs. Shannon-Hartley Theorem: Full Comparison

Information Theory vs. Physical Reality

Distinguishing Shannon's Noisy-Channel Coding Theorem from the Shannon-Hartley Formula.

In the pantheon of communication engineering, Claude Shannon’s 1948 paper, "A Mathematical Theory of Communication," serves as the bedrock. However, students and engineers often conflate his general theorem on error-free communication with the specific formula used to calculate bandwidth capacity. To understand modern telecommunications, one must distinguish between the Existence Proof and the Physical Bound.

A. Shannon's Noisy-Channel Coding Theorem

The Fundamental Existence Proof

This theorem is the universal law of reliability. It states that for any communication channel, there exists a capacity $C$ such that if the information rate $R$ is less than $C$, then an error-correcting code exists that allows the probability of error to approach zero ($P_e \to 0$).

The General Mutual Information Capacity:

C = maxP(x) I(X; Y)

Key Mathematical Implications:

• Entropy and Mutual Information: $I(X; Y) = H(X) - H(X|Y)$. This defines capacity as the maximum reduction in uncertainty about the transmitter $X$ given the received signal $Y$.
• The $P_e$ Threshold: If $R > C$, the probability of error is bounded away from zero; reliable communication is impossible regardless of coding complexity.
• Non-Constructive: The theorem proves a code exists but does not define the algorithm to build it.

B. The Shannon-Hartley Theorem

The Physical Channel Limit

This is a specific application of the general theorem. It applies the concept of capacity to a continuous-time analog channel subject to Additive White Gaussian Noise (AWGN).

The AWGN Capacity Formula:

C = B log2(1 + S/N)

Key Mathematical Implications:

• Logarithmic Scaling: Doubling the Signal Power ($S$) does not double the capacity; it only increases it logarithmically.
• Linear Bandwidth Scaling: Doubling the Bandwidth ($B$) provides a linear increase in capacity (assuming $S/N$ remains constant).
• The Gaussian Assumption: This formula assumes noise follows a normal distribution, which represents the "worst-case" interference for a given power.

Comparative Analysis

Feature	Shannon’s Coding Theorem	Shannon-Hartley Theorem
Scope	Abstract / Information Logic	Physical / Electromagnetic
Channel Type	Any (Discrete or Continuous)	AWGN (Analog Band-limited)
Variables	Probability Distributions ($P(x)$)	Watts, Hertz, Joules
Operational Goal	Reliability (Error Correction)	Efficiency (Spectral Bounds)

The Mathematical Bridge: Spectral Efficiency

To bridge these two concepts, we analyze Spectral Efficiency ($\eta$), measured in bits per second per Hertz (bps/Hz).

The relationship between the Shannon-Hartley bound and energy efficiency is expressed by:

η = C / B = log2(1 + S/N)

By substituting $S/N = (E_b/N_0) \cdot (R/B)$, and setting $R = C$ (at the limit):

Eb/N0 = (2η - 1) / η

Physical Interpretation:

This equation allows us to visualize the trade-off between power and bandwidth. As bandwidth $B \to \infty$ (and thus $\eta \to 0$), we apply L'Hôpital's rule to the expression above, which yields the Ultimate Shannon Limit:

E_b/N₀ = ln(2) ≈ -1.59 dB

"Shannon's Theorem promises us that perfect communication is possible; the Shannon-Hartley Theorem defines the boundaries of the universe in which we must build it."

Return to BER vs SNR Main Page →

Constellation Diagrams Main Page →