Finite-alphabet mutual information
Mutual information measures statistical dependence in bits.
Key knowledge
For finite random variables X,Y, I(X;Y)=Σₓ,ᵧ p(x,y)log₂[p(x,y)/(p_X(x)p_Y(y))]=H(X)−H(X|Y). It is nonnegative and zero exactly when X and Y are independent.
Conditions and scope
- Zero joint-mass terms contribute zero.
- Statistical dependence does not identify causation or semantic meaning.
Sources · 1
References
Sources retain their recorded scope. A source link alone does not establish verification of every statement.
Evidence and review
Definitions 4–6: finite PMF relative entropy, conditional entropy and mutual information; nonnegativity and independence equality.
2026-10-10 · Finite discrete mutual information and equality condition.
Entry details
Permanent link to this entry ↗- Record identity
- info-mutual-information
- Revision
- university-reference.1
- Structured reference record
- University reference