Variational inference and the evidence lower bound
Research context
research method · established · As of 2026-10-10
Variational inference approximates a posterior by choosing q from a specified family to minimize KL(q||posterior).
Key knowledge
Variational inference approximates a posterior by choosing q from a specified family to minimize KL(q||posterior). The evidence lower bound is expected log joint minus expected log q, and its gap to log evidence is that KL divergence. A mean-field family excludes dependence and can understate uncertainty; coordinate ascent can reach different local optima. Optimizing an approximate posterior is not evidence that the probabilistic model is true.
Values and equations
ELBO(q)=E_q log p(x,z)−E_q log q(z); log p(x)=ELBO(q)+KL(q||p(z|x))
- x
- Observed data
- z
- Latent variables
- q
- Candidate approximate posterior
Conditions and scope
- The posterior normalizer exists and q has suitable support; reverse KL can penalize mass outside posterior support infinitely.
- State approximation family, optimization convergence, prior and posterior-predictive validation.
Sources · 1
References
Sources retain their recorded scope. A source link alone does not establish verification of every statement.
Evidence and review
Author checked this English statement, displayed equation where present and stated hypotheses against these source locations: §§2–4: KL/ELBO, mean field, coordinate ascent and MCMC comparisons. The check is bounded to the recorded claims; no complete proof reconstruction, software proof replay or universal literature completeness is claimed.
2026-10-10 · Author source-body/statement/equation/domain check, with graph associations scoped individually. Separate independent content/source/inventory review is required before branch closure.
Entry details
Permanent link to this entry ↗- Record identity
- research-info-variational-inference
- Revision
- research.1
- Structured reference record
- Postgraduate & research reference