Stochastic gradient and the assumptions behind convergence
Research context
research method · established · As of 2026-10-10
Stochastic gradient updates parameters using a sampled gradient or subgradient.
Key knowledge
Stochastic gradient updates parameters using a sampled gradient or subgradient. For convex objectives, unbiased estimates with bounded second moment and suitable steps yield expected optimization-error bounds; strong convexity changes the rate. For general nonconvex objectives, a guarantee about small expected gradient norm is a stationarity statement, not proof of a global minimum, semantic correctness or out-of-distribution performance.
Values and equations
theta_(t+1)=theta_t−eta_t g_t
- theta_t
- Parameter vector
- eta_t
- Positive step size
- g_t
- Sampled gradient or subgradient estimator
Conditions and scope
- The gradient estimator's bias/variance, objective regularity, sampling scheme and step schedule are part of each bound.
- Finite precision, correlated minibatches and changing data can violate an ideal convergence hypothesis.
Sources · 1
References
Sources retain their recorded scope. A source link alone does not establish verification of every statement.
Evidence and review
Author checked this English statement, displayed equation where present and stated hypotheses against these source locations: Chapters 2–6 PAC/VC, 11 validation, 13 regularization, 14 stochastic gradient and 23 PCA. The check is bounded to the recorded claims; no complete proof reconstruction, software proof replay or universal literature completeness is claimed.
2026-10-10 · Author source-body/statement/equation/domain check, with graph associations scoped individually. Separate independent content/source/inventory review is required before branch closure.
Entry details
Permanent link to this entry ↗- Record identity
- research-info-stochastic-gradient
- Revision
- research.1
- Structured reference record
- Postgraduate & research reference