GLOBAL KNOWLEDGE ARK

Reference library

Foundations

Explore concepts, laws, equations and methods, with clear explanations of their use and sources. Choose an area of study or find a specific entry.

More filters

Reference entries

Showing 151–180 of 1709 reference entries

Mathematics & logicTotal derivative in finite dimensions

Differentiability means a linear approximation with smaller-order error.

DefinitionMathematics & logicDouble integrals on simple regions

A continuous double integral can be evaluated by slicing.

Mathematical statementMathematics & logicGradient, divergence and curl

These coordinate derivatives have different input and output types.

DefinitionMathematics & logicStokes' circulation theorem

Circulation around a boundary equals oriented curl flux.

Mathematical statementMathematics & logicDivergence theorem

Outward boundary flux equals integrated divergence.

Mathematical statementMathematics & logicLocal existence and uniqueness for autonomous ODEs

Local Lipschitz continuity ensures one local trajectory.

Mathematical statementMathematics & logicConstant-coefficient linear ODE systems

The matrix exponential solves a finite linear system.

Mathematical statementMathematics & logicLaplace, heat and wave equation problems

A PDE becomes a solution problem only with a domain and data.

ModelMathematics & logicRandom variables and probability distributions

A distribution assigns probabilities to a random variable's values.

DefinitionMathematics & logicNormal distribution

A normal model is specified by location and positive scale.

DefinitionMathematics & logicBinomial and Poisson count models

Different count mechanisms require different distributions.

DefinitionMathematics & logicWeak law of large numbers

A sample mean concentrates near its mean under stated assumptions.

Mathematical statementMathematics & logicClassical iid central limit theorem

Centered sample averages have an asymptotic normal law.

Mathematical statementMathematics & logicConditional masses and densities

A conditional distribution renormalizes a joint law.

DefinitionMathematics & logicLikelihood and Bayesian updating

Likelihood is a function of a parameter given observed data.

MethodMathematics & logicNull tests, significance and p-values

A p-value is calibrated under a specified null model.

MethodMathematics & logicSampling, randomization and confounding

Selection and treatment assignment answer different questions.

MethodMathematics & logicPermutation inference under exchangeability

Distribution-free procedures still require assumptions.

MethodInformationFinite-alphabet mutual information

Mutual information measures statistical dependence in bits.

DefinitionInformationNear-lossless source coding threshold

Entropy bounds asymptotic compression for an iid source.

Mathematical statementInformationDiscrete memoryless channel capacity

Capacity is the maximal asymptotically reliable rate.

Mathematical statementInformationHamming distance and error correction

Separated codewords permit bounded symbol-error recovery.

Mathematical statementInformationInternet protocol layers

Protocol layers divide communication responsibilities.

DefinitionInformationTCP reliable byte-stream transport

TCP orders bytes and retransmits detected losses.

MethodInformationEncryption, hashes and message authentication

Cryptographic primitives serve distinct security properties.

DefinitionInformationPublic-key digital signatures

A signing key and a verification key have different roles.

DefinitionInformationComputable reductions and undecidability

Reduction direction determines which impossibility transfers.

Mathematical statementInformationDecision classes P and NP

Polynomial solving and polynomial verification differ.

DefinitionInformationNP-hardness and NP-completeness

NP-completeness combines verification and reduction hardness.

DefinitionInformationBounded-error randomized computation

Random choices permit a stated probability of error.

Definition
Mathematical statementSource checked

Discrete memoryless channel capacity

Information

Capacity is the maximal asymptotically reliable rate.

Key knowledge

For a finite discrete memoryless channel p(y|x), C=max_pₓ I(X;Y) bits per channel use. Rates below C allow vanishing average block error; rates above C do not.

Conditions and scope

  • Channel uses follow the same transition law independently conditional on inputs.
  • No feedback or extra input-cost constraint is included; finite-block performance is a separate question.
Sources · 2

References

Sources retain their recorded scope. A source link alone does not establish verification of every statement.

  1. Tsachy Weissman / Stanford University2 sources
    1. EE276 2024, Lectures 7 and 10Tsachy Weissman / Stanford UniversityCourse notes · PDFweb.stanford.eduRead source
    2. EE276 2024, Channel Coding ConverseTsachy Weissman / Stanford UniversityCourse notes · PDFweb.stanford.eduRead source

Evidence and review

Definitions 4–6 and Theorem 7: achievable rate, memoryless channel and maximal mutual information. Opening channel-coding theorem: rates below C achievable, rates above C not achievable with vanishing average block error.

2026-10-10 · Finite DMC capacity and coding threshold.

Related reference entries

Explore relationships within this area of study and its connections to other areas.

Showing 1–4 of 4 relationships

Scientific relationships · 4 / 4

The relationship type describes the connection. Its linked statements, sources and conditions define the scope.

  1. Uses definition
    Relationship sources · 1

    Linked statements

    • For a finite discrete memoryless channel p(y|x), C=max_pₓ I(X;Y) bits per channel use. Rates below C allow vanishing average block error; rates above C do not.
  2. Uses definition
    Relationship sources · 1

    Linked statements

    • For Y=X+Z with iid real Gaussian noise Z∼N(0,N), N>0, independent of input, and average input power E[X²]≤P, capacity is C=(1/2)log₂(1+P/N) bits per channel use. Gaussian input attains the information maximum.
  3. Related to
    Relationship sources · 2

    Linked statements

    • For a two-sender discrete memoryless MAC with independent messages, the capacity region is the closure of rates bounded by I(X1;Y|X2,Q), I(X2;Y|X1,Q) and I(X1,X2;Y|Q), over laws p(q)p(x1|q)p(x2|q)p(y|x1,x2). Time-sharing Q convexifies the region. Independent conditional inputs are a hypothesis; feedback, cooperating encoders or input cost constraints change the region.

    A multiuser capacity region replaces a single-user scalar.

  4. Related to
    Relationship sources · 1

    Linked statements

    • For a binary symmetric channel with crossover probability 0<p<1/2 and fixed target block error 0<epsilon<1, the best rate has normal approximation C−sqrt(V/n) Q-inverse(epsilon)+O(log(n)/n). Here C=1−h2(p) and V=p(1−p)[log2((1−p)/p)]². Capacity is the leading asymptotic limit; finite length, decoder cost and the remainder matter for actual reliability.

    Capacity is the approximation's leading term.

Entry detailsPermanent link to this entry ↗
Record identity
info-channel-capacity
Revision
university-reference.1
Structured reference record
University reference