Foundations

A randomized mechanism is (epsilon,delta)-differentially private if every neighboring dataset pair and every output event obey Pr[M(D) in S]≤exp(epsilon)Pr[M(D-prime) in S]+delta.

For a vector query with global L1 sensitivity Delta-1, adding independent Laplace noise of scale Delta-1/epsilon to each coordinate gives epsilon-DP under the chosen adjacency.

For order alpha>1, RDP bounds the Rényi divergence between mechanism outputs on every adjacent dataset pair by epsilon-alpha.

A norm-bounded adversarial evaluation asks whether an attacker can change a model's prediction using perturbations within a specified set around an input.

Gaussian smoothing defines a classifier by the most probable prediction of a base classifier under N(0,sigma²I) noise.

Scaled dot-product attention computes softmax(QK-transpose/sqrt(d-k))V, allowing token representations to combine context through learned query/key/value projections.

A DDPM adds Gaussian noise in a forward Markov chain and learns a parameterized reverse process to generate samples.

An autoregressive model factors a sequence law into conditional next-element distributions, while a diffusion model generates through a learned reverse noise process.

Samuel, Zhou and Zou's COLING 2025 study compares five contamination-detection approaches across four language models and eight benchmarks, including a controlled fine-tuning oracle setting.

PAC learning bounds population risk with probability at least 1−delta from iid samples.

For a finite hypothesis class H, iid observations and losses in [0,1], Hoeffding plus a union bound gives probability at most 2|H|exp(−2n epsilon²) that any empirical risk differs from its population risk by more than epsilon.

VC dimension is the largest size of a point set on which a binary hypothesis class realizes every labeling.

Bayesian inference combines likelihood and prior into a posterior conditional on observed data.

For predictors h, population risk is expected loss under a declared data distribution; empirical risk averages loss over a sample.

Regularized empirical minimization adds lambda times a complexity penalty to sample loss.

Stochastic gradient updates parameters using a sampled gradient or subgradient.

PCA finds an orthogonal low-rank subspace minimizing squared reconstruction error for centered observations.

Training fits parameters, validation selects models/tuning, and a final held-out test estimates performance of the selected procedure on its sampling population.

A Bayesian network uses a directed acyclic graph and local conditional distributions to factor a joint law.

Variational inference approximates a posterior by choosing q from a specified family to minimize KL(q||posterior).

MCMC constructs a Markov chain with a desired stationary posterior and estimates integrals from correlated draws; convergence, mixing and effective sample size require checks.

Observational conditioning p(Y|X=x) and intervention p(Y|do(X=x)) describe different operations.