Inference depends on models and assumptions

An observation model joins several kinds of assumptions

An inferred temperature, mass, distance or source population is usually not a number directly recorded by a detector. A physical proposal predicts radiation or another messenger; propagation and an instrument transform it; noise, background, selection and processing determine the record available for analysis. Inference works back through that chain under stated assumptions.

A generative observation model specifies how data could arise for chosen parameter values. A latent quantity is part of that model but is not directly observed. A nuisance parameter, such as an uncertain background level, matters for the data even when it is not the scientific target. Calling it a nuisance does not make it known.

In the earlier synthetic instrument, E[Y] = Rx + b relates incoming photon-number means x to recorded channel counts Y. It is a statement about means until a sampling law is also supplied. Independent Poisson channel counts, a Gaussian approximation to a processed rate and an empirical noise distribution are different statistical assumptions.

Chandra’s forward-fitting reference illustrates a disciplined order: predict the detector-channel spectrum through the response, then compare it to the recorded spectrum under a specified statistic. Transforming an observed spectrum into a smooth source-looking curve does not remove the need for that response or create direct observations of every physical source parameter.

A likelihood is not a probability that a model is true

For a fixed observed dataset d, the likelihood L(θ; d) is the sampling mass or density p(d | θ), regarded as a function of θ. If observations are conditionally independent under the model, their factors multiply. Otherwise their joint law is needed. A numerical optimizer finds a preferred value for its stated objective, not the truth of the entire physical explanation.

A Bayesian posterior also requires a prior probability law for the parameters and a finite, positive normalization. It is proportional to likelihood times prior. A prior, a positivity constraint, an assumed source population or a fixed background can change the resulting inference; these choices belong in the scientific report.

Likelihood and posterior comparisons must refer to the same recorded information and declared coordinate. A density at one measured value and the probability of a censored event are not interchangeable, as the preceding lesson showed. Selection and measurement uncertainty can also alter the appropriate data law.

Terms constant with respect to all fitted parameters can be omitted when locating a likelihood maximum or comparing its values within that same setup. A term that changes with a fitted noise width is not such a constant. The following original example makes the consequence visible.

Keep the normalization when the width changes

Use two independent residuals r1 = −1 and r2 = +1 in a fixed dimensionless coordinate. The model mean is known to be zero. Assume Gaussian residuals with a common unknown standard deviation σ > 0. Their density product is L(σ) = (2πσ²)⁻¹ exp(−1/σ²), because the sum of squared residuals is two.

Each Gaussian density contributes both a residual exponential and a factor 1/(√(2π)σ). Widening the distribution reduces the standardized residual penalty but spreads its density over a wider range. The full likelihood accounts for both changes.

Compare the specified widths σ = 1 and σ = 2. The full ratio is L(2)/L(1) = (1/4)exp(3/4) ≈ 0.529250004. If the width-dependent prefactor is discarded, the residual-only ratio becomes exp(3/4) ≈ 2.117000017 and reverses the preference.

Equivalently, the natural-log likelihood relative to σ = 1 is ln[L(σ)/L(1)] = −2 ln σ − 1/σ² + 1. Here σ is dimensionless. For residuals with a physical unit, define an appropriate fixed scale before taking a logarithm of a width ratio. The log of the likelihood ratio remains dimensionless.

A fitted width remains conditional on the chosen model

For this toy, differentiating the log likelihood gives −2/σ + 2/σ³. It vanishes at σ = 1; the likelihood decreases toward zero as σ tends to either zero or infinity, so this is the maximum. That calculation solves the stipulated optimization problem. Two residuals do not establish a reliable physical noise model or a coverage guarantee.

The comparison between two specified widths is a likelihood ratio. It is not an integrated Bayes factor for models with unknown parameters and priors. Nor does its preferred width prove whether a discrepancy comes from detector noise, unmodelled source variation or an incorrect physical mean.

Hogg, Bovy and Lang explain how a data-generating model gives meaning to a fitting objective and discuss intrinsic scatter separately from known measurement uncertainties. Their treatment also exposes the width penalty when an intrinsic variance is fitted. The appropriate expression must still include every normalization that depends on the fitted parameters.

Gaussian residuals are an assumption here, not a universal law for raw photon counts. Correlation, outliers, censoring, uncertainty in predictors, an estimated background or selection can require another joint model. Changing one of these conditions means reassessing the inference, not merely replacing a label on the same fitted number.

The observable predictions determine what the data can identify

If distinct parameter combinations give exactly the same observable distribution, those observations cannot distinguish them. This is lack of identifiability. Finite noisy data can also give broad uncertainty even in an identifiable model. Computational convergence is a third question: a well-converged calculation may describe a weakly identified or scientifically inadequate model.

The poorly conditioned response from the calibration lesson illustrates a related practical difficulty. Two nearly similar response columns retain a mathematical difference between source bands but make the recovered balance highly sensitive to small recorded changes. An algorithm cannot create precision that the response did not preserve.

A prior or constraint can select among weakly constrained solutions. Report its role and examine whether the scientific conclusion changes under reasonable alternatives. That additional assumption can make a result useful, but it should not be described as observational identification supplied by the detector alone.

Finally, compare the model’s predictions with data features relevant to the scientific question. Small residuals on fitted points are useful but do not prove a unique physical mechanism, a valid extrapolation or a complete uncertainty budget. Later lessons develop validation and simulation directly. The next lesson preserves the inputs, choices and outputs needed to make this particular inference inspectable and reproducible.

Worked examples

Try the reasoning

1. Does specifying E[Y] = Rx + b fully specify a likelihood for Y?

   Show the answer and reasoning

   No.

   A mean relation needs a sampling distribution and any dependence assumptions before it defines the data law.

2. May a term depending on a fitted σ be discarded as a constant?

   Show the answer and reasoning

   No.

   Its change affects the relative likelihood or optimizer.

3. What additional probability input is needed for a Bayesian posterior beyond the likelihood?

   Show the answer and reasoning

   A stated prior, with a valid finite positive normalization.

   The posterior is conditional on both the sampling model and the prior.

4. Can more exact optimization distinguish parameter combinations that predict the same observable law?

   Show the answer and reasoning

   No.

   That is an identification limitation of the observations and model, not merely a computational error.

What to carry forward

• Physical inference depends on the complete observation and sampling model.
• Retain normalization terms that vary with fitted parameters.
• Separate an optimization result from identification, model adequacy and posterior or coverage claims.