Cosmology · Cosmological parameters and statistical inference

Model comparison and sensitivity to assumptions

Compare a fixed-probability model with three declared proper-prior alternatives using exact evidence and model odds.

Preparation: ratios, powers of ten and ordinary arithmetic. Physical quantities and notation are introduced as they are needed.

Back to the chapter

After reading, you can

Compute evidence as a prior average of likelihood.
Distinguish within-model priors from prior probabilities of models.
Explain why model preference remains conditional on the compared alternatives.

A comparison begins with explicit models

A model comparison asks how declared alternatives account for the supplied observations. A model includes its response, sampling law, parameters and any probability distributions used for those parameters.

A maximum likelihood evaluates a model at its best-fitting allowed parameter choice. Bayesian evidence instead averages the likelihood over a normalized parameter prior within that model.

Prior probabilities assigned to the models are another input. Posterior model odds equal the evidence ratio, or Bayes factor, multiplied by those prior model odds.

Cosmological comparisons can depend on observational nuisance assumptions and the allowed physical parameter space. The finite example below supplies no actual cosmological fit, significance or preference.

Keep the data event and each prior fixed

Supply the ordered event 1111 from four independent Bernoulli trials with success probability θ. Its likelihood is θ⁴. Every quantity in this construction is dimensionless.

Model M₀ fixes θ = 1/2, so its evidence is Z₀ = (1/2)⁴ = 1/16. It has no free θ to average over.

Model M₁ instead assigns θ a uniform proper prior on [0,b], with density 1/b. Compare separately declared bounds b = 1, 3/4 and 1/2.

These are predeclared sensitivity cases. Choosing a prior range after inspecting the data and then treating the result as prior-independent evidence would change the question.

The best allowed likelihood is different from its prior average

For M₁, integrate the likelihood against the prior: Z₁ = (1/b)∫₀ᵇθ⁴dθ = b⁴/5. Its maximum likelihood is instead L_max,1 = b⁴.

The Bayes factor is B₁₀ = Z₁/Z₀ = 16b⁴/5. At b = 1 it equals 3.2, while the ratio of maximum likelihoods is 16.

At b = 3/4, B₁₀ = 81/80 = 1.0125 even though M₁ reaches likelihood 0.31640625. At b = 1/2 the maximum likelihoods tie, but B₁₀ = 1/5.

The prior range is part of the model definition. The different evidences follow exact integration over those different declared predictive assumptions.

Specified four-success event; evidence averages a proper prior, while L_max selects one parameter value
Prior bound b	L_max,1	Evidence Z₁	Bayes factor B₁₀	P(M₁\|data), equal model priors	P(M₁\|data), prior P(M₁)=1/4
1	1	1/5	16/5	16/21 ≈ 0.761905	16/31 ≈ 0.516129
3/4	0.31640625	81/1280	81/80	81/161 ≈ 0.503106	27/107 ≈ 0.252336
1/2	1/16	1/80	1/5	1/6 ≈ 0.166667	1/16 = 0.0625

A Bayes factor alone is not a posterior model probability

Assume the complete compared list contains only M₀ and the selected M₁ case. With equal prior model probabilities, posterior P(M₁|data) = B₁₀/(1+B₁₀).

For b = 1 this gives 16/21. If the prior model probabilities instead assign 1/4 to M₁ and 3/4 to M₀, the prior odds are 1/3 and the posterior probability becomes B₁₀/(3+B₁₀) = 16/31.

The observations and within-model prior have not changed between these two calculations. Their different posterior probabilities arise from different model-prior odds.

All these probabilities are conditional on the supplied two-model list. A comparison does not eliminate an omitted alternative or establish that either available model is a complete physical account.

Parameter counting is not an evidence calculation

An extra parameter does not automatically incur a fixed evidence penalty. If a new parameter has a normalized independent prior and the likelihood does not depend on it, its prior integrates to one and the evidence stays unchanged.

Conversely, changing an influential prior range or shape can change the model’s predictive average substantially. Proper normalization and the meaning of every prior are necessary for an evidence calculation.

Optional posterior prediction gives the probability of a next success under M₁ as 5b/6 after the four successes. This is conditional prediction within the stipulated model, rather than a real forecast.

Report the alternatives, parameter priors, model priors and data definition together. A maximized fit, an evidence adjective or a model probability does not certify a cosmological mechanism as true.

A supplied ordered sequence of four independent successes has likelihood θ⁴. M₀ fixes θ = 1/2, giving evidence 1/16. M₁ uses a proper uniform prior on [0,b], separately comparing predeclared bounds 1/2, 3/4 and 1. Markers identify those three cases; curves evaluate the stated family between them rather than fitting observations. Panel A uses horizontal b from 0.5 to 1 and vertical probability or likelihood from 0 to 1.05. Solid blue circles identify maximum b⁴; dashed orange squares identify evidence b⁴/5. The green dotted reference is M₀ evidence 1/16. Panel B has its own vertical Bayes-factor scale from 0 to 3.5: violet diamonds show B₁₀ = 16b⁴/5 with a dotted reference at 1. A Bayes factor is not a probability. Panel C uses a separate probability scale from 0 to 1 for posterior P(M₁) conditional on the supplied closed two-model list. Solid blue circles use equal model priors, B₁₀/(1+B₁₀); dashed orange squares use prior P(M₁) = 1/4, giving B₁₀/(3+B₁₀). The exact table gives, at b = 1/2, evidence 1/80, factor 1/5 and posterior probabilities 1/6 and 1/16; at b = 3/4, 81/1280, 81/80, 81/161 and 27/107; at b = 1, 1/5, 16/5, 16/21 and 16/31. Within-model priors and model-prior odds are different inputs. If the likelihood does not depend on an additional parameter and its prior is proper and independent, integrating that prior leaves the evidence unchanged. These conditional examples supply no actual cosmological model preference or significance. Original Ark teaching model.

Worked reasoning

Compare the two ratios at b = 1

Does the maximum likelihood ratio 16 equal the Bayes factor?

M₁ reaches likelihood 1 at θ = 1, whereas M₀ has likelihood 1/16.
Averaging θ⁴ over the uniform [0,1] prior gives evidence 1/5 for M₁.
Divide evidences: B₁₀ = (1/5)/(1/16) = 16/5 = 3.2.

No. The maximized ratio and the prior-averaged ratio are different calculations.

Try the reasoning

For B₁₀ = 16/5 and prior P(M₁)=1/4, what posterior P(M₁) follows?
Show the answer and reasoning
16/31.
With prior odds 1/3, the posterior probability is B₁₀/(3+B₁₀), conditional on this two-model list.
Must an unused parameter with a normalized independent prior lower evidence?
Show the answer and reasoning
No.
If the likelihood is independent of it, integrating its normalized prior multiplies the evidence by one.
Are the three b choices prior-independent accounts of the same predictive model?
Show the answer and reasoning
No.
Each declared prior range is part of the model and changes its prior-predictive average.

What to carry forward

Evidence is a likelihood averaged over a proper within-model prior.
Posterior model odds also require prior model odds.
A conditional comparison neither proves a model true nor excludes unexamined alternatives.

References

Sources supporting the explanations, physical distinctions and examples in this lesson.