Cosmology · Consistency, tensions and claims of new physics

Standards for evaluating claims of new physics

Compare declared response hypotheses on training and held-out quantities while separating a numerical fit from a coherent physical explanation.

Preparation: ratios, powers of ten and ordinary arithmetic. Physical quantities and notation are introduced as they are needed.

Back to the chapter

After reading, you can

Calculate training and held-out residuals for three supplied response hypotheses.
Explain why equal training fits need not identify the physical origin of a discrepancy.
State additional requirements for a viable physical extension.

A changed fit is the beginning of a physical assessment

A discrepancy can motivate a new physical hypothesis, a revised calibration model or a check of the original assumptions. A numerical improvement alone does not decide which explanation is responsible.

A useful extension makes linked predictions. Once its parameters are constrained by one set of information, other quantities can test those links without being used to repair the fit.

Physical proposals also need specified dynamics, conservation laws where applicable and a controlled relation to established successful regimes. A response formula by itself is a smaller claim than a complete theory.

The following deterministic example compares declared response hypotheses. It supplies training and held-out references, but no observational uncertainty law, likelihood, model prior or empirical claim about new physics.

Keep the hypotheses and held-out quantity fixed

Let all outputs Y_i have arbitrary positive unit U. Supply dimensionless training references y₁/U = 1 and y₂/U = 6/5, and predeclare a held-out reference y₃/U = 2.

Use dimensionless parameters a∈[1/2,3/2] and d,b∈[−1/2,1/2]. The baseline predicts (Y₁,Y₂,Y₃)/U = (a,a,2a).

The linked extension predicts (a,a+d,2a−d). The calibration alternative predicts (a,a+b,2a). They share a training-response form but link the extra parameter to the third output differently.

Minimize only the unweighted training squared-error loss L_train = (Y₁−y₁)²+(Y₂−y₂)², in U². The held-out reference does not enter this fit. This declared loss is not a likelihood or automatically a chi-squared statistic.

Two exact training fits can imply different explanations

For the baseline, L_train/U² = (a−1)²+(a−6/5)². Differentiating gives the allowed optimum a = 11/10, with training residuals prediction minus reference(+1/10,−1/10).

The baseline training loss is 1/50 U². Both extended hypotheses fit the training references exactly with a = 1 and d = b = 1/5, also within the declared parameter bounds.

The identical zero training losses do not distinguish a changed physical response from a calibration explanation. The distinction appears in how each hypothesis connects the third output to the fitted parameters.

Keep those fitted parameters fixed before evaluating the held-out quantity. Allowing another fit after looking at that reference would change the procedure and remove its independent-check role.

Training optima and predictions; every parameter is dimensionless and within its stated bounds
Hypothesis	Fitted parameters	Y₁/U	Y₂/U	Y₃/U	L_train/U²
Baseline	a = 11/10	11/10	11/10	11/5	1/50
Linked extension	a = 1,d = 1/5	1	6/5	9/5	0
Calibration alternative	a = 1,b = 1/5	1	6/5	2	0

Evaluate a prediction that was not fitted

Against held-out reference y₃/U = 2, the baseline predicts 11/5 and has residual +1/5. The linked extension predicts 9/5 and has residual −1/5.

Their held-out squared residuals are both 1/25 U². Thus the linked extension removes the training residuals while giving no improvement in this particular declared held-out squared error.

The calibration alternative predicts 2 and has zero held-out residual. That agreement is a property of the supplied deterministic responses and references; it is not a probabilistic proof of calibration as the physical cause.

Signs matter even when squares agree. Retaining the residuals as well as the loss makes the different linked predictions visible and supports later tests of the proposed response.

Held-out evaluation with training optima fixed; residual means prediction minus reference
Hypothesis	(Y₃−y₃)/U	Held-out squared residual/U²
Baseline	1/5	1/25
Linked extension	−1/5	1/25
Calibration alternative	0	0

This supplied deterministic response comparison fixes references (y₁,y₂,y₃)/U = (1,6/5,2) for arbitrary positive unit U. Only outputs 1 and 2 are training; output 3 is predeclared held out and shaded. Dimensionless bounded parameters specify three algebraic hypotheses: baseline predictions (a,a,2a), linked extension (a,a+d,2a−d), and calibration alternative (a,a+b,2a). Training minimizes the unweighted sum of squared prediction-minus-reference residuals. It gives baseline a = 11/10, linked a = 1,d = 1/5, and calibration a = 1,b = 1/5. The horizontal linear plotting domain is output index 0.8 to 3.2 with only the three discrete responses at 1,2,3 supplied; connecting lines are guides. The vertical linear axis is dimensionless Y/U from 0.8 to 2.4, deliberately nonzero based and explicitly labeled. Dotted light hollow circles mark supplied reference 1,1.2,2. Solid blue small circles show baseline 1.1,1.1,2.2. Dashed orange squares show linked extension 1,1.2,1.8. Dotted green diamonds show calibration alternative 1,1.2,2. Markers at common positions are nested, never displaced to suggest different values. Output 3 is tested after training fits are fixed. Its prediction-minus-reference residual is +1/5 for baseline, −1/5 for linked and 0 for calibration, giving held-out squared residuals 1/25,1/25,0 in U². Training losses are respectively 1/50,0,0 in U²; the native table prints these normalized losses. Equal exact training fit does not give the same linked prediction, and here the linked extension does not improve the held-out squared loss over baseline. Calibration reproducing this supplied reference does not uniquely establish a cause. These are not complete cosmological theories, observations, probabilistic fits or physical new-physics viability claims. The deterministic loss is not chi-square without an error model, likelihood, evidence, posterior odds, Bayes factor, calibrated significance or automatic complexity penalty. Direct labels, different patterns and marker shapes plus exact loss table give noncolor access. Original Ark teaching model.

A viable extension must connect more than fitted numbers

For an actual new-physics claim, specify what dynamics produce the response and which conservation, stability and regime-of-validity conditions it satisfies. Explain how successful predictions in established limits are recovered.

Trace the same proposal through relevant expansion, growth, lensing or early-Universe observables. Include calibration, selection, covariance and shared information when comparing it with data.

If posterior model odds or evidence ratios are claimed, supply normalized likelihoods, parameter priors and model probabilities as needed. Parameter count alone does not determine a Bayes factor or a universal penalty.

The example shows how held-out predictions can distinguish response links that training data leave ambiguous. Its fixed deterministic losses provide neither a current empirical ranking of cosmological models nor an uncertainty-qualified forecast.

Worked reasoning

Fit the baseline and test its held-out prediction

Find the baseline optimum, training loss and held-out residual.

Set the derivative of (a−1)²+(a−6/5)² to zero: 4a−22/5 = 0, giving a = 11/10.
The two training residuals are±1/10, so L_train/U² = 2/100 = 1/50.
Predict Y₃/U = 2a = 11/5 and subtract held-out reference 2 to obtain residual +1/5.

a = 11/10, training loss U²/50, and held-out residual +U/5.

Try the reasoning

What held-out prediction follows from the exact linked-extension training fit?
Show the answer and reasoning
Y₃ = 9U/5.
Training gives a = 1 and d = 1/5, so 2a−d = 9/5.
Do the two zero training losses identify the physical origin of the training mismatch?
Show the answer and reasoning
No.
The linked extension and calibration alternative have identical training responses but different held-out links.
Is this unweighted deterministic training loss sufficient to calculate posterior model odds?
Show the answer and reasoning
No.
No probability law, normalized likelihood or prior/model probabilities have been supplied.

What to carry forward

A fit improvement must be assessed alongside linked independent predictions.
Equal training fits can leave physical and calibration explanations unresolved.
A new physical explanation needs coherent dynamics, valid limits and tested observational responses.

References

Sources supporting the explanations, physical distinctions and examples in this lesson.