Cosmology · Consistency, tensions and claims of new physics

Calibration and systematic-error checks

Use a supplied nonlinear instrument response to distinguish channel agreement, endpoint calibration and independent checks across an operating range.

Preparation: ratios, powers of ten and ordinary arithmetic. Physical quantities and notation are introduced as they are needed.

Back to the chapter

After reading, you can

Distinguish agreement between channels from agreement with an external reference.
Calculate forward residuals and bias after an affine inverse calibration.
Explain why refitting a check removes its held-out role.

Two channels can share the same error

Comparing instruments is useful, but two agreeing channels may share a calibration failure. An external reference or independently characterized check addresses a different question from their mutual agreement.

Calibration establishes a response under stated conditions and over a stated range. Validation checks whether the fitted response describes additional measurements that were not used to determine it.

A check near the middle of a fitted range can reveal curvature missed by endpoint data. A check beyond that range instead tests an extrapolation, even when the underlying physical response remains defined there.

Use a supplied noiseless nonlinear response below. Its residuals are deterministic teaching values; no measurement-error distribution, confidence interval or actual instrument performance is asserted.

Supply a monotonic response and a restricted affine fit

Choose a positive unit U, let x = X/U and y = Y/U, and define the physical operating domain 0≤x≤4. Supply identical channels y_A = y_B = (21/20)x+1/5+(3/100)x(x−2).

Fit an affine response only to the endpoint references x = 0 and x = 2. Since the quadratic term vanishes at both endpoints, the fitted function is y_aff = (21/20)x+1/5.

Reserve x = 1 as a held-out check inside the fitted interval. Evaluate x = 4 separately as extrapolation beyond the calibration interval [0,2] but within the supplied physical operating domain [0,4].

The derivative of the true response is 99/100+(3/50)x, positive throughout [0,4]. The example therefore has no hidden multiple-valued physical response; the failure concerns the chosen affine approximation.

Agreement does not reveal a shared curved residual

For either channel, the external forward residual is Y_true−Y_aff = U(3/100)x(x−2). The channel difference Y_A−Y_B is identically zero at every x.

At the held-out x = 1, the true output is 1.22U while the affine prediction is 1.25U. Their residual is −0.03U even though the two channels agree exactly.

At x = 4, the physical response is 4.64U and the extrapolated affine response is 4.40U. The residual grows to +0.24U.

The zero endpoint residuals follow from how the fit was chosen. They are not independent evidence that the response is affine between the endpoints or outside them.

Forward responses in units U; fitted endpoints, held-out check and extrapolation have different roles
x	Role	Y_true/U	Y_aff/U	(Y_true−Y_aff)/U
0	Calibration endpoint	1/5	1/5	0
1	Held-out check	61/50	5/4	−3/100
2	Calibration endpoint	23/10	23/10	0
4	Extrapolation check	116/25	22/5	6/25

Forward residual and inferred-input bias differ

An affine-calibrated input estimate uses x̂ = (y_true−1/5)/(21/20). Substituting the true response gives x̂−x = x(x−2)/35.

At x = 1, this fitted inverse gives x̂ = 34/35 rather than 1, a dimensional input bias X̂−X = −U/35. At x = 4, it gives x̂ = 148/35, with bias 8U/35.

The forward residual is measured in output units U; the inferred-input bias follows after dividing by the fitted slope 21/20. Equal numerical values are not expected even though the chosen input and output units share the same symbol.

This x̂ is the inverse of the fitted affine function applied to the nonlinear output. It is not the exact inverse of the supplied nonlinear physical response.

Bias introduced by applying the fitted affine inverse
x	x̂	x̂−x	(X̂−X)/U
0	0	0	0
1	34/35	−1/35	−1/35
2	2	0	0
4	148/35	8/35	8/35

Choose arbitrary positive readout unit U and dimensionless true input x = X/U over 0 to 4. Both noiseless channels share true y = Y/U = 21x/20+1/5+3x(x−2)/100. Fit only the endpoints x = 0 and 2 with y_aff = 21x/20+1/5. Horizontal linear x runs from 0 to 4; vertical linear residual divided by U runs from −0.05 to 0.27. The solid blue external-residual curve is (Y_true−Y_aff)/U = 3x(x−2)/100. The dashed orange mutual-channel difference (Y_A−Y_B)/U is identically zero over the entire domain. Hollow circles at x = 0 and 2 are fitted endpoints, a hollow square at x = 1 is the held-out check, and a hollow diamond at x = 4 is the extrapolation check. External residuals at x = 0,1,2,4 are 0,−3/100,0,6/25; the channel difference is zero at all four. The background beyond x = 2 is shaded and diagonally marked to label a range outside the endpoint fit; it does not mean the true response is undefined. True y at these inputs is 1/5,61/50,23/10,116/25; the affine prediction is 1/5,5/4,23/10,22/5. The true derivative is 99/100+3x/50, positive throughout. The native table applies the fitted affine inverse x̂ = (y_true−1/5)/(21/20), giving x̂ = 0,34/35,2,148/35 and x̂−x = 0,−1/35,0,8/35. This dimensionless inferred-input bias equals (X̂−X)/U and differs from the forward residual by the fitted slope. The fitted affine inverse is not the exact inverse of the true nonlinear response. Both channels can agree while sharing the same error relative to an external standard. Endpoint fitting is deliberately inadequate here, and a held-out point used in a repair becomes training information requiring a new check. No actual instrument, empirical bias, noise law, normality, uncertainty, confidence interval or probability is supplied. Patterned versus solid lines, marker roles, hatching, exact values and table provide noncolor access. Original Ark teaching model.

A successful repair requires a new check

The held-out residual identifies a specific failure of the endpoint-only affine calibration. Using x = 1 to fit a quadratic can repair this supplied response, but then x = 1 becomes training information.

A subsequent validation claim needs a different independent check. Reusing the same point as if it were still held out would conceal the change in how the calibration was chosen.

Practical checks should span relevant operating conditions, include suitable reference standards and examine residual structure. More channels can help, but common response errors require independent information to expose them.

The two-endpoint design was chosen to demonstrate a failure mode. It is not an adequate general calibration protocol, and the deterministic residuals here supply no probabilistic rule for judging real measurements.

Worked reasoning

Follow the held-out output through the affine inverse

What does the affine calibration infer from the true output at x = 1?

The supplied true response gives y = 21/20+1/5−3/100 = 61/50.
Apply the fitted inverse: x̂ = (61/50−1/5)/(21/20) = 34/35.
Subtract the reference x = 1: x̂−x = −1/35, while both channels still agree.

The inferred input is 34U/35, biased by −U/35 relative to X = U.

Try the reasoning

Find the fitted-inverse bias at x = 4.
Show the answer and reasoning
X̂−X = 8U/35.
The dimensionless bias is x(x−2)/35 = 8/35; equivalently divide the forward residual 6U/25 by slope 21/20.
Does Y_A−Y_B = 0 establish accurate external calibration?
Show the answer and reasoning
No.
Both supplied channels share the same nonlinear response and therefore the same external residual.
After using x = 1 to refit the response, is that point still an independent validation check?
Show the answer and reasoning
No.
It has become training information; validation needs another independently selected check.

What to carry forward

Channel agreement can coexist with a common calibration error.
A forward residual and an inferred-input bias answer different questions.
A check used for refitting must be replaced before claiming independent validation.

References

Sources supporting the explanations, physical distinctions and examples in this lesson.