Fair comparisons require shared definitions and assumptions

Are the two numbers answers to the same question?

Two printed numbers can differ because they describe different quantities, units, epochs or sample definitions. A density at an earlier epoch and a present density are not competing estimates of the same target. Nor are an average over one selected region and an average over another automatically interchangeable.

Even when the target is the same, the inferences may use different observations or physical assumptions. A parameter derived from a cosmological fit can depend on fixed components, allowed curvature, neutrino treatment, foreground choices or prior information. The result’s label should carry those conditions.

The historical Planck 2018 parameter analysis names its data combinations and model assumptions explicitly. It is an example of how to report a conditional inference, rather than a source of new best-fit values in this lesson. Directly sampled fit parameters and quantities derived from them are also different reporting categories.

A fair comparison begins by aligning the definitions. It then asks what information or assumption differs, rather than treating every central-value difference as a contradiction. Sometimes that difference reveals a useful model test; sometimes the apparent conflict disappears once the target is stated correctly.

The central difference can stay fixed while its uncertainty changes

Choose two estimates A = 10 and B = 12 in the same hypothetical unit. Each has standard uncertainty 1. The central difference is Δ = B − A = 2 in that unit. No real cosmological measurement is represented by these numbers.

Errors can move together or in opposite directions. Correlation, written ρ, summarizes that linear relationship after scaling by the individual standard deviations. Positive correlation means a tendency to move together, negative correlation a tendency to move oppositely, and zero correlation means no covariance in this calculation.

For the difference, u_Δ² = u_B² + u_A² − 2ρu_Au_B, where u_A and u_B are the standard uncertainties. Here u_Δ = √(2 − 2ρ). This variance relation is exact for the linear difference when the stated variances and covariance exist; it does not require a normal distribution.

At ρ = 0, the standard uncertainty is √2 ≈ 1.414. At ρ = +0.75 it is √0.5 ≈ 0.707. At ρ = −0.75 it is √3.5 ≈ 1.871. The estimates and their individual uncertainties have not changed, but the uncertainty of their difference has.

Shared shifts can cancel in a difference

If both estimates move upward by the same calibration shift, subtracting them cancels that shared part. This explains why positive correlation can reduce uncertainty in their difference. The same shared shift would not cancel in their sum.

If errors tend to move oppositely, the difference changes more strongly. That is why negative correlation makes the difference’s standard uncertainty larger in the chosen example. The sign is tied to the operation being performed, not to whether correlation is generally good or bad.

The correlation is an input to be established from the measurement and analysis. It cannot be selected afterward merely to make two results look more or less consistent. Shared data, calibration and modelling can create covariance, but their actual contribution must be assessed.

Zero covariance also should not be carelessly expanded into every kind of statistical independence. The present linear calculation needs the specified covariance structure. Broader independence claims require more than one reported correlation coefficient.

A standardized difference is not yet a discovery claim

One might divide the central difference 2 by its standard uncertainty to create a standardized comparison. Turning that ratio into a probability or significance requires a distribution and a defined statistical procedure. The variance table alone has supplied neither a cosmological null hypothesis nor a discovery test.

The individual error bars may represent different interval conventions, or uncertainty distributions may be asymmetric. Shared observations may also make two analyses less independent than they appear. Simply checking whether two bars overlap cannot replace the appropriate comparison.

The measurement processes themselves can be affected by selection, masks, foreground residuals and calibration. If a physical model is changed between analyses, the conditional target may change too. A complete assessment separates those causes instead of compressing them into one printed tension number.

This does not require dismissing a persistent discrepancy. It makes the question sharper: under which common definitions and assumptions does the discrepancy survive, and which additional observation could distinguish a calibration issue, a selection effect or a limitation of the physical model?

Build a comparison that another reader can assess

State the common target and units, then the relevant epoch and sample. Name the data combinations, calibration choices and masks. Identify the physical model, fitted and fixed quantities, and any prior information. Finally state the interval convention and the covariance information used for the difference or combination.

These details are not decorative qualifications: each can change what two numbers mean together. The toy example proves one such change exactly. A central difference of 2 has three different standard uncertainties when the stated correlation changes.

The beginner sequence now has a consistent reading method. A map needs geometry and a legend; a plot needs quantities and interval meanings; a cleaned signal needs a separation model; a catalogue needs selection; a power estimate needs a finite-sky account. A numerical comparison needs all the relevant definitions and relationships carried forward.

Worked reasoning

Try the reasoning

1. May the correlation be chosen just to make two results seem compatible?

   Show the answer and reasoning

   No.

   It must follow from the measurement and analysis, including any shared information.

2. If A and B both receive the same additive calibration shift, does that shared shift remain in B − A?

   Show the answer and reasoning

   No. The identical additive shift cancels.

   This illustrates why a shared positively correlated contribution can reduce uncertainty of a difference.

3. Does a difference divided by its standard uncertainty automatically establish a discovery probability?

   Show the answer and reasoning

   No.

   A distribution, target hypothesis and valid statistical comparison are still required.

What to carry forward

• Two results must describe a common target before their values can be fairly compared.
• The uncertainty of a difference depends on covariance as well as the individual bars.
• A conditional numerical discrepancy needs a defined model and statistical procedure before it becomes a broader scientific claim.