Independent evidence and testable predictions

What does agreement actually add?

Two teams announce the same cosmic result. That sounds reassuring, but what did each team measure? If both analysed the same telescope map, their agreement can test analysis choices. It does not create a second independent sky observation. If they used different instruments, some instrumental errors differ, but both may still observe the same structures and share physical assumptions.

Independence is therefore a question about a particular source of error or information. We should ask which measurements, calibrations, objects, sky volumes and theoretical ingredients are shared. Agreement is strongest when a plausible failure in one route would not automatically reproduce the same result in the other.

This does not make repeated analyses useless. Independent software and modelling choices can reveal implementation mistakes or sensitivity to foreground treatment. The value becomes clearer when we name the level being tested instead of treating the number of papers as a count of independent experiments.

One hot history, different physical records

Consider the hot early-Universe account. A near-thermal microwave background is one record. Primordial light-element abundances are another. Thermal radiation and nuclear reactions are different physical processes, so the account must connect them through a consistent expansion and thermal history.

A useful comparison asks whether the ordinary-matter density inferred from background-radiation patterns is consistent with that inferred from suitable deuterium-abundance measurements and a Big Bang nucleosynthesis calculation. Nucleosynthesis means the production of atomic nuclei. Turning deuterium absorption lines into an early abundance requires a model of the gas; translating that abundance into a density adds nuclear reaction rates and thermal-history assumptions.

Cooke, Pettini and Steidel's 2018 study provides a dated example. It combined a near-pristine absorption-system analysis with six earlier systems and compared the inferred density with Planck results available then. The observations have different dominant measurement problems. The comparison nevertheless shares an early-Universe framework, so it is a consistency test with declared assumptions, not two completely assumption-free measurements.

The CMB itself supplies multiple tests. Its average spectrum constrains thermal behaviour; its small angular variations test perturbations, geometry and evolution. Those observables are related and should be modelled together. A coloured map's striking appearance is not the test: the test is a quantified pattern compared with a prediction and its uncertainty.

Count new information, not catalogue names

Different survey names can hide overlapping observations. The Planck parameter paper gives a concrete example: it excluded WiggleZ from its default baryon acoustic oscillation combination because the survey overlapped the BOSS-CMASS volume and the relevant correlations were not quantified. Baryon acoustic oscillations, or BAO, are a statistical clustering pattern used as a cosmic distance ruler after its physical scale is modelled.

The same paper also discusses supernova compilations that share objects. Reprocessing an existing supernova can improve its measurement or calibration, but that object does not become a second independently observed supernova merely because it appears in a new compilation.

When shared information is known, an analysis can include covariance, which describes how estimated quantities vary together. When it is unknown, a conservative choice may be to avoid combining the affected data as independent. Dropping a redundant sample can therefore produce a more defensible result than reporting a smaller error bar from a naive combination.

Why a shared error refuses to average away

Use a synthetic two-measurement example. Each measurement has independent random uncertainty with standard deviation 2 units and a common additive calibration error with standard deviation 3 units. A standard deviation measures the spread assigned by an uncertainty model. Assume zero-mean independent random components and a zero-mean common error, independent of those components.

The average's random variance is 2²/2 = 2 square units. The common calibration error enters the average unchanged, with variance 3² = 9 square units. Total variance is therefore 2 + 9 = 11 square units, giving standard deviation √11 ≈ 3.32 units. Variances add here because the named components are independent; the common term is counted once for the averaged result.

If we wrongly treated each measurement's total uncertainty √(2² + 3²) = √13 as independent, we would quote √(13/2) ≈ 2.55 units for their average. That would be too optimistic. More repeated measurements reduce the independent part while the shared calibration component remains. These numbers illustrate a statistical model; they are not the uncertainty budget of a real cosmological survey.

Predict a difference that matters

A testable explanation identifies an observable consequence and the conditions under which it follows. A discriminating test asks for a consequence that competing explanations predict differently. First specify the models, then identify the quantity, its expected difference and the measurement precision needed to resolve it.

Suppose two expansion models predict almost identical distances over a survey's redshift range but different growth rates of structure. Collecting more distance data only in that same narrow range may discriminate poorly. A structure-growth measurement may add information, provided its relation to gravity, galaxy selection and other effects is modelled. A different observable can be more useful than a larger version of the same dataset.

Predictions made before examining a new dataset are especially valuable because the model has less opportunity to accommodate accidental features. A fit to already-seen data still matters, but extra adjustable parameters can absorb noise. Keep track of which observations set the parameters and which supply an additional check.

One discrepancy rarely names its own cause. It can arise from chance, underestimated uncertainties, selection, calibration, analysis choices, or missing physical ingredients. A good follow-up changes something that would affect these possibilities differently. For example, a new instrument can test one class of calibration problem while the same analysis on a new sample can test sensitivity to population selection.

A fair comparison can be decisive without being absolute

Agreement across well-understood probes earns confidence in a framework. It does not prove every ingredient or make every alternative equally plausible. An alternative that cannot reproduce several established observations faces a more serious problem than one that makes similar successful predictions and differs in a narrow, testable regime.

Likewise, a result disfavoring a specified class of anisotropic models constrains that class under its stated assumptions. It does not eliminate every logically possible departure from isotropy. State the tested alternatives and uncertainty, and keep the next decisive observation connected to an actual predicted difference.

Scientific caution should sharpen a conclusion. 'These data constrain this parameter within these models' is useful because another researcher can reproduce it and propose a different test. An undefined statement that everything is uncertain leaves no such route forward.

Worked reasoning

Try the reasoning

1. Two telescopes observe the same CMB sky with independent detectors. What can their agreement test, and what remains shared?

   Show the answer and reasoning

   It can test many instrument-dependent effects; the underlying sky realization and some modelling assumptions remain shared.

   Instrument independence does not erase cosmic sample variance or common theoretical assumptions. The analysis should state which uncertainty components differ.

2. In the synthetic uncertainty model, average four measurements with independent random standard deviation 2 units and common calibration standard deviation 3 units. Find the total standard deviation.

   Show the answer and reasoning

   √(2²/4 + 3²) = √10 ≈ 3.16 units.

   Only the independent variance is divided by four. The common error is inherited by the average.

3. A rival model fits a distance dataset by adding parameters. What would strengthen the case that it captures new physics?

   Show the answer and reasoning

   A successful distinct prediction in additional data, together with physical consistency and systematic checks.

   A more flexible fit to the original dataset alone need not distinguish a physical improvement from accommodation of noise or an omitted measurement effect.

What to carry forward

• Ask which data, errors and assumptions are independent before combining results.
• Different physical probes test a framework more strongly when their likely failure modes differ.
• A shared systematic contribution does not vanish by averaging more correlated measurements.
• Design the next observation around a difference between explicit predictions.