Astronomy · Observations as evidence

Uncertainty and the strength of a claim

Read a measurement together with its uncertainty, distinguish repeatability from accuracy, and make claims that remain valid when assumptions are stated.

Continue in the approved chapter order. Necessary vocabulary, quantities and assumptions are explained in the lesson.

Back to the chapter

After reading, you can

Explain why uncertainty is part of a result rather than an admission of failure.
Combine independent standard uncertainties for a simple difference.
Distinguish evidence for a measured change from evidence for a particular physical cause.

A measurement is an estimate with a method

Suppose a brightness estimate is reported as “100”. The number alone leaves important questions unanswered. What quantity and units does it describe? Which wavelength band and time interval? How repeatable is it, and which calibration uncertainties remain? A well-described result answers those questions rather than offering extra digits as a substitute.

Uncertainty characterises how well the available information constrains a measured quantity. Error is a difference between an estimate and the quantity’s value, which is often not known exactly. An uncertainty is not a known error that we can simply subtract. It describes what remains uncertain after using a stated measurement model and relevant corrections.

This distinction makes a result more useful. You can compare it with another measurement, test whether a proposed change is resolvable and decide what better observation would help. “Uncertain” does not mean “anything is equally possible”; it means the strength and limits of the evidence must be expressed.

Scatter is one contribution, not the whole budget

Random variation can appear as scatter when a stable target is measured repeatedly under comparable conditions. Detector counting statistics, readout fluctuations and changing atmospheric effects can contribute. The observed spread may also contain real source variability, so it is not automatically a measurement-noise estimate.

Systematic effects can shift several results together. A reference brightness may be wrong, a clock may be offset or a background model may consistently subtract too much. Apply a correction when it is justified and retain uncertainty in that correction. A correction does not make its input exact.

A software uncertainty may include some counting and detector terms while omitting atmospheric, calibration or model effects. State what it includes. A small formal error bar cannot certify a complete physical model or a correct identification. NIST’s Type A and Type B labels classify methods of evaluating uncertainty; they are not interchangeable labels for random and systematic error.

Ask what a plus-or-minus value means

Standard deviation is a measure of spread: in repeated measurements, it describes the scale of deviations around their mean. A standard uncertainty is expressed as an estimated standard deviation. An expanded uncertainty multiplies a combined standard uncertainty by a stated coverage factor for a particular reporting purpose. A range based on physical bounds is another kind of statement. These should not be mixed without explanation.

In this lesson, u(x) denotes the standard uncertainty assigned to an estimate x, in the same units as x. When an example writes x = 100 count units with u(x) = 4 count units, the 4 is a standard uncertainty under its stated model. It is not a promise that the unknown quantity is trapped between 96 and 104.

A probability coverage statement needs assumptions about distributions, how uncertainties were estimated and the coverage procedure. Do not attach a universal 68% or 95% meaning to every unexplained ± sign. When reading a paper, look for the authors’ definition and what the uncertainty budget includes.

Compare a change with the uncertainty of the change

Let A and B be estimates of the same quantity from two observations. Define their difference D = B − A. If their standard uncertainties u(A) and u(B) are independent, the standard uncertainty of the difference is u(D) = √[u(A)² + u(B)²]. Independence matters: the formula does not simply add or subtract the two error-bar lengths.

For an original synthetic example, A = 100 count units with u(A) = 4; B = 88 count units with u(B) = 3. Their difference is −12 count units, and u(D) = √(16 + 9) = 5 count units. The magnitude of the difference is 12/5 = 2.4 times this standard uncertainty. That ratio describes this difference under the model; it is not a 2.4-to-1 probability and not the probability that a proposed cause is true.

If estimates share uncertain inputs, covariance matters. For the difference, u(D)² = u(A)² + u(B)² − 2 cov(A,B), where cov(A,B) describes how the estimated errors vary together and has squared units. A shared additive offset can cancel in a difference, while other common calibration effects may not. Use a measurement model instead of assuming that all common errors either vanish or add.

Averages help when their assumptions fit

For n independent, same-condition measurements of a stable quantity, an estimate of the standard uncertainty of their mean is s/√n, where s is their sample standard deviation. The square root expresses a conditional improvement in estimating the mean. It is not a law that every uncertainty shrinks whenever more numbers are collected.

If four independent measurements have sample standard deviation 8 count units, this contribution to the mean’s standard uncertainty is 8/√4 = 4 count units. Sixteen otherwise comparable independent measurements with the same sample spread would give 8/√16 = 2. These are synthetic calculations; a real sample’s spread is estimated and can change.

A shared calibration uncertainty remains a separate contribution. Correlated fluctuations reduce the independence of a sequence. A variable target can be smeared by averaging, and choosing only reassuring measurements can bias the result. Check the time sequence and quality record before treating its scatter as repeat measurements of one constant value.

Separate a measured pattern from its explanation

There are several levels of claim. “This method returned different estimates” describes an output. “The source’s received brightness changed” requires adequate uncertainty and instrumental checks. “A planet crossed the star” requires a physical model and tests against alternatives. Evidence at one level does not automatically establish the next.

Searching many times, objects or analysis choices also creates opportunities to find a striking fluctuation by chance. Keep the search procedure and follow-up tests visible. A threshold chosen after seeing the most attractive feature is weaker evidence than its numerical label may suggest. This beginner lesson does not prescribe a universal discovery threshold.

An honest conclusion can be modest and still useful: “The relative dimming is suggestive under this noise model; more observations and calibration checks are needed.” Or it may be strong: several independent methods survive relevant tests and agree. Scientific caution is the practice of matching a claim to its support, not refusing to draw conclusions.

Questions that strengthen an interpretation without replacing a full uncertainty analysis.
Before claiming…	Ask…
A detected change	Were comparable quantities measured, and is the uncertainty of the difference understood?
A precise value	Does the uncertainty include calibration and model limitations as well as scatter?
No change	What size and timescale of change could the observations have resolved?
A physical cause	Which alternatives were tested with independent evidence?

Worked example

Calculate a difference before judging it

Synthetic independent estimates are 52 count units with standard uncertainty 3, and 48 with standard uncertainty 4. What difference and uncertainty should be compared?

Use a declared order: second minus first gives D = 48 − 52 = −4 count units.
Independence gives u(D) = √(3² + 4²) = 5 count units.
The magnitude is 0.8 times this standard uncertainty; the result alone offers little basis for a confident change claim under this model.

D = −4 count units with standard uncertainty 5 count units. This is a statement about the comparison, not proof that the source is exactly constant.

Try the reasoning

For an invented stable sequence of nine independent same-condition measurements, s = 6 count units. What is s/√n?
Show the answer and reasoning
2 count units.
√9 = 3, so 6/3 = 2. This estimates the repeatability contribution to uncertainty of the mean under the stated assumptions.
An article reports “10 ± 2” without defining the units or the ± term. Can you assign a confidence level?
Show the answer and reasoning
No. The reporting meaning and assumptions are missing.
The term might be a standard uncertainty, an expanded uncertainty, a spread, or a bound. Those are different statements.
Two pipelines agree because both use the same incorrect calibration file. Does their agreement establish accuracy?
Show the answer and reasoning
No. They share a possible systematic error.
Independent checks must address shared inputs as well as compare final numbers.

What to carry forward

State the quantity, units, method and meaning of the uncertainty.
Scatter, shared calibration effects and model limitations can all matter.
Combine uncertainties under justified assumptions, and keep a measured change distinct from its physical explanation.

References

Teaching resources and research supporting this lesson’s explanations and examples.