Random uncertainty and systematic effects

Error and uncertainty answer different questions

An error is a difference between a measured quantity value and a reference quantity value. The exact error relative to an unknown true value is generally unknown. Measurement uncertainty describes the dispersion attributed to the measurand using the available information and stated model. It is not a measured mistake that can simply be subtracted.

A correction may address a recognized systematic effect while still having uncertainty. For example, subtracting an estimated detector offset changes the reported value, but imperfect knowledge of that offset remains relevant to the result. Ignoring it would confuse applying a correction with knowing the correction exactly.

First identify the quantity, interval, band, extraction and processing conditions. A spread among repeated pixel readings, uncertainty in their mean, uncertainty in a calibrated source rate and variation between astronomical sources are different quantities. Their numbers cannot be exchanged merely because each is called scatter.

A standard uncertainty is expressed as a standard deviation, in the same unit as the associated quantity. Variance has the squared unit. Neither a standard deviation nor a plus/minus sign, by itself, gives a universal probability that an unknown value lies in a reported interval.

Type A does not mean random, and Type B does not mean systematic

The historical NIST Technical Note 1297 calls an uncertainty evaluation Type A when it uses statistical analysis of observations and Type B when it uses other information. Type B information can include calibration documentation, a physical model or a justified assigned distribution. These terms describe how a numerical uncertainty was evaluated.

Random and systematic instead describe the roles of effects in the current measurement process. Repeated independent arrivals can produce random count variation. A common uncertain response correction can affect all measurements made under that calibration. Changing the process or the way a quantity enters its model can change the role of an effect.

A statistically estimated calibration correction can therefore address a systematic effect. Conversely, an externally specified model of an influence that varies during the present measurement can contribute uncertainty arising from a random effect. NIST’s Appendix D2 gives separate columns for evaluation method and effect for precisely this reason.

Neither label establishes independence, Gaussian behaviour or completeness of a budget. A useful uncertainty budget lists the inputs, corrections, standard uncertainties, sensitivities and correlations that matter for the actual result, and records how each was evaluated. It must also avoid counting the same contribution twice.

Processing changes the covariance structure

Return to the preceding synthetic two-band instrument. Stipulate independent Poisson recorded counts Y1 and Y2 with means 79 and 51. Their known background means are b = (5, 7), and the fixed response inverse is C = [[1.5, −0.25], [−0.5, 1.75]]. Define the linear estimate xhat = C(Y − b). The response and background means are treated as known exactly for this example.

Subtracting a known background mean does not subtract the random background arrivals that occurred in a realization. The variances remain those of the total recorded counts, 79 and 51. Subtracting five and seven from the variances would remove fluctuations that are still present.

For a fixed linear transformation, covariance transforms exactly as Vx = CVY Cᵀ, where VY = diag(79, 51) and Cᵀ denotes transpose. This follows by applying C to the deviations from the mean and taking the expected outer product. It needs the specified second moments but does not require a Gaussian distribution.

The first estimate uses both counts, as does the second. Their covariance is (1.5)(−0.5)(79) + (−0.25)(1.75)(51) = −81.5625 in squared incoming-count units. An upward fluctuation in a detector channel can increase one recovered band estimate while decreasing the other. This is dependence introduced by processing, not evidence that the source’s two physical bands fluctuate oppositely.

The covariance matters even if only a sum is reported

The resulting covariance matrix is [[180.9375, −81.5625], [−81.5625, 175.9375]]. The standard uncertainties of the individual estimates are approximately 13.4513 and 13.2641 incoming-count units. Their correlation coefficient, covariance divided by the product of standard deviations, is approximately −0.457138.

Suppose the desired quantity is the sum S = xhat1 + xhat2. Then Var(S) = Var(xhat1) + Var(xhat2) + 2Cov(xhat1, xhat2) = 193.75, giving u(S) ≈ 13.9194 incoming-count units. Dropping the covariance would give 356.875 for the variance, which describes a different and incorrect uncertainty calculation for this model.

There is a direct check. Summing the rows of C gives S = 1.0(Y1 − 5) + 1.5(Y2 − 7). Independence of the original counts therefore gives Var(S) = 1² × 79 + 1.5² × 51 = 193.75. Both routes agree when the transformation is retained.

Negative covariance does not always make every derived uncertainty smaller. For a difference, the covariance term has the opposite sign. The relevant combination and its sensitivities determine the effect. Reporting only two marginal error bars can hide information needed by a later analyst.

The model states what this budget includes

The example isolates counting variation under a fixed linear response. An estimated background, uncertain response coefficients, flat-field corrections, temporal drift or correlations between channels require additional inputs or a fuller joint model. The supplied covariance does not include those effects merely because calibration appeared earlier in the pipeline.

For a nonlinear measurement model, first-order uncertainty propagation uses derivatives and input covariances evaluated near the estimates. NIST Appendix A states its approximation and includes covariance terms. Near a strong nonlinearity, boundary or poorly identified direction, an appropriate distributional or simulation treatment may be needed instead of extending that approximation without checking.

Repeating observations can reduce a suitable independent counting contribution to an average. It does not automatically resolve an uncertain response common to all repetitions, an incorrect background model or omitted source selection. More repetitions of the same incomplete process need not reveal what it never measures.

Report which components are included, their evaluation and correlation assumptions, and whether an uncertainty is standard or expanded. An expanded uncertainty also needs its coverage factor and the basis for any coverage claim. The next lesson asks how inclusion in a catalogue changes the data distribution before inference even begins.

Worked examples

Try the reasoning

1. Does subtracting an estimated calibration offset eliminate uncertainty in that offset?

   Show the answer and reasoning

   No.

   The corrected value still depends on imperfect knowledge of the correction.

2. Can a correction addressing a systematic effect have a Type A uncertainty evaluation?

   Show the answer and reasoning

   Yes.

   Type A labels statistical evaluation, while systematic describes the effect’s role in the stated process.

3. If a Poisson count has total mean 79 and a known background mean 5 is subtracted, does its variance become 74?

   Show the answer and reasoning

   No; it remains 79 in the supplied model.

   Subtracting a fixed mean does not remove realized counting fluctuations.

4. What unit belongs to covariance when both quantities are incoming photon-number estimates?

   Show the answer and reasoning

   Squared incoming-count units.

   Covariance is an expected product of deviations; each deviation contributes its quantity unit.

What to carry forward

• Uncertainty and error differ; applying a correction does not make its value exact.
• Type A and Type B classify evaluation methods, independently of an effect’s random or systematic role.
• Propagate covariance with the result: processing can create dependence from independent detector counts.