Approximation, precision and measurement uncertainty

Identify what makes a value exact or estimated

A model can stipulate a length of exactly 8/3 m, even though its decimal expansion does not terminate. An exact count of twelve objects also does not depend on how many decimal digits you write. Exactness comes from the quantity’s definition or the model’s stated conditions, rather than from the length of a displayed number. Real measurements require their own information and limitations.

An approximation replaces a value with a nearby, more convenient expression, such as 8/3 m ≈ 2.67 m. An estimate may come from rough reasoning or from a measurement procedure that uses observations and a model. Name its origin. A reported measured value is an estimate of the defined quantity; writing many digits does not establish the unknown quantity’s value exactly.

Use equality for an exact relation and ≈ for a rounded value

The relation 1 cm = 10 mm is exact. So is the rewrite 12.345 cm = 123.45 mm when these numbers describe the same stated value. Rounding to two decimal places in centimetres instead gives 12.345 cm ≈ 12.35 cm. The approximation sign identifies the changed value. Keep the unit and the reporting instruction alongside the result.

For a known input x and its rounded report q, the signed difference q − x can be calculated exactly. For a measurement estimate, that is a difference from the input estimate, not from an unknown physical quantity. Measurement error is a measured quantity value minus a reference quantity value; a reference can itself have uncertainty, and the true quantity value is generally not known exactly. A display’s rounding difference does not determine that measurement error.

Specify whether you are counting places or significant figures

Decimal places count positions after the decimal point in a chosen unit. Significant figures count the digits retained in a conventional numerical report, beginning with the first nonzero digit. Leading zeros locate the decimal scale; they are not significant figures. Trailing zeros after a decimal point can deliberately retain significant figures. Scientific notation makes that choice visible: 4.50 × 10^2 declares three significant figures in its coefficient.

A bare whole-number report such as 450 does not by itself explain which trailing zeros were intended as significant. Give the context or use an unambiguous form. If 450 is an exact count, it remains an exact count rather than an uncertain measurement limited by those printed digits. Significant-figure conventions describe a reporting choice; the digits alone do not state a measurement uncertainty, its type or a probability.

Declare nearest rounding and what happens at an exact tie

This lesson rounds to the nearest allowed value, with exact ties away from zero. To p decimal places in a chosen unit, the reporting step is s = 10^−p of that unit. Choose the nearer multiple of s; if both are equally near, choose the one with greater magnitude. Thus exact 12.345 cm rounds to 12.35 cm at two decimal places, and −12.345 cm rounds to −12.35 cm. Their signed report-minus-input differences are +0.005 cm and −0.005 cm.

For a known exact input, nearest rounding alone gives |q − x| ≤ s/2. This bounds the numerical rounding difference relative to that input, not an unknown measurement error or total uncertainty. Other conventions, including ties to an even last retained digit, can give different tie results, so read the declared rule. Rounding a small positive value to zero does not assert that the original value was zero.

Significant-figure rounding uses the scale of the first nonzero digit. Under the same tie rule, exact 0.04995 cm rounded to three significant figures is approximately 0.0500 cm = 5.00 × 10^−2 cm. The carry changes the leading digit and retains three significant figures. If a rounded scientific-notation coefficient reaches ten, renormalize it while preserving the requested significant figures.

Keep exact or unrounded information through the calculation

For an exact model length 8/3 m, first converting to centimetres gives 800/3 cm. A final report to two decimal places is approximately 266.67 cm. If you first round 8/3 m to 2.67 m and then convert, you obtain 267 cm. The exact conversion factor does not restore the information removed by that earlier rounding.

Use exact fractions, symbolic constants or the available unrounded values through intermediate steps. Round once for the requested final report and unit. If only a rounded input is available, state that limitation rather than reconstructing digits that were never supplied. A calculator’s many output digits do not provide new information about a measured input or an imperfect model.

Define the measured quantity and the conditions

The measurand is the quantity intended to be measured. Define it clearly: for example, a rod’s length between named endpoints at a stated temperature and under specified support conditions. Describe the procedure, instrument, relevant reference and operating conditions. A reading without that information may not identify the same quantity as another reading with the same unit.

Resolution concerns the smallest change in the quantity being measured that produces a perceptible change in an indication. A digital display changing in increments of 0.01 cm describes its displayed granularity, but that increment alone does not account for all measurement limitations. Do not automatically assign uncertainty ±0.005 cm, or any other bound, from the final displayed digit. A standard uncertainty needs relevant information, a measurement model and an assessment method; we do not estimate one from absent data here.

Separate repeated agreement from closeness to a reference

Measurement precision concerns agreement among repeated indications or measured values under specified conditions. Repeatability is precision under a specified set of closely maintained conditions, such as the same procedure, operators, measuring system, location and a short time interval. A tight cluster of readings can indicate good repeatability while all readings remain offset from a relevant reference.

Measurement accuracy concerns closeness to a true quantity value; in practice, reference values and their uncertainties help assess the comparison. Accuracy is not established by repeatability or by a long decimal display. Calibration establishes a relationship between reference quantity values and instrument indications, with their uncertainties, under stated conditions. It does not mean every later reading is exact, and calibration is distinct from adjusting an instrument. State what was calibrated and which conditions apply.

Identify the uncertainty parameter before interpreting ±

Measurement uncertainty is a nonnegative parameter characterizing the dispersion of quantity values attributed to the defined measurand, using the information available. It is not the measurement error itself, and it does not automatically give a guaranteed interval containing an unknown true value. Writing ± without identifying its meaning leaves important information missing.

A standard uncertainty u expresses uncertainty as a standard deviation. An expanded uncertainty U is formed from a combined standard uncertainty u_c using a coverage factor k > 1: U = ku_c. These must be supplied or assessed with an appropriate model and method. If a report states a coverage factor or coverage probability, retain it and its conditions. A stated k alone does not establish a particular coverage probability or a guaranteed bound; do not invent either from the digits or the ± symbol.

An explicitly stipulated hard-bound model is different. In that model, every permitted value is guaranteed by assumption to lie in a given interval. The explorer’s nominal reading 12.34 cm has a stipulated hard interval [12.30, 12.38] cm, equivalently nominal value ±0.04 cm under that declared meaning. That is a toy bounding model, with no standard-uncertainty, expanded-uncertainty or probability interpretation supplied. Its guarantee belongs to the assumption, not to the notation alone.

Use interval arithmetic only in its declared model

Suppose an explicit model permits x in [a, b] and y in [c, d]. Adding gives x + y in [a + c, b + d]. If all endpoints are positive, multiplying gives xy in [ac, bd], because increasing either positive input increases the product. These are bounds for values allowed by the stated model. If every pair in the two intervals is allowed, the endpoints can be reached. Otherwise the interval may be an enclosing bound rather than an exact description of all attainable results.

For a rectangle model permitting every combination of L in [1.98, 2.02] m and W in [0.99, 1.01] m, the side sum is in [2.97, 3.03] m. Its perimeter is in [5.94, 6.06] m and its area is in [1.9602, 2.0402] m². These conclusions use positive dimensions, the rectangle formulas and the explicit hard bounds. The area interval is not symmetric about the nominal area 2 m². Do not replace it by 2 ± 0.04 m², which would exclude some permitted values.

These rules are not standard-uncertainty propagation formulas. Standard uncertainties involve a measurement model and information about distributions and dependencies; they cannot be substituted into a hard-bound rule without justification. The positive-product endpoint rule also must not be extended unchanged to arbitrary signed intervals. State the domain and assumptions before calculating.

Change units without changing the kind of uncertainty

The exact factor 1 cm = 10 mm scales a value and a stated uncertainty magnitude by ten. It also scales both endpoints of an explicitly given hard interval: 12.34 cm with hard bounds [12.30, 12.38] cm becomes 123.4 mm with hard bounds [123.0, 123.8] mm. A standard uncertainty stays a standard uncertainty, an expanded uncertainty stays expanded, and a stipulated bound stays a stipulated bound. Exact conversion does not improve the measurement.

Choose the final unit before choosing its decimal-place report. Two decimal places in cm give reporting step 0.01 cm = 0.1 mm; two decimal places in mm give step 0.01 mm. They are different physical rounding steps. Consequently the same quantity can have different rounded reports when you keep the decimal-place count while changing unit. Preserve the unrounded input and the stated uncertainty information, even when a displayed nominal value is rounded.

A report should identify the estimate, unit and meaning of its uncertainty. If ± denotes expanded uncertainty, give the supplied U, coverage factor and any supplied coverage probability with its conditions. If it denotes a hard bound, state that model explicitly. Follow a stated reporting policy when rounding the value or uncertainty for display, and retain the underlying information. Rounding does not shrink the original uncertainty or justify recentering an original interval on a new rounded value.

Compare a rounded report with its unchanged input information

Begin with the explorer’s exact positive input 12.345 cm, two decimal places and centimetre units. Its rounded report is 12.35 cm, with signed report-minus-input difference +0.005 cm. Select the exact negative preset −12.345 cm, a signed displacement rather than a segment length: at two centimetre decimal places it reports −12.35 cm and difference −0.005 cm. The exact small positive preset 0.04995 cm reports 0.05 cm at that setting; coarser rounding can display zero without changing the input to zero.

The fourth preset is nominal reading 12.34 cm with the explicitly stipulated hard interval [12.30, 12.38] cm. Rounding it to zero decimal places produces report 12 cm, which is outside that original interval. The interval remains attached to the original estimate; do not write 12 ± 0.04 cm or shift the interval to the rounded value. A zero report-minus-input difference at two decimal places would only mean that no display rounding occurred, not that a physical measurement error or uncertainty was zero.

Choose zero, one or two decimal places and either cm or mm. Four presets, three place counts and two units give 4 × 3 × 2 = 24 exact states. The explorer uses nearest rounding with ties away from zero and displays trailing zeros at the requested places. Unit conversion preserves the input and hard bounds, while the same place count changes the physical rounding step. At two mm places, the exact positive input reports 123.45 mm; its two-cm-place report converted afterward would be 123.50 mm. Displayed differences are numerical rounding differences from the supplied exact value or nominal estimate, never calculated unknown physical errors.

Check what the report actually claims

Identify the defined quantity, its unit, the source of the input and the conditions. Decide whether the value is stipulated exact, estimated from measurements or already rounded. Declare decimal places or significant figures and the tie rule. Keep unrounded information through calculation and conversion, then distinguish an exact rewrite from a final approximation.

For a measurement, keep resolution, precision, accuracy and calibration distinct. Identify whether uncertainty is standard, expanded or a separately stipulated hard bound, and preserve supplied factors, probabilities and assumptions. Do not infer guaranteed bounds from digits, ±, repeated agreement or a coverage factor alone. A rounding difference can be known while physical measurement error remains uncertain. The next lesson, Estimation and checking reasonable answers, uses rough comparisons and contextual checks to judge whether a result is plausible.

Worked examples

Try it yourself

1. A model stipulates exact length 8/3 m. Which sign fits each statement: 8/3 m … 800/3 cm and 8/3 m … 2.67 m? Would a real reading written with many digits become exact merely from that display?

   Show the answer and reasoning

   8/3 m = 800/3 cm is an exact unit rewrite. 8/3 m ≈ 2.67 m is a rounded approximation. A long displayed reading does not become an exact physical quantity value.

   The metre-to-centimetre factor is exact, while the finite decimal removes information from this non-terminating fraction. Exactness depends on definitions or stipulations. A measurement retains its model, conditions and uncertainty information regardless of how many output digits are shown.

2. Under ordinary reporting conventions, count decimal places and significant figures in 0.00450, 4.50 and 450.0. What further information is needed for a bare report 450? Does any digit count alone give measurement uncertainty?

   Show the answer and reasoning

   0.00450 has five decimal places and three significant figures; 4.50 has two and three; 450.0 has one and four. The trailing-zero intention in a bare 450 needs context or an explicit form such as 4.50 × 10^2. No digit count alone specifies measurement uncertainty.

   Leading zeros locate the decimal point, while deliberately retained trailing decimal zeros are significant under the stated convention. An exact count and a measurement report can use the same numeral with different meanings. Keep the report’s context and uncertainty type explicit.

3. Using nearest rounding with ties away from zero, round exact 2.345 cm and −2.345 cm to two decimal places. Separately round exact 0.04995 cm to three significant figures. Give the two signed report-minus-input differences for the first pair.

   Show the answer and reasoning

   The reports are approximately 2.35 cm and −2.35 cm, with differences +0.005 cm and −0.005 cm. The three-significant-figure report is approximately 0.0500 cm = 5.00 × 10^−2 cm.

   At an exact tie choose the greater magnitude, so negative and positive differences have different signs. Significant-figure rounding can carry into the next scale; retain the requested trailing zeros and renormalize scientific notation. These differences concern the known inputs, not unknown measurement errors.

4. Report the exact model length 8/3 m to two decimal places in cm. Compare converting first with first rounding the metre value to 2.67 m. Why does an exact conversion not fix the second route?

   Show the answer and reasoning

   Converting first gives 800/3 cm ≈ 266.67 cm. First rounding to 2.67 m and converting gives 267 cm. The earlier rounding discarded information before the requested report.

   Retain the exact fraction or unrounded value through conversion and apply the final rounding in the requested unit. The exact factor changes representation but cannot reconstruct discarded digits. If only a rounded input was supplied, that limitation must remain part of the result’s interpretation.

5. A display changes in 0.01 mm increments, and repeated readings under maintained conditions are 8.11, 8.12 and 8.11 mm. What do the display step and close repeated readings describe? Do they establish accuracy or justify uncertainty ±0.005 mm without further information?

   Show the answer and reasoning

   The display step describes displayed granularity, and the close repeated readings give information about precision or repeatability under those conditions. They do not establish accuracy or justify that uncertainty bound alone.

   Resolution, repeated agreement and closeness to a suitable reference are distinct questions. A calibration relation, reference uncertainties, measurement model and relevant conditions matter. We do not derive a standard uncertainty or guaranteed interval from the last digit or this small list alone.

6. A report gives estimate 0.732 m, combined standard uncertainty 0.004 m and expanded uncertainty 0.012 m with stated k = 3. Convert them to cm. Does the factor establish a particular coverage probability or a guaranteed interval? What must accompany a ± expression?

   Show the answer and reasoning

   The converted estimate is 73.2 cm, combined standard uncertainty 0.4 cm and expanded uncertainty 1.2 cm, with k = 3 unchanged. No particular probability or guaranteed bound follows from k alone. A ± expression needs its unit, uncertainty type and supplied coverage information or explicit bound model.

   Multiply each magnitude by the exact factor 100 and retain the report’s interpretation. The supplied relation U = ku_c remains 1.2 cm = 3 × 0.4 cm. Coverage probability requires the relevant stated method and conditions; it cannot be invented from digits, the symbol or the factor.

7. A toy model permits every pair x in [1.9, 2.1] m and y in [2.8, 3.2] m as hard-bounded positive lengths. Find intervals for x + y and xy. Why must these endpoint rules not be treated as standard-uncertainty propagation?

   Show the answer and reasoning

   x + y is in [4.7, 5.3] m and xy is in [5.32, 6.72] m². These are results of the explicitly stipulated positive hard-bound model, not calculated standard uncertainties.

   Add the two lower and two upper endpoints; for the positive product use 1.9 × 2.8 and 2.1 × 3.2. All pairs are allowed, so the endpoints are attainable. Standard uncertainty depends on a measurement model, distributions and dependencies; its interpretation differs from guaranteed model bounds.

What to carry forward

• Exactness comes from a definition or stipulation; use ≈ for a rounded approximation and distinguish a known rounding difference from physical measurement error.
• Declare decimal places or significant figures and the tie rule; this lesson uses nearest rounding with ties away from zero.
• Retain information through calculation and exact unit conversion, then round in the requested final unit; the same decimal-place count can mean a different physical step.
• Define the measured quantity and conditions, and keep resolution, precision, accuracy, calibration and uncertainty distinct.
• Identify standard or expanded uncertainty versus an explicit hard-bound model, preserve the supplied information, and never infer guaranteed bounds or probabilities from digits, ± or k alone.