GLOBAL KNOWLEDGE ARK

Engineering & Technology · Quality Engineering, Experiments & Process Evidence

Measurement-system evaluation, calibration and uncertainty

A process appears to shift after a new operator starts measuring it. The parts might have changed, but the measurement method might also have changed. Before adjusting the production process, investigate the observations as the output of a measurement system.

Review measurement and uncertainty first. Means, ranges, square roots and percentages are used directly. The inference lesson defines its paired-difference model and supplies its Student-t quantile; mathematics develops probability and inference further. These examples teach interpretation rather than prescribe production sampling, monitoring or release rules.

Back to the chapter

After reading, you can

Evaluate measurement variation and reference relationships before using observations to judge a process change.

01

Specify the measurement conditions

A measurement result concerns a defined quantity and includes a method and relevant uncertainty. Instrument resolution, fixturing, environment, operator technique and item geometry can influence it. Repeating the same display reading is useful evidence about a particular setup; it does not establish every source of uncertainty or long-term consistency.

Repeatability concerns variation under stated repeat conditions. A reproducibility study changes specified conditions, such as operators or laboratories, and examines the result. The terms need explicit definitions for the study. A comparison is more informative when representative items are measured repeatedly across the relevant conditions with an order that avoids systematic confounding.

02

A small range is only one property

In our original fixed-reference exercise, operator A reports 19.98, 20.00, 20.02 mm; B reports 20.08, 20.10, 20.12 mm. Both ranges are 0.04 mm, yet the means differ by 0.10 mm. The supplied reference value is 20.00 mm. These observations suggest a difference to investigate rather than proving that B is the sole cause or that the reference is exact.

Calibration establishes a relationship to a specified reference under stated conditions. Adjustment changes the instrument or system; it is a separate action, even when a workflow includes both. A correction inferred from calibration also has uncertainty. Check the range, conditions, traceability and continued stability needed for the particular result rather than treating a calibration label as a permanent guarantee.

Similar repeat spread can coexist with a location differenceOriginal same-reference measurement exercise, mm. A 19.98, 20.00, 20.02; B 20.08, 20.10, 20.12. Both ranges 0.04 mm, means differ 0.10 mm. The supplied reference value 20.00 is not treated as uncertainty-free truth.Original repeated readings · one reference item, fixed setupReference value 20.00AB19.9620.0020.0420.0820.1220.16Both ranges 0.04 mm · observed mean difference 0.10 mm
Repeated readings A 19.98, 20.00, 20.02 have mean 20.00 mm; B 20.08, 20.10, 20.12 have mean 20.10 mm. Both ranges 0.04 mm. The difference suggests a configuration/operator relationship to investigate; it does not isolate a cause or establish bias against an exact truth.
03

Make operator and setup comparisons on the same items

A small original follow-up crosses two stable identified items P/Q, two operators A/B and two setups S1/S2. Each operator measures each item in each setup once in each of two rounds, removing and repositioning the item for every reading. The table is grouped for interpretation, not run order. One possible interleaved first-round order is C6, C3, C5, C4, C1, C8, C2, C7; the second is C7, C2, C4, C8, C3, C5, C1, C6. Record environment and a reference check with each round. This constructed example illustrates a crossed comparison; it is not a qualified gauge R&R study or a representative sample of all operators, days or item geometries.

Compare like with like. For item P under S1, B−A=20.02−20.00=0.02 mm using the two-reading means. The same operator contrast is 0.02 under S2 and for Q. For P measured by A, S2−S1=20.10−20.00=0.10 mm; that contrast is also 0.10 for B and Q. Q−P is 1.00 mm under every matched operator/setup combination. Because every condition measures both items, an item difference is no longer inseparable from choosing one operator or setup.

The second reading is 0.02 mm above the first in every cell. That shared round pattern might reflect a time-dependent condition, not eight independent demonstrations of random scatter. Interleaving avoids assigning one entire setup to the first round, but does not identify the physical reason for the round shift. Two rounds do not establish long-term stability, and agreement between conditions does not identify the correct absolute value. The separately supplied reference and uncertainty information remains necessary.

Suppose an apparent production increase of 0.12 mm was recorded while measurement changed from A/S1 to B/S2. In this follow-up, the same unchanged item shows precisely that contrast: 20.12−20.00=0.12 mm for P and 21.12−21.00=0.12 for Q. Thus measurement-condition change can explain the entire apparent increase in this teaching case. Re-measure retained before/after items under matched conditions before attributing it to production. The follow-up does not retroactively prove the cause of the earlier six-reference-reading exercise, nor does it establish a universally transferable 0.12 mm correction.

Original readings in mm; each C row is one item/operator/setup cell, visited once in each round. No actual measurement study is reported.
CellItemOperator / setupRound 1Round 2
C1PA / S119.9920.01
C2PB / S120.0120.03
C3PA / S220.0920.11
C4PB / S220.1120.13
C5QA / S120.9921.01
C6QB / S121.0121.03
C7QA / S221.0921.11
C8QB / S221.1121.13
Matched items separate two relative measurement contrastsFor P, mean readings A/S1 20.00, B/S1 20.02, A/S2 20.10, B/S2 20.12 mm. For Q, corresponding means are 21.00, 21.02, 21.10, 21.12 mm. Subtracting each item’s A/S1 mean makes their plotted relative means coincide: A 0.00→0.10 mm, B 0.02→0.12 mm from S1 to S2. Lines connect categorical setups, not a continuous geometry coordinate. Each cell has two separately repositioned readings; a common +0.02 mm round difference remains to investigate. Relative agreement does not identify a correct absolute value or estimate variance components.Matched items separate two relative measurement contrastsConstructed cell means · each item centred on its own A/S1 mean0.000.020.040.060.080.100.120.14Relative mean (mm)S1S2SetupB (triangle)A (circle)Both P and Q give the same relative lines; their absolute means differ by 1 mm.Setup difference 0.10 mm; operator difference 0.02 mm. Neither establishes absolute truth.
For P, mean readings A/S1 20.00, B/S1 20.02, A/S2 20.10, B/S2 20.12 mm. For Q, corresponding means are 21.00, 21.02, 21.10, 21.12 mm. Subtracting each item’s A/S1 mean makes their plotted relative means coincide: A 0.00→0.10 mm, B 0.02→0.12 mm from S1 to S2. Lines connect categorical setups, not a continuous geometry coordinate. Each cell has two separately repositioned readings; a common +0.02 mm round difference remains to investigate. Relative agreement does not identify a correct absolute value or estimate variance components.
04

Combine relevant uncertainty components

Suppose a separate teaching model supplies standard uncertainty components 0.02 mm from repeat variation, 0.03 mm from the reference relationship and 0.04 mm from setup. Assume they are uncorrelated, enter with sensitivity coefficients 1 and cover distinct contributions without double-counting. Combined standard uncertainty is the square root of 0.02²+0.03²+0.04², approximately 0.0539 mm.

Multiplying that standard uncertainty by a stated factor 2 gives expanded uncertainty about 0.108 mm. A coverage interpretation requires the distribution, information and factor choice to be appropriate; factor 2 does not universally guarantee 95% coverage. Correlation or another measurement equation can require a different combination. Use the model and uncertainty of the actual result, not a convenient sum of every available specification.

Interpret the difference without overclaiming

A’s mean is 60.00/3=20.00 mm and B’s is 60.30/3=20.10 mm. Their common range 0.04 mm does not cancel the 0.10 mm mean difference. Repetition has exposed both short-term spread and a between-condition difference, while leaving the physical cause unresolved.

The separate budget gives √0.0029=0.05385 mm standard uncertainty. It does not derive from those six readings and must not be presented as their estimated uncertainty. A planned follow-up could cross operators and setups on identified items to distinguish candidate effects, while retaining reference and environmental information for interpretation.

Try the reasoning

  1. Does a small repeated-reading spread establish closeness to a reference?

    Show the answer and reasoning

    No. Precision and the relationship to a reference are different questions.

  2. Why can adding reference uncertainty twice be wrong?

    Show the answer and reasoning

    If it is already included in another component, the budget would double-count the same contribution.

  3. Does expanded uncertainty with factor 2 always mean 95% coverage?

    Show the answer and reasoning

    No. Its coverage interpretation depends on the model, distributions, available information and factor choice.

  4. A proposed process comparison uses operator A/S1 before the change and B/S2 after it, with observed increase 0.12 mm. Which additional comparison would most directly address the measurement explanation?

    Show the answer and reasoning

    Measure identified retained before/after items under common conditions, with the order interleaved and relevant reference/environment records. The crossed teaching table shows a 0.12 mm A/S1-to-B/S2 difference on unchanged items, so the observed increase alone does not isolate production. Measuring only new items repeatedly with B/S2 would leave the original condition change confounded with the process comparison. Any correction needs its own applicability and uncertainty evidence.

  5. For a separate revised uncertainty model, setup standard uncertainty falls from 0.04 to 0.01 mm while repeat and reference components remain 0.02 and 0.03 mm. Under the original uncorrelated, unit-sensitivity assumptions, what changes?

    Show the answer and reasoning

    Combined standard uncertainty becomes √(0.02²+0.03²+0.01²)=√0.0014≈0.0374 mm, compared with 0.0539 mm before. With stated factor 2, expanded uncertainty is about 0.0748 mm. This smaller supplied budget does not prove the setup improved, establish the coverage probability or justify treating the observed 0.12 mm condition difference as another independent standard-uncertainty component.

What to carry forward

Characterise the measurement process and its result-specific uncertainty before interpreting a change in the observed production values.

References

Teaching resources and research supporting this lesson’s explanations and examples.

  1. National Institute of Standards and Technology (NIST)5 sources
    1. e-Handbook §2.4.1: Gauge-study issuesNIST/SEMATECHReference guideitl.nist.govRead source
    2. e-Handbook §2.3: CalibrationNIST/SEMATECHReference guideitl.nist.govRead source
    3. e-Handbook §2.5.2: Approach to uncertaintyNIST/SEMATECHReference guideitl.nist.govRead source
    4. e-Handbook §2.4.2: Design considerations for gauge studiesNIST/SEMATECHReference guideitl.nist.govRead source
    5. e-Handbook §2.4.5.5: Geometry/configuration differencesNIST/SEMATECHReference guideitl.nist.govRead source