Controls, blinding, and experimental design

Specify the comparison before collecting readings

An experiment deliberately changes a factor and observes a response. A factor is something whose settings are compared; the response is the quantity recorded. The design states which settings are used, in what order, with what measurement procedure and what analysis. Write the question first.

Here the question is whether changing one declared setting from configuration A to configuration B changes a voltage indication. A is the control condition: the chosen reference configuration for comparison. B is the test condition. Keep the supplied voltage source, connections, readout procedure and other declared settings the same. A control reading need not be an exactly known true value or have zero uncertainty.

All numerical readings below are generated by an original exact teaching model. They are not laboratory observations or evidence of performance for a real device. The configuration effect means a change in indication in this model, not necessarily a change in the source voltage. Real causal interpretation requires applicable measurement evidence and a design that supports the intended comparison.

Identify unwanted changes and confounding

A nuisance factor can change the response although it is not the factor being investigated. Time order, temperature or a different operator may matter in a particular apparatus. Confounding occurs when the comparison mixes the intended factor with another influence, so that the reported contrast does not separate their contributions. Keep relevant conditions controlled or recorded; calling a configuration the control does not resolve every nuisance influence.

Declare four equally spaced chronological slots j = 1, 2, 3, 4. The index j is dimensionless; it is not a time in seconds. Let bⱼ = 0 for A and bⱼ = 1 for B. Supply the indication model rⱼ = β + δbⱼ + dj + εⱼ, with β = 100 mV, δ = 2 mV and d = 1 mV. Here β is a baseline, δ is the fixed B-minus-A change, dj is common linear drift with slot index, and εⱼ is an additional contribution. One millivolt is 0.001 V; every term is a voltage.

The model applies only to these four slots and allowed configurations. It assumes the same β, δ and d for every slot, additive effects, and no dependence on the preceding configuration. Such dependence is called carryover. Initially set every εⱼ to 0 mV.

Define the signed outcome and its unit

Use two A readings and two B readings. Their arithmetic means are the sum of the two readings divided by two. Define the outcome D = mean_B(r) − mean_A(r); positive D means the B mean is higher. D is a voltage, because the means and their subtraction retain the reading unit.

Averaging the supplied equation separately for A and B gives D = δ + d[mean_B(j) − mean_A(j)] + mean_B(ε) − mean_A(ε). The common baseline β cancels. The difference of mean indices is dimensionless, so its product with d is a voltage. This identity reveals exactly what the raw contrast includes in this model.

If all A readings are early and all B readings late, the drift term can enter D. The raw contrast alone then does not isolate δ when the drift contribution is unknown. Other analyses could use additional information or assumptions about drift; our calculation makes no claim that fitting this supplied model is impossible. The next lesson, Regression, parameter estimation, and model selection, develops further analysis tools.

Counterbalance order and compare within blocks

In the fixed order ABBA, A occupies slots 1 and 4, while B occupies slots 2 and 3. Both mean indices equal 2.5. The common linear drift therefore cancels from D in the supplied model. This is a counterbalanced order: the allocation balances this particular order influence. It is a deliberately chosen order, not a randomized assignment.

Blocking groups trials with a known nuisance condition in common and compares the intended settings within those groups. For our two consecutive pairs, each contains one A and one B. An additional constant offset shared by both readings in a pair cancels from that pair's B-minus-A difference. Drift within a pair still matters.

The first AB pair gives 3 mV and the following BA pair gives 1 mV. Their equally weighted mean is 2 mV because the linear-drift contributions have opposite signs. A curved drift, carryover or a configuration-dependent nuisance contribution need not cancel. Choose blocks and permitted orders for the actual apparatus; the arithmetic is not a guarantee for arbitrary conditions.

Randomize with a stated mechanism

Randomization uses a chance mechanism to assign allowed configurations or run orders before observing their responses. Specify that mechanism. It is distinct from selecting a representative sample of a wider population and does not automatically make measurement errors independent. A blocked experiment can randomize assignments within its blocks.

Suppose all six orders with exactly two A and two B slots are allowed, and a mechanism chooses each with probability 1/6 before readings are obtained. Keep β, δ and d fixed as supplied, set εⱼ = 0 mV, and retain the absence of carryover. The expected D is the sum of the six D values multiplied by 1/6. This is an average over hypothetical assignments, not an extra set of measured readings.

The six contrasts are 4, 3, 2, 2, 1 and 0 mV, so their expected value is 2 mV. A single selected order can still mix drift with the configuration change. Randomization balances this supplied drift in expectation; it does not make every realization accurate, remove every systematic contribution or authorize a confidence interval without further analysis.

State who is blinded and what is hidden

Blinding withholds specified information from a person while decisions that could depend on it are made. State the role, the hidden information and the time of disclosure. In a physics context, CMS describes withholding access to selected analysis information while choices are developed and checked, followed by unblinding. This supports the method's purpose, not a guarantee that all bias disappears.

In an original procedure, an operator can see which configuration is connected, while an analyst receives readings with coded configuration labels. A separate key identifies which code means A and which means B. The analyst fixes processing and permitted exclusions before the key is released, then applies the declared B-minus-A calculation. This is analyst blinding; the operator is not blinded to configuration.

Physical settings or recognizable patterns may reveal an assignment despite coded labels. Describe those limits and who can access the key. If hiding the setting from the operator is impractical, record that fact and use other feasible safeguards. Blinding does not replace a comparison condition, a suitable run order, measurement calibration or an uncertainty assessment.

Predeclare the outcome, stopping and deviations

Before collecting the readings, specify D, its sign and unit, the configurations, the allowed order mechanism, four complete valid slots, the measurement procedure, and rules for invalid or missing readings. State when collection stops and how interrupted runs are handled.

Preregistration records an analysis plan before the relevant analysis so that planned and subsequent choices can be distinguished. Center for Open Science guidance permits transparent changes and clearly identified exploratory analysis. A saved plan does not prove that the measurement model is right, and useful unexpected findings can be reported with their actual status.

For ABBA, stopping after the first AB pair reports 3 mV instead of the planned four-slot contrast of 2 mV in our exact model. These are different data collections. If apparatus failure or a safety concern interrupts a real run, preserve the acquired records and document the departure; do not quietly present an outcome-dependent early stop as the completed declared design.

Limit the conclusion and repeat with a purpose

Now retain the same model but allow each additional contribution to obey |εⱼ| ≤ 0.25 mV, with no probability distribution or independence supplied. For ABBA, each configuration mean has an additional contribution between −0.25 mV and +0.25 mV. Their difference lies between −0.50 mV and +0.50 mV, so 1.50 mV ≤ D ≤ 2.50 mV. This is a deterministic model bound, not a statistical confidence interval or a result standard uncertainty.

The bound covers only those stipulated ε contributions and the stated model. Unmodeled curved drift, carryover, changes of settings or reference information would require further assessment. A matched control and a balanced contrast can support a particular comparison; they do not by themselves establish a mechanism, a universal law or suitability under different conditions.

A fresh, complete acquisition under a specified design can examine whether a comparison persists and whether assumptions need revision. Recalculating the same four rows supplies no new readings, and repeating a run can retain shared unwanted effects. State what was repeated and under which conditions. Broader reproducibility and replication require further treatment; begin here by retaining the plan, order, raw records, deviations and scoped conclusion.

Worked examples

Try it yourself

1. For ABAB in the supplied model with εⱼ = 0 mV, find D and its common linear-drift contribution.

   Show the answer and reasoning

   D = 3 mV, including a 1 mV drift contribution.

   A occupies slots 1 and 3, with mean index 2; B occupies 2 and 4, with mean index 3. Thus D = 2 mV + (1 mV)(3 − 2) = 3 mV. The table means 102 mV and 105 mV agree.

2. In ABBA with εⱼ = 0 mV, increase the baseline β by 7 mV for all four readings. Does D change?

   Show the answer and reasoning

   No. Both configuration means increase by 7 mV, so D remains 2 mV.

   The new A mean is 109.5 mV and the new B mean is 111.5 mV. Their difference is 2 mV. This cancellation concerns one shared additive baseline; a change affecting only B would enter the contrast.

3. Modify ABBA by adding qj² to each reading, with q = 0.50 mV and εⱼ = 0 mV. Keep the original β, δ and d. Find D and assess whether ABBA cancels this added drift.

   Show the answer and reasoning

   D = 1 mV; the added curved drift contributes −1 mV and does not cancel.

   For B, mean(j²) = (2² + 3²)/2 = 6.5; for A it is (1² + 4²)/2 = 8.5. Their difference is −2. The original linear drift still cancels, so D = 2 mV + (0.50 mV)(−2) = 1 mV. This is a supplied alternative model, not evidence of actual curved drift.

4. For ABBA with |εⱼ| ≤ 0.25 mV, assign +0.25 mV to both B readings and −0.25 mV to both A readings. Find D. Does the supplied bound imply a 95% confidence statement?

   Show the answer and reasoning

   D = 2.50 mV, at the upper bound. No 95% confidence statement is supplied.

   The additional B-minus-A mean contribution is 0.25 mV − (−0.25 mV) = 0.50 mV. Adding δ gives 2.50 mV. A bound on each additional contribution contains no probability law, sampling statement or coverage calculation.

5. The operator sees A or B on the apparatus. An analyst receives coded configuration labels and fixes processing before a separate key is released. Which role is blinded to configuration, and what limitation remains?

   Show the answer and reasoning

   The analyst is intended to be blinded to configuration; the operator is not. Codes may still be revealed by recognizable clues.

   Blinding is specified by role, information and disclosure time. Record whether clues or access to the key undermine the intended masking. Coded labels neither hide a visibly operated setting from the operator nor replace design and measurement checks.

6. An ABBA plan specifies four complete valid slots and D from both configuration means. With εⱼ = 0 mV, collection instead stops after the first AB pair because its contrast is positive. What contrast is then available, and may it be presented as the completed four-slot outcome?

   Show the answer and reasoning

   The available pair contrast is 3 mV. It is not the completed four-slot outcome, which the model would give as 2 mV.

   The first pair gives 104 mV − 101 mV = 3 mV. The planned second, reversed pair would give 1 mV, making the full mean 2 mV. Preserve the partial records and disclose the changed stopping decision and analysis; positivity alone does not complete the declared design.

What to carry forward

• A control is a declared comparison condition; it does not automatically supply an exact true value.
• Specify the outcome's sign, units and procedure, and inspect what nuisance contributions its contrast contains.
• A fixed balanced order cancels only the influences supported by its assumptions; blocking and randomization have distinct roles.
• A randomization expectation concerns a stated chance mechanism and does not guarantee every realization.
• Describe blinding by person, information and disclosure time, including practical limits.
• Retain the prior plan, raw records, stopping decisions and deviations; new acquisitions can test persistence without guaranteeing accuracy or universal causation.