Chemistry · Lesson

Open problems, research design and communicating uncertainty

Design bounded comparisons, retain independent units and communicate how evidence changes a specified question.

By the end, you can

Calculate a paired-unit estimate using the correct independent count.
Compare alternative predictions under explicit deterministic bounds.
Evaluate multiple assigned false-alarm opportunities.
Apply explicit Bayes and Shannon definitions to a hypothetical two-alternative test.

A research question should distinguish possible explanations

State the observation to explain and the alternatives a proposed study could distinguish. A hypothesis should connect its assumptions to observable consequences within a defined system. Decide which result would weaken or support the claim.

A mechanistic account, a useful prediction and a historical explanation can need different evidence. A model can describe an observation well while leaving its interpretation unresolved.

Use observations that address the stated alternatives

A useful comparison can place alternatives where their predictions differ. The original low/high case uses an assigned deterministic error bound, so it establishes compatibility under that rule. It does not assign a probability distribution for observational error or certify a unique explanation.

The paired-unit example retains before and after readings within each independent unit. Its sample variability is calculated from the five unit differences. Repeating a reading within the same unit does not create an independently assigned unit; a design and uncertainty account should preserve that hierarchy.

The NIST primary guidance distinguishes paired differences and experimental units. The GUM separately requires the conditions and contributions relevant to a measurement uncertainty account. An estimated standard error across invented unit differences is only the stated sampling estimate here, without a complete calibration or model-uncertainty budget.

Each paired difference belongs to one independent unit

Independent unit	Before / AU	After / AU	Difference / AU
1	12	14	2
2	16	19	3
3	19	20	1
4	22	25	3
5	26	30	4

Original arbitrary-unit paired responses. Five assigned independent units supply five differences. Exact technical duplicates within a unit leave these differences and the independent count unchanged; this is not an actual treatment study.

A study needs the right comparison and experimental units

Define materials, conditions, endpoints, controls and independently assigned experimental units. A repeated reading or subsample is not automatically a new independent unit. The design determines what a comparison can establish.

Randomization, blinding, predefined analysis and suitable control observations can address different sources of bias. Larger samples do not automatically remove systematic errors, missing alternatives or inadequate measurement.

A decision probability and model uncertainty answer different questions

Repeated opportunities for a false alarm can increase the probability of at least one under an explicitly independent null-decision model. Preserve how checks were selected and related; an actual design may violate the independence assignment. Statistical thresholds alone do not establish effect size, practical consequence or a mechanism.

The original Bayesian example assigns two alternatives, their priors and conditional result probabilities. Expected information uses base-two Shannon uncertainty about that label. A result can reduce this defined uncertainty while leaving omitted alternatives, flawed likelihoods or practical usefulness unresolved.

Shannon’s original information treatment supplies the definition, while all probabilities and examples here are new assignments. Expected label information is different from thermodynamic entropy and does not by itself justify an actual study, instrument claim or purchase.

An uncertain result can still change what is reasonable to conclude

A measurement uncertainty, sampling variability and model uncertainty describe different limitations. State what a reported interval or probability represents. A statistical threshold does not by itself measure an effect’s practical importance or prove a mechanism.

Evidence can update a defined comparison without giving certainty. Bayesian probabilities require their prior and likelihood assumptions; frequentist quantities answer different questions. Neither removes the need to examine how the data were produced.

A useful report connects its conclusion to the evidence

Report the tested scope, observed quantities, transformations, uncertainty and remaining alternatives. Preserve negative or unresolved results when they affect interpretation. A reproducible record helps other researchers test the account.

The original research cases below illustrate bounded inference and follow-up choices. They make no announcement of a discovery and do not turn an open question into established knowledge through a confident label.

Worked examples

Use the independent unit in a paired comparison

Assign five independent units with paired before responses [12,16,19,22,26] and after responses [14,19,20,25,30], all in arbitrary units (AU). Find the mean paired difference and its sample-based standard error under this independent paired-unit model.

Within-unit differences after minus before are [2,3,1,3,4] AU. Their mean is 13/5 = 2.60 AU. Pairing keeps the comparison within the same unit.
Squared deviations from the mean sum to 5.20 AU². Sample variance of differences is 5.20/(5−1) = 1.30 AU², giving sample standard deviation 1.140175425 AU.
With five independent differences, the estimated standard error of their mean is s/√5 = 0.509901951 AU. This estimate concerns variation across the stipulated units; it is not a complete measurement-uncertainty budget.
Ten before/after readings do not make ten independent paired units. The numbers are invented, without an actual treatment effect, randomized design or mechanistic conclusion.

Mean paired difference 2.60 AU; sample-based standard error 0.509901951 AU for five independent units.

Choose a comparison that separates predictions

Invent two conditions called low and high. Model A predicts [10,20] AU and model B [10,10] AU. Assigned observed values are [10.5,19.5] AU, with a deterministic absolute-error bound of 1.00 AU per condition. Which predictions are compatible?

At low, both predictions differ from the observation by 0.50 AU, within the assigned bound. That condition does not distinguish A and B under this rule.
At high, A differs by 0.50 AU and B by 9.50 AU. The high observation is compatible with A and incompatible with B under the fixed error bound.
No probabilistic error distribution or probability that a model is true has been assigned. Compatibility compares only these predictions and bounds. Other alternatives can still agree; a unique mechanism does not follow from this two-model test.

Low permits both assigned predictions; high permits A and excludes B under the stated deterministic bound.

Several independent checks can create more false alarms

Under six independent null checks, assign a false-alarm probability of 0.0500 to each check. Find the probability of at least one false alarm across this toy decision process.

Probability of no false alarm on one check is 0.9500. Independence gives probability of none across six as 0.95⁶ = 0.735091890625.
The complementary probability of at least one is 1 − 0.95⁶ = 0.264908109375, about 26.49%. It exceeds the assigned per-check probability.
This is a probability model for decisions made under the stipulated null conditions. It does not make a p-value the probability that a hypothesis is false. Dependence, selected analyses and a real decision rule would require their own account.

Probability of at least one false alarm 0.264908109375 under six independent assigned checks.

Expected information concerns specified alternatives

Assign alternatives A/B with priors 0.500/0.500. A hypothetical test has P(+|A) = 0.800 and P(+|B) = 0.200, with positive and negative as exhaustive outcomes. Find the posterior probabilities and expected reduction in Shannon uncertainty about this two-alternative label.

P(+) = 0.500×0.800 + 0.500×0.200 = 0.500. Bayes’ rule gives P(A|+) = 0.800 and P(A|−) = 0.200. The prior and conditional result probabilities are explicit teaching assignments.
For a binary label, H(p) = −p log₂p − (1−p)log₂(1−p). Prior uncertainty is H(0.500) = 1 bit. Either outcome gives uncertainty H(0.800) = H(0.200) = 0.721928094887 bits.
The expected posterior uncertainty is therefore 0.721928094887 bits, and expected information gained is 1 − 0.721928094887 = 0.278071905113 bits.
This expected Shannon information concerns the stipulated alternatives. It is not thermodynamic entropy, guaranteed truth, observed instrument performance or a complete measure of practical usefulness. Practical study choices also require evidence beyond this two-alternative information model.

Posteriors for A are 0.800 after positive and 0.200 after negative; expected information gained is 0.278071905113 bits.

Try it yourself

Duplicate each technical reading within its original unit, making 20 readings rather than 10 without changing any values. How many independent paired units and what estimated standard error remain?
Show the answer and reasoning
Five independent paired units; mean difference 2.60 AU and estimated standard error 0.509901951 AU remain.
Exact within-unit duplicate readings do not create new independent unit differences. The stipulated across-unit estimate retains n = 5.
Change only high-condition observation to 10.5 AU. Under the same deterministic 1.00 AU bound, which models remain compatible at low and high?
Show the answer and reasoning
Low permits both; high permits B and excludes A.
High differences are now 0.50 AU for B and 9.50 AU for A. Compatibility changes with the assigned observation, without proving a unique explanation.
Increase only the number of independent null checks to 12. Find the probability of at least one false alarm.
Show the answer and reasoning
1 − 0.95¹² = 0.459639912337, about 45.96%; probability of none is 0.540360087663.
Use the complement of all twelve no-alarm outcomes under the stipulated independence and per-check rate.
Keep equal priors but change P(+|A) to 0.600 and P(+|B) to 0.400. Find posteriors for A and expected information gained.
Show the answer and reasoning
P(A|+) = 0.600 and P(A|−) = 0.400; expected posterior uncertainty 0.970950594455 bits, expected information gained 0.029049405545 bits.
The outcome-conditioned probabilities are closer to the equal prior, giving a smaller expected reduction in uncertainty about these assigned alternatives.

What to carry forward

A comparison needs predictions that can differ under its stated conditions.
The independent unit determines the sample count in a paired analysis.
Several false-alarm opportunities require an explicit combined decision account.
Probabilities and information measures need their assigned priors, likelihoods and scope.

References

Teaching resources and reference material supporting this lesson’s explanations and examples.