Sampling choices and follow-up priorities

A useful sample need not copy the population proportions

Choosing which objects to observe is part of the measurement design. A population study may aim to estimate a mean in a known catalogue frame, compare rare and common groups, or characterize a conditional relation. Those aims need not favour the same allocation of observations.

A frame is the explicit list from which targets can be selected. A stratum is a defined group within that list. Deliberately assigning more observations to a small stratum can improve its characterization, but the combined unweighted sample then represents a different group mixture from the frame.

For an original exact benchmark, take a frame of 200 catalogue targets: 150 in a brighter group and 50 in a fainter group. Supply a dimensionless outcome of two for every brighter-group target and eight for every fainter-group target. These constants simplify the design arithmetic; they are not measured properties of real astronomical populations.

The frame mean is (150 × 2 + 50 × 8)/200 = 3.5. An independent question about all sky sources would require the catalogue’s own selection and coverage model. Sampling correctly within this frame does not establish that it contains every astronomical source.

A balanced sample and a frame mean use different weights

Suppose 40 targets can be observed. Simple random sampling without replacement from the full frame gives every target inclusion probability 40/200 = 0.2. The expected composition is 30 brighter and 10 fainter targets; these are expectations, not a fixed composition for every sample.

Alternatively, randomly select 20 targets within each stratum. Inclusion probabilities are 20/150 = 2/15 for a brighter target and 20/50 = 0.4 for a fainter one. The fainter group is deliberately oversampled relative to its frame fraction.

The balanced sample’s unweighted mean is (20 × 2 + 20 × 8)/40 = 5. To estimate the stated frame mean, use its group proportions: (150/200) × 2 + (50/200) × 8 = 3.5. Equivalently, inverse inclusion weights 7.5 and 2.5 recover (20 × 7.5 × 2 + 20 × 2.5 × 8)/200 = 3.5.

The exact agreement follows from the constant-within-stratum benchmark. Real outcomes vary and require sampling uncertainty; nonresponse and nonrandom choices within a stratum need additional treatment. Known frame counts alone do not repair every selection effect. Both the balanced comparison and the frame estimate can be useful when their different targets are stated.

The brightest window need not distinguish the explanations

A follow-up decision concerns a possible future measurement rather than a population average. It should ask which available target, band or time window could most improve the scientific inference under the current uncertainty and practical constraints.

For a separate original example, compare two already specified hypotheses H1 and H2. Each predicts the integrated mean count in either of two candidate 100-second windows. Assume Poisson counts, no background or read noise, known response and no remaining free amplitude.

In an early bright window both hypotheses predict 1000 counts. In a later faint window H1 predicts 40 and H2 predicts 10. Under H1, the ideal source-count SNRs are √1000 ≈ 31.6228 and √40 ≈ 6.3246. Those values describe detection strength in the supplied zero-background counting model.

The early window gives a strong possible detection but identical distributions under the two hypotheses. It cannot discriminate them in this setup. The fainter window has smaller SNR but different predicted count distributions. These are forecast means integrated over each window, not observed counts or instantaneous source rates.

Expected evidence is conditional and can be asymmetric

For one observed count N with positive predicted means λ1 and λ2, the natural-log likelihood ratio is ln[p(N | H1)/p(N | H2)] = N ln(λ1/λ2) + λ2 − λ1. The factorial cancels because the same observed N is used in both likelihoods.

If H1 is the declared distribution for the expectation, replace N by its mean λ1. Then E_H1[ln LR] = λ1 ln(λ1/λ2) + λ2 − λ1. In the early window this is zero because both means are 1000. In the late window it is 40 ln 4 + 10 − 40 ≈ 25.4518 nats.

A nat is the unit used for information expressed with natural logarithms. The expectation is over possible future counts under H1; it does not promise that the realized count will produce that value or favour H1. Choosing the reverse expectation or averaging over uncertain hypotheses defines a different criterion.

Thus this fixed-H1 calculation is not posterior probability, a guarantee of discovery or prior-averaged information gain. It is one explicit way to compare these two simple predictions. Real follow-up should retain uncertainty in model parameters and nuisance quantities rather than silently treating every prediction as fixed.

Scientific value must be balanced with feasibility and model risk

A later window can be informative in the model and still be unavailable because of visibility, instrument response, weather, source position, latency or competing observations. Include those costs and constraints in the choice. If a reference or repeated measurement is needed to interpret the result, budget it too.

Loredo’s adaptive-design framework averages possible future outcomes under predictive distributions and evaluates a stated utility. Its maximum-entropy simplification has conditions on the noise and design dependence; it should not be transferred automatically to signal-dependent Poisson observations such as this toy.

Adaptive priorities can also inherit model mistakes. If the current alternatives agree in a region because they both omit the same physics, always observing elsewhere may miss a revealing failure. Preserve the possibility of testing unexpected behaviour, and record the model, uncertainty, selection and utility that produced each choice.

For sampling, state the population and inclusion design. For follow-up, state what the next measurement could distinguish and under which forecast. A convenient or bright observation is valuable only in relation to that purpose. The next lesson formalizes detection rules, recovery probabilities and the conditions for ending or extending a search.

Worked examples

Try the reasoning

1. Does a correct sample from the 200-target frame automatically estimate the mean of every source on the sky?

   Show the answer and reasoning

   No.

   Catalogue coverage and selection define a separate population-inference problem.

2. Must a random 40-target full-frame sample contain exactly 30 brighter and 10 fainter targets?

   Show the answer and reasoning

   No.

   Those counts are its expected composition, not guaranteed realized counts.

3. Why does the early window have zero discriminatory value for the stated two hypotheses?

   Show the answer and reasoning

   They predict exactly the same Poisson distribution there.

   High detection SNR does not distinguish identical predictions.

4. Is an expected log likelihood ratio under H1 the same as information averaged over uncertain hypotheses?

   Show the answer and reasoning

   No.

   The averaging law and scientific utility differ and must be stated.

What to carry forward

• Target allocation and weighting should match the declared population question.
• A fainter measurement can distinguish predictions better than a bright detection.
• Follow-up priorities remain conditional on predictive uncertainty, utility, selection and practical constraints.