Astronomy · Testing models against the sky

Different explanations for similar data

Use a family of functions that agree at every available calibration node to distinguish a precise fit, parameter identification and a physical explanation.

Continue in the approved chapter order. Necessary vocabulary, quantities and assumptions are explained in the lesson.

Back to the chapter

After reading, you can

Explain why identification depends on the observing design and allowed model family.
Calculate where a finite set of calibration points is insensitive to a parameter.
Choose a new discriminating input while distinguishing new evidence from a prior or penalty.

A fitted curve is not automatically a unique explanation

Different source configurations or response models can predict similar recorded data. Finite precision can hide a small difference; an observing design can also make a parameter’s effect exactly invisible. These are different limitations, and neither is repaired simply by printing more digits from a fitting routine.

Identification asks whether different permitted parameter values lead to distinguishable observable predictions under the available design. It depends on the parameter domain, model family, measured quantities, reference information and observing precision. A numerical optimizer can return a single value even when the observations do not select it uniquely.

A physical mechanism claim is broader still. Several mathematical descriptions can fit an available curve, while independent spectral, spatial, temporal or dynamical evidence is needed to connect the description with an actual process. A successful parameter fit is one part of that argument.

The following original dimensionless response family isolates the design issue exactly. It does not represent a measured colour, radius, spectrum or astronomical emission engine. Its purpose is to make the unobserved freedom inspectable.

The extra parameter vanishes at every supplied node

Let u be a dimensionless explanatory quantity on 0 ≤ u ≤ 3. Let y = Q/Q0 be a dimensionless normalized response, where Q0 is an arbitrary fixed positive reference with the same unit as Q. The response is not a probability.

Define fα(u) = 1 + u² + αg(u), with g(u) = u(u − 1)(u − 2) and allowed parameter 0 ≤ α ≤ 0.25. Supply calibration targets only at u = 0, 1 and 2, with corresponding y values 1, 2 and 5.

At each calibration node one factor of g is zero. Thus g(0) = g(1) = g(2) = 0 and every allowed α predicts exactly the same three targets. The sensitivity ∂fα/∂α = g(u) is zero at every available calibration input.

Repeating those exact three nodes with arbitrarily high precision cannot determine α in this design. That is an exact absence of sensitivity, not merely a tiny effect beneath the noise. The family can be distinguished at other inputs, so this does not assert global nonidentifiability under every possible observing design.

Agreement at the nodes leaves different interpolations and extrapolations

Compare the two fixed candidates α = 0 and α = 0.25. At u = 0.5, g = 0.5 × (−0.5) × (−1.5) = 0.375. Their responses are 1.25 and 1.34375, a difference of 0.09375. They differ inside the calibrated interval even though both reproduce all three nodes.

At u = 1.5, g = −0.375. The responses are 3.25 and 3.15625, so the difference changes sign to −0.09375. A single assumed offset between the curves would not describe this permitted freedom.

Beyond the calibration interval [0,2], at u = 3, g = 6. The candidates predict 10 and 11.5, a difference of 1.5. The full table retains the shared nodes and the selected inside- and outside-range samples on the declared domain.

A plot with common linear scales can reveal where the functions meet and separate, but the exact polynomial factor is the reason the calibration cannot constrain α. Matching a finite collection of points does not determine an arbitrary function between them or guarantee its extrapolation.

Original normalized response functions on 0 ≤ u ≤ 3. The only calibration nodes are 0, 1 and 2; all other rows are predictions, not extra observed data.
u	g(u)	f0(u)	f0.25(u)	Difference f0.25 − f0
0	0	1	1	0
0.5	0.375	1.25	1.34375	0.09375
1	0	2	2	0
1.5	−0.375	3.25	3.15625	−0.09375
2	0	5	5	0
2.5	1.875	7.25	7.71875	0.46875
3	6	10	11.5	1.5

Same node values, different response functions

Scroll sideways to see the full diagram. Its explanation is below.

This original mathematical schematic compares two fixed members of the hypothetical family f_alpha(u)=1+u^2+alpha*g(u), where g(u)=u*(u-1)*(u-2), 0 <= alpha <= 0.25 and 0 <= u <= 3. A has alpha=0; B has alpha=0.25. u is a dimensionless normalized explanatory quantity, not time. The response y=f_alpha(u)=Q/Q0 is dimensionless, with one arbitrary fixed positive reference Q0. It is not a probability or an observed astronomical signal. The upper plot puts both responses on exactly the same linear axes: u from 0 to 3 and y from 0 to 12, both zero based. A is a solid cyan curve with circular selected-sample markers; B is a dashed orange curve with diamond selected-sample markers. Three hollow white circles mark their exact shared calibration nodes (u,y)=(0,1),(1,2),(2,5); identical points are not shifted apart. The other four selected samples are mathematical evaluations, not observations. The lower plot shows the different quantity Delta f=B-A=0.25*u*(u-1)*(u-2), on a separate linear vertical scale from -0.25 to 1.75, with an explicit zero line. Its horizontal scale is identical to the upper plot. Purple hollow circles mark all seven table rows. Delta f is positive between 0 and 1, negative between 1 and 2, zero at 0,1,2, and positive beyond 2. Its internal maximum/minimum occur at u=1 minus/plus 1/sqrt(3), with values plus/minus 1/(6*sqrt(3)), approximately plus/minus 0.096225044865. At u=3 it is 1.5. These are signed differences between positive response functions, not negative physical counts. Blue-tinted shading marks only the interval spanned by the calibration nodes, [0,2]. The dashed boundary at u=2 separates the plotted region beyond the nodes. Neither shading nor curves denote uncertainty, a posterior, validated behavior over a continuous interval or actual physical mechanisms. A dense mesh samples the same analytic functions; it supplies no new observations. At these three exact calibration nodes g(u)=0, so every alpha in the declared family gives the same values. Repeating only those nodes with arbitrary precision cannot determine alpha within this family. A new input with g(u) not equal to zero could constrain alpha within the stipulated family under suitable measurement/model conditions; it would not prove that the family contains the true mechanism. A regularization preference for one alpha is an added assumption, not identifying observational evidence. u (dimensionless) | g(u) (dimensionless) | A: f0(u)=Q/Q0 | B: f0.25(u)=Q/Q0 | Delta f=B-A (dimensionless) | Calibration node? 0 | 0 | 1 | 1 | 0 | yes 0.5 | 0.375 | 1.25 | 1.34375 | 0.09375 | no 1 | 0 | 2 | 2 | 0 | yes 1.5 | -0.375 | 3.25 | 3.15625 | -0.09375 | no 2 | 0 | 5 | 5 | 0 | yes 2.5 | 1.875 | 7.25 | 7.71875 | 0.46875 | no 3 | 6 | 10 | 11.5 | 1.5 | no

Constraints and regularization must be identified as added information

A routine might report α = 0 because that is its starting value, because it uses a minimum-norm convention, or because a penalty favours smaller coefficients. The three calibration values do not distinguish those choices within this family. Repeating the same routine can reproduce the same answer without making α observationally identified.

Regularization adds a chosen penalty to a fitting objective; a Bayesian prior supplies an explicit probability law. Either can be useful for selecting or stabilizing solutions. Its influence should be reported rather than attributed to information newly supplied by the calibration data.

The same distinction applies when a model’s parameter bounds exclude alternatives. A scientifically justified bound can be informative, but the result is conditional on that bound. A chosen parameter domain cannot silently become a measured exclusion outside it.

In less exact problems, small but nonzero sensitivity can produce broad practical uncertainty rather than a perfectly flat direction. Conditioning, finite-sample uncertainty and complete rank loss should be distinguished. The present construction is deliberately exact so that its limitation can be proved without a noise calculation.

Choose a prediction that changes under the competing candidates

A new measurement at a u with g(u) ≠ 0 can constrain α within the declared family, subject to its measurement and reference conditions. For an exact supplied response y at u = 0.5, solve α = (y − 1.25)/0.375. A response 1.34375 would give α = 0.25.

That algebra does not establish a precision for a real measurement. Uncertainty in y, u, calibration and the response family would need to be propagated or modelled. Nor does it show that one new point can distinguish every alternative family someone might propose.

A more useful physical investigation combines predictions for genuinely complementary observables or new regimes with independent reference checks. State which freedom each new observation addresses. A new band or time window can help because the allowed alternatives differ there, not simply because it adds another row to a table.

The next lesson follows both candidate functions through a declared training, tuning and final-test ledger. It shows how a lower error on the data used to choose a candidate can coexist with a worse prediction on an untouched outside-range point.

Worked examples

Test sensitivity at the available calibration inputs

What value of α is selected by exact targets (0,1), (1,2) and (2,5) within the stated family?

Evaluate g(u) = u(u − 1)(u − 2) at u = 0, 1 and 2.
All three values are zero, so fα at those inputs does not depend on α.
Every α in the allowed interval [0,0.25] reproduces the same targets.

No unique α is selected by these calibration values. Their exactness does not add sensitivity to the missing parameter direction.

Identify what a complementary input would add

Within the stipulated family, what α follows from an additional exact response y = 1.34375 at u = 0.5?

At u = 0.5, the base response is 1.25 and g is 0.375.
Subtract: 1.34375 − 1.25 = 0.09375.
Divide by 0.375 to obtain α.

α = 0.25 within this family and exact-input toy. A physical identification or uncertainty claim needs further assumptions and evidence.

Try the reasoning

Is y = 11.5 invalid because it exceeds one?
Show the answer and reasoning
No.
y is a normalized response, not a probability.
Does the three-node failure imply that this family can never be identified by any design?
Show the answer and reasoning
No.
At suitable other inputs g(u) is nonzero and the response depends on α.
A routine returns α = 0.000000000. Do those digits establish a measured value from the three nodes?
Show the answer and reasoning
No.
The observations are exactly insensitive to α there; the routine may be selecting a convention.
Does distinguishing these two functions prove one unique astronomical engine?
Show the answer and reasoning
No.
The functions are declared mathematical alternatives; mechanism identification requires an appropriate physical model and independent evidence.

What to carry forward

Identification depends on the allowed models and observing design.
Exact agreement at finitely many points can leave predictive freedom between and beyond them.
Distinguish information from new observations from a choice imposed by a prior, penalty or parameter bound.

References

Teaching resources and research supporting this lesson’s explanations and examples.