Biology · Biological Frontiers & Open Questions
Predictive biology, AI-assisted research and validation challenges
A prediction method has mean squared error one in one context and twenty-five in another. Calling it simply accurate would conceal where its claim holds. Predictive biology asks what can be inferred about a biological quantity from available information, how the prediction is tested and what the test leaves unresolved.
Earth, Origins & the History of Life introduces the difference between biological history and processes possible today. Development, Differentiation & Stem Cells introduces cell states and organismal organisation. Genomics, Omics & Bioinformatics distinguishes molecular records, identity, proportions and validation. Systems & Quantitative Biology supplies state, observation and model-comparison reasoning. Each lesson also explains its local terminology; the original examples require only fractions, means and signed differences. Follow the concrete cases before returning to the broader research question.
Back to the Biology mapBy the end, you can
Identify the quantity a biological model predicts and evaluate its performance and limits in relevant held-out contexts.
State the target before judging the model
A predictive task specifies its inputs, target quantity and intended domain. A protein-structure task concerns a represented geometry, while another task might concern activity, expression or an organismal outcome. Success at one target can be valuable without answering all the others.
Machine learning can learn useful regularities from supplied records. Its output still depends on the task, training information and evaluation. A prediction that agrees with observed outcomes is not automatically an explanation of the causal process generating them. Several mechanisms can produce the same predictive relationship.
Structural prediction provides a concrete example of a defined computational task. A model’s structural agreement can help generate functional hypotheses, but activity, state changes and relevant host-context outcomes need their own evidence. A missing functional result cannot be silently replaced by a positive geometric comparison.
A structural prediction does not fill every biological evidence field
Read the diagram in words
The structural comparison has a supplied positive result. Activity, dynamics and host-context outcome remain unknown. None of those three unknowns becomes true or false merely because geometry agrees.
| Different biological question | Evidence supplied for this question |
|---|---|
| Agreement with a selected structural target | Agreement is supplied in the selected structural comparison |
| Activity under the relevant functional test | Evidence not supplied for this question |
| State changes under the relevant conditions | Evidence not supplied for this question |
| Outcome in the relevant host context | Evidence not supplied for this question |
Original anonymous evidence inventory, not a protein sequence, structure, design or actual AlphaFold output. Geometry fit is a stipulated observed agreement for one target. Activity,dynamic state and host-context function are unmeasured, represented bynull. No confidence score, biological mechanism, binding prediction, treatment utility or inference algorithm is supplied.
New records must match the intended claim
Training records influence the fitted prediction rule. Evaluation records should test the claim on relevant information not already used to determine that rule. Repeated records from the same biological unit or closely related contexts can remain dependent. A file split alone does not guarantee the biological independence or novelty needed for the question.
The example supplies four distinct evaluation units in each of two held-out contexts. All values use one assigned scale. The errors in A have magnitude one; those in B have magnitude five. The records therefore show different prediction performance across the supplied contexts without claiming a universal result from eight units.
Changing an evaluation split can also change what is extrapolated. A test on another location, time, lineage or measurement system may ask a harder or simply different question from a test on a new record within the same setting. A useful report identifies that target of generalisation and the uncertainties the available records can support.
A prediction can be accurate in one domain and weaker in another
Read the diagram in words
In A the errors are −1/1/−1/1, so each squared error is 1 and their mean is 1. In B all errors are −5, so each square is 25 and the mean is 25. The eight units have distinct identities.
| Distinct evaluation unit identity | Assigned observed outcome | Assigned prediction | Observed minus predicted value | Squared error in compatible squared units |
|---|---|---|---|---|
| a1 | 2 | 3 | -1 | 1 |
| a2 | 4 | 3 | 1 | 1 |
| a3 | 6 | 7 | -1 | 1 |
| a4 | 8 | 7 | 1 | 1 |
Mean of four squared errors: (1 + 1 + 1 + 1) / 4 = 1
| Distinct evaluation unit identity | Assigned observed outcome | Assigned prediction | Observed minus predicted value | Squared error in compatible squared units |
|---|---|---|---|---|
| b1 | 2 | 7 | -5 | 25 |
| b2 | 4 | 9 | -5 | 25 |
| b3 | 6 | 11 | -5 | 25 |
| b4 | 8 | 13 | -5 | 25 |
Mean of four squared errors: (25 + 25 + 25 + 25) / 4 = 25
e = y − ŷ; MSE = Σ e²/n
Original held-out predictions with no source biological data, machine-learning implementation, parameter fitting or algorithm claim. Training records are unspecified and both evaluation contexts are separate from them. Every evaluation unit is distinct; all observed/predicted values use a common arbitrary scale. Squared error is a defined numerical criterion, not model mechanism, causality, calibration, safety or clinical utility. Four records per domain are not universal accuracy evidence.
A generated hypothesis needs an evidence path
An AI-assisted research output can be a prediction, suggested explanation, summary or candidate to investigate. These products have different validation needs. A plausible statement is not a measured result, and a reference supplied by a system is useful only when its actual content supports the claim.
For a biological prediction, evaluation should preserve the identities, relevant conditions and meaning of the target. A chosen numerical metric answers a limited question. Good performance on that metric does not alone establish calibration, causal interpretation, successful transfer or practical benefit.
A productive use of computational assistance makes the next test clearer. It can expose alternative predictions, help organise evidence or identify where a model is weak. The important research distinction is between what the system generated, what was independently checked and what new biological observation would actually change the claim.
Worked example
Compute error within each evaluation context
Define error as observed minus predicted. In A, outcomes 2, 4, 6 and 8 have predictions 3, 3, 7 and 7. Errors are −1, 1, −1 and 1. Their squares sum to four, and dividing by four gives mean squared error one.
In B, the same assigned outcomes have predictions 7, 9, 11 and 13. Every error is −5; every square is twenty-five. The sum is one hundred and the mean is twenty-five.
The criterion compares these two supplied evaluation sets. It does not explain why the performance differs, estimate every possible future error or establish a causal mechanism. Separate structural agreement also leaves the three unmeasured functional evidence fields unknown.
Check your understanding
Why is a protein-structure prediction not automatically a prediction of activity in a host?
Show reasoning
Structure, activity and host-context outcome are different targets. Geometric agreement can inform a hypothesis, while functional and contextual evidence remains independently necessary.
What does mean squared error twenty-five mean in context B?
Show reasoning
Each of the four assigned errors is −5, so each squared error is 25. Their mean is 25 in the squared units of the supplied outcome scale. It is not a probability or a universal failure rate.
Why might splitting records into two files leave an evaluation dependent?
Show reasoning
The files can still contain repeated records of the same unit or closely related biological contexts. The split must reflect the intended new-unit or new-context question, rather than relying solely on file identity.
What to carry forward
Predictive performance belongs to a defined quantity, evaluation record and domain; interpretation and practical benefit require additional evidence.
References
Resources supporting this lesson’s explanations and examples.
- Highly accurate protein structure prediction with AlphaFold (Nature596:583–589,2021; doi:10.1038/s41586-021-03819-2)John Jumper and colleaguesResearch abstract
- Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure (Ecography 40, 913–929 (2017); doi:10.1111/ecog.02881)David R. Roberts and colleaguesOriginal validation analysis