From molecular mechanisms to validated applications

A mechanism and an application require different demonstrations

A molecule can engage a target, change a pathway and alter a measured endpoint. Each step supports a different claim. A biomarker or mechanistic explanation alone does not establish benefit for an intended application or patient population.

Validation concerns a specified claim, method, context and range. A successful comparison in one setting does not make a method or intervention universally valid. Independent evidence and testing of competing explanations are needed as a claim broadens.

Truth-conditioned accuracy and comparator agreement are different accounts

Choose a synthetic binary target-presence classifier with established reference labels: two hundred present cases and six hundred absent. Assign sensitivity 0.910 and specificity 0.940 as exact model rates. Counts are true positives 182, false negatives 18, true negatives 564 and false positives 36. No patient or actual diagnostic test is involved.

Positive predictive value is 182/(182+36)=0.834862385; negative predictive value is 564/(564+18)=0.969072165. Accuracy is (182+564)/800=0.9325. Each denominator answers a different question: true condition for sensitivity/specificity, returned call for predictive values and all cases for accuracy.

Transfer the same assigned conditional rates to a fifty-thousand-case model population with known prior target prevalence 0.0200. Expected present/absent counts are 1,000/49,000; expected true positives 910 and false positives 2,940. Positive predictive value falls to 910/(910+2,940)=0.236363636, even though sensitivity and specificity have not changed. These are assigned model rates, not uncertain clinical performance estimates.

In a separate two-method table without reference truth, assign both positive 84, new-positive/comparator-negative 9, new-negative/comparator-positive 15 and both negative 92. Overall agreement is 176/200=0.880. Conditioning on comparator positives gives positive agreement 84/99=0.848484848; conditioning on its negatives gives negative agreement 92/101=0.910891089.

Reverse which method is comparator: positive/negative agreements become 84/93=0.903225806 and 92/107=0.859813084. Overall agreement remains 0.880. Changing the denominator does not create reference truth, sensitivity or specificity; methods can agree while both are wrong.

A classification needs a defined reference and population

For a binary classifier, sensitivity is the positive-call fraction among genuinely positive cases under the chosen reference; specificity is the negative-call fraction among genuinely negative cases. Both require a defined benchmark and intended population.

The fraction of positive calls that are truly positive also depends on the base rate. If a comparison method does not establish truth, the table measures agreement with that method. Similar arithmetic then has a different meaning, and agreement can depend on which method provides the denominator.

Pass fractions and repeated readings do not create missing evidence

For an independent invented evidence funnel, start with 360 candidate entities. Specify successive conditional pass fractions 0.600, 0.450, 0.400, 0.750 for defined affinity evidence, target engagement in the chosen cellular model, a relevant model phenotype with controls and independent model/readout confirmation. Each fraction applies to the set passing the preceding stage.

Expected stage counts are 360→216→97.2→38.88→29.16, with combined pass probability 0.0810. Noninteger counts are expectations rather than realized fractional candidates. Increasing only cellular-engagement fraction from 0.450 to 0.750 gives expected final count 48.6. These arithmetic filters supply no unobserved clinical outcome, safety or treatment-effect evidence.

For a separate study-unit model, specify four preparations independently assigned and manipulated, each giving three observational readouts in AU: (7, 9, 8), (11, 13, 12), (15, 17, 16), (19, 21, 20). The experimental unit for the stated mean-preparation question is the preparation. Its means are 8, 12, 16, 20 AU, with overall mean 14 AU and independent-unit count N=4; there are twelve observational readings.

The sample standard deviation of preparation means is s=√[Σ(x_i−14)²/(N−1)]=√(80/3)=5.16397779 AU. The preparation-mean standard error is SE=s/√N=2.58198890 AU under this independent-unit model. Treating all twelve observations as independent instead gives the inappropriate smaller SE 1.37068883 AU.

Duplicating every reading within its preparation produces twenty-four observational values while retaining exactly the same preparation means, N=4 and preparation-level SE 2.58198890 AU. No new independent unit is created. A real design needs its actual assignment and dependence evidence; this model does not establish a clinical effect, confidence-interval coverage or a universal rule for every culture, well or sample.

Repeated readings and independent units are different

A set of readings can be nested within a preparation or other independently assigned experimental unit. Repeating a measurement can characterize reading variability without creating additional independently assigned units. State which level supports the proposed comparison.

Random assignment, blinding, predefined endpoints, appropriate controls and reporting uncertainty help address different sources of bias. A larger number of repeated observations does not automatically remove bias or establish that an effect will transfer to a new context.

Report what the evidence actually establishes

A sequence of evidence filters can illustrate how a candidate set narrows, provided each fraction is defined conditionally on the preceding set. Multiplying those fractions is a bookkeeping model when its assumptions hold. It is not a forecast of clinical success from a mechanism alone.

A useful report names the material, endpoint, population or model, comparator, experimental units, uncertainty and unresolved alternatives. Separate measured findings from model assumptions and predictions. That clarity makes chemical evidence easier to use without extending its claim beyond the tested setting.

Worked examples

Try it yourself

1. Use the same specified sensitivity/specificity in fifty thousand cases with known prior target prevalence 2.00%. Find expected true/false positives and positive predictive value.

   Show the answer and reasoning

   Expected TP 910 and FP 2,940; positive predictive value 0.236363636.

   There are 1,000 present and 49,000 absent cases. Different base rate changes predictive value without changing the assigned conditional rates.

2. Reverse the comparator in the imperfect-reference table. Find positive/negative/overall agreement and whether truth is established.

   Show the answer and reasoning

   Positive 0.903225806; negative 0.859813084; overall 0.880. Truth remains unestablished.

   Condition on the other method’s positive 93 and negative 107 totals; symmetric total agreement stays 176/200.

3. Increase only the funnel’s cellular-engagement conditional fraction from 0.450 to 0.750. Find final expected count and whether clinical benefit follows.

   Show the answer and reasoning

   Expected final 48.6; clinical benefit does not follow.

   360×0.600×0.750×0.400×0.750=48.6, while clinical outcomes remain unobserved.

4. Duplicate each observational value within every preparation. Find observation count, independent N and preparation-level SE.

   Show the answer and reasoning

   Twenty-four readings; N=4 independent preparations; SE=2.58198890 AU unchanged.

   Within-preparation duplication preserves all preparation means and creates no new independently assigned unit.

What to carry forward

• Sensitivity and specificity require a suitable known-reference definition.
• Predictive values depend on the population’s base rate.
• Agreement with an imperfect comparator does not establish reference truth.
• Additional observations within a preparation need not create independent experimental units.