Biology · Research Design, Data & Scientific Practice

Data provenance, metadata, visualisation and honest reporting

Six invented records contain values four, six, eight, zero, one missing value and a fifty flagged by a pre-specified quality rule. The four eligible observed values total eighteen. A zero records an outcome; the missing value does not.

Life, Evidence & Biological Scale introduces evidence and biological organisation. Organisms, Environments & Ecological Niches and Population Ecology & Life-History Patterns develop the importance of boundaries and denominators. Their English chapters are available in this working manuscript; their publication remains pending. The examples here require arithmetic, averages and ordinary graph reading. The sample-variance and confidence-interval example explains its notation as it is introduced; it can be revisited after the simpler comparisons.

Back to the Biology map

By the end, you can

Preserve original observations, distinguish zero from missingness and exclusions, and trace each reported value through documented transformations.

Keep zero, missing and excluded records distinct

A raw record should retain what was observed and the context needed to interpret it. In the original six-row table, zero is a valid observation. One entry is missing. A value fifty remains in the raw file but is excluded from the primary summary because its quality flag meets a rule specified before outcomes were known. The exclusion is not justified merely because fifty looks unusual.

The eligible observed values are four, six, eight and zero. Their total is eighteen, their count four and their mean 4.5. Substituting zero for the missing entry adds no amount but changes the denominator to five, producing 3.6. It changes the analysis's meaning. Missingness might depend on the outcome or other conditions; this table alone does not establish why the value is absent.

Exclusions and corrections should remain visible. An investigator can report a justified primary summary and a sensitivity analysis, each with its rule, rather than overwriting the record until one preferred average remains. A value observed outside a quality criterion, an absent observation and an actual zero carry different information.

A zero is an observation; missingness is different

A missing entry has no observed value. The original fifty and its quality flag remain part of the raw record even when it is excluded from a named summary.

Read the diagram in words

Four, six, eight and an observed zero total eighteen and average 4.5. One separate record is missing and the fifty is flagged under a pre-specified quality criterion. Treating the missing record as zero gives eighteen/five = 3.6; including all nonmissing values gives sixty-eight/five = 13.6.

Retained raw record
Retained raw record	Original observed value	Meets the pre-specified quality criterion	Value not observed
Record one	4	Yes	No
Record two	6	Yes	No
Record three	8	Yes	No
Record four	0	Yes	No
Record five	?	Yes	Yes
Record six	50	No	No

Primary eligible observed-value summary
Named denominator count	Total of the included values	Mean under that rule
4	18	4.5

Missing value assumed zero
Named denominator count	Total of the included values	Mean under that rule
5	18	3.6

All nonmissing values, including the flagged value
Named denominator count	Total of the included values	Mean under that rule
5	68	13.6

Six invented raw records. A pre-specified quality criterion flags record six independently of its numerical outcome. Missingness mechanism is unspecified. Alternative calculations illustrate assumptions only, with no imputation recommendation or actual data-cleaning procedure.

A number needs an interpretable history

Metadata describe the records: what units were measured, what each field means, how identities connect repeated measurements, the observation time and the analysis context. Provenance records where a value came from and the transformations that produced the reported result. A table labelled result without this context is difficult to evaluate or reuse.

The original derivation graph starts with the retained raw observations and their definitions. A separate eligibility step applies the pre-specified quality rule and retains observed values. A summary then uses the eligible count and sum to obtain 4.5; a figure displays that named result. Each transformation has its own record, and no arrow replaces the original file.

Data can be easier to find and reuse while still requiring controlled access. FAIR principles concern findability, accessibility, interoperability and reuse, including identifiers, metadata, provenance and appropriate access procedures. They do not establish that the data are correct or that identifiable, sensitive records should be made public without restrictions.

The reported mean has a traceable derivation

Directional relations distinguish input, rule, interpretation and derived output. Clear provenance supports checking rather than proving correctness.

Read the diagram in words

Original observations and metadata feed the documented eligibility step. Four eligible values then total eighteen, giving mean 4.5. The figure refers to that named calculation. Every input remains available to explain the derivation; the graph is not a biological causal pathway.

Record or derivation stage
Record or derivation stage
Original six records retained
Field definitions, identities and units
Pre-specified quality and observed-value rule
Four eligible observed values
Total eighteen divided by four
Named mean shown in the figure

Documented provenance relation
Record or derivation stage →	→ Record or derivation stage	Documented provenance relation
Original six records retained	Four eligible observed values	Original input retained
Field definitions, identities and units	Four eligible observed values	Definitions needed to read the input
Pre-specified quality and observed-value rule	Four eligible observed values	Rule controlling inclusion
Four eligible observed values	Total eighteen divided by four	Calculated from the eligible inputs
Total eighteen divided by four	Named mean shown in the figure	Display refers to the named calculation

Calculated from the eligible inputs
Record or derivation stage	Assigned response units
Total eighteen divided by four	4.5
Named mean shown in the figure	4.5

Original conceptual provenance chain for the six-row teaching record. The nodes represent records or documented processing stages, not biological mechanisms, physical movement or an actual software execution. Raw data remain retained.

Make the displayed comparison answer the stated question

A figure should identify its measure, units, groups and denominator. A point can represent an individual observation, a group mean or a model estimate; those meanings should not be left to guesswork. When observations are nested, the display should retain their group identities rather than imply that every dot is an independent allocation.

The six-row integrity example is best understood with both its original records and its summary. The valid zero is visible, the missing entry has no invented height, and the excluded fifty retains its flag. A graph that places all three at zero would erase exactly the distinction the reader needs.

Honest reporting includes outcomes that do not fit the expected pattern, the specified analysis, relevant deviations, contributions and conflicts that could affect interpretation. An attractive graph or a precise-looking decimal cannot substitute for that record. When an error is found, preserve enough history to explain the correction and what conclusions it changes.

Worked example

Recompute the named summary

Use the four eligible observed values 4 + 6 + 8 + 0 = 18. Divide by four to obtain 4.5. The fifty is still recorded but its pre-specified quality flag excludes it from this summary; the missing value is absent from the observed-value denominator.

If the missing entry is silently recoded as zero, the total remains eighteen but the count becomes five. Eighteen divided by five is 3.6, a difference of -0.9 from the primary mean. This is an analysis change, not an additional observed low value.

Including every nonmissing raw value would give total 68 over five values, mean 13.6. That sensitivity comparison can be reported with its different rule. It does not establish that either rule is appropriate without examining the stated quality criterion and the question.

Check your understanding

Why does replacing the missing value with zero change the mean when the sum stays eighteen?
Show reasoning
It changes the denominator from four observed eligible values to five entries treated as observed. The resulting 3.6 represents a different assumption, not a recorded zero.
Should the excluded value fifty be erased from the raw file?
Show reasoning
No. Keep the original value, its quality flag and the exclusion rule. The primary analysis can use the justified eligible set while retaining a traceable record and separately named sensitivity comparison.
Does making data FAIR guarantee that their biological interpretation is valid?
Show reasoning
No. Findability, access, interoperability and reuse support evaluation; they do not certify measurement quality, design assumptions or conclusions. Appropriate restrictions can also remain for sensitive records.

What to carry forward

Keep the original records and their meaning intact; name every denominator, exclusion and transformation used to produce the display.

References

Resources supporting this lesson’s explanations and examples.