Health data quality: measuring it against its intended use

One question comes first: does the data belong to the right patient? Identity matching, national health identifiers and provenance as the basis of trust are covered in our interoperability article. Once identity is secure, six questions describe the quality of a data point. The EMA framework groups them differently, with uniqueness under coherence and traceability under reliability, but all six are there.

Accuracy

Does the data correctly represent the reality it describes? It is checked against an authoritative source and through plausibility checks. An implausible value signals a possible error, not a certain one: it calls for verification, not automatic correction.

Completeness

Is everything the use case needs actually there? A record can be large and still incomplete on the point that matters. Completeness is measured against a set defined in advance: date, unit, status, context and provenance.

Consistency

Does the data follow the same conventions, and does it contradict itself? A medication listed as active in one place and stopped in a discharge letter is a contradiction to resolve, not to hide.

Timeliness

Is the data recent enough for the decision? An old allergy can still matter, while a weight or a kidney function result needs a date that fits the use.

Uniqueness

Do two similar records describe the same event? Not necessarily: the EMA notes that two distinct measurements yielding the same value are not a duplicate. Merging too quickly can erase a real change. Keep the source identifier and the timeline.

Provenance

Who produced the data, when, with what method, from which source? A lab measurement, a patient entry, a value extracted from a document and an algorithmic estimate do not deserve the same confidence, and must stay distinguishable.

A value without context loses its meaning. For a lab result, keep the unit, the reference range, the method when known, the laboratory and the date. For a medication, distinguish prescribed, dispensed, reported as taken and stopped: that is what an accurate medication list depends on. For a connected device, keep the model, the algorithm, the signal quality and the measurement conditions, as with any real-world data.

Above all, never turn a gap into a normal value. “Not recorded”, “not measured” and “negative” are three different states, and the screen should show them as such.

For each use case, a single sheet is enough to set the requirements: the decision it supports, the minimum data set, the preferred source, the freshness window, unit rules, tolerances, how missing data is handled and who validates.

A kidney function alert before a prescription does not have the requirements of an educational chart: the level of control should be proportionate to the risk of the decision. The EMA framework points the same way: acceptance thresholds, a completeness rate for instance, are set with respect to the question being asked.

Be wary, on the other hand, of universal rules such as “every field must be filled in”. They encourage default values and false completeness. An explicitly missing field is better than information made up to pass a check.

A correction should not erase history. Keep the original value, the author, the reason, the date and the new value. Analyses and models must be able to tell which version they used. Standards allow for this: in FHIR, the Provenance resource describes the activity that created, revised or signed a version of a resource, the agents involved and when it happened.

The traceability chain starts at the source and follows every step: extraction, transformation, storage, display, use. When AI extracts information from a document, keep the document, the area the information came from, the model version and its confidence level. A low-confidence extraction should never be displayed as fact: define what a clinician must validate before it enters the clinical summary.

This is what is at stake in a digital twin, which brings together sources that are different by nature. With MyTwin for clinicians, the patient’s digital twin is built from the medical record, test results, imaging, treatments and medical history, then enriched with the data the patient generates between appointments. MyTwin makes recommendations; the clinical decision stays with the clinician, who must be able to trace every piece of information back to its source.

Quality is not a one-off clean-up. A dashboard can track the share of complete critical fields, non-compliant units, conflicting identities, duplicates, corrections, out-of-date data and alerts that could not be computed for lack of data. Break them down by source and by department: that is what brings causes to light. Transport indicators, such as rejected messages or delivery delays, belong to integration quality, covered in our interoperability article.

Every indicator should lead to an action: fixing an interface, training a team, changing a transformation rule, escalating to the vendor or suspending a source. And a good overall rate is not reassuring on its own: a serious error can be concentrated in a rare field and vanish in the average.

Hold a regular review with clinicians, quality, data and security teams, the data protection officer and patient representatives. Prioritize by how severe an error is, how often it occurs and how hard it is to detect.

For a training set, quality includes representativeness, how labels are defined and temporal consistency. A poorly chosen label reproduces the biases of past practice: predicting healthcare costs rather than illness is a well-known example, documented in a study published in Science. Likewise, a variable recorded after the event must not be used to predict it.

Keep development, validation and test sets separate. Document exclusions, transformations and missing data: France’s data protection authority, the CNIL, notes in its guidance on AI development that documenting datasets ensures their traceability, and recommends monitoring data drift in real-world conditions, the gap between training data and the data the model actually meets. A new lab, sensor, coding scheme or population can cause it: post-deployment monitoring is covered in our article on AI in medical imaging, and using such models to prioritize follow-up in the one on patient risk stratification.

The law points the same way. In Europe, the GDPR requires personal data to be accurate and, where necessary, kept up to date, and limited to what is necessary for the purposes it is processed for. Collecting more is therefore not a neutral reflex: every variable should serve an explicit purpose and be protected according to its sensitivity.

1. Choose one precise clinical use case.
2. Map the sources and the transformations.
3. Define the critical fields and their rules.
4. Check identity, dates, units and provenance.
5. Quantify missing data and contradictions.
6. Organize human validation and correction.
7. Track indicators by source and by population.
8. Suspend or recalibrate when quality is no longer sufficient.

Quality is not a clean-up done once and for all: it is a property of the system and of the care pathway, and it has to be monitored.