Garbage in, garbage out: data quality in health research

Somewhere in almost every hospital database there is a patient with a body temperature of 983 degrees, a newborn weighing forty kilograms, and a man born on January 1, 1900, who is somehow still attending appointments. A dropped decimal here, pounds in a kilogram field there, a stand-in birth date that means "unknown" to the registration clerk and means "126 years old" to a query.

None of these values were typed by liars. They were typed by busy people into systems designed for billing. The phrase garbage in, garbage out is usually said with a shrug. In health research it should be said with a checklist.

Why health data goes wrong in the first place

Electronic health record data is the exhaust of care and billing, not an instrument of science. Diagnosis codes are chosen with reimbursement in mind. Templates and copy-paste propagate yesterday's note into today's. Default values fill fields nobody had time to measure. A patient's condition stops appearing in the data not when it resolves but when the clinician stops re-coding it.

Worse than the random errors are the systematic ones. Data exists only when patients show up, so sicker patients have denser records — a distortion sometimes called informed presence. A missing lab result is not a normal lab result; it usually means nobody ordered the test, which is itself clinical information. Research that reads routine-care data naively inherits every one of these distortions as bias. That inheritance is the central honest caveat in all real-world evidence work: the data is plentiful and genuinely useful, and it was never collected to answer your question.

A shared vocabulary: the Kahn framework

For years, every research group described data problems in its own dialect — "dirty," "sparse," "weird." In 2016, Michael Kahn and colleagues published a harmonized framework that gave the field a common vocabulary, organized around three categories.

Each category can be assessed in two modes: verification, checking data against its own internal rules and expectations, and validation, checking it against external truth — a registry, a gold-standard chart review, known population distributions. The framework's quiet achievement is that it makes quality discussable: two sites can disagree about a dataset using the same nouns.

Checks you can run, not vibes

Quality talk becomes engineering the moment it compiles into executable checks. The OHDSI community's Data Quality Dashboard runs thousands of automated checks against any dataset expressed in the OMOP common data model — each check a Kahn-style question turned into a test with a pass-fail threshold. Is every gender concept a valid concept? Does any drug exposure end before it starts? What fraction of measurements carry plausible units?

The output is the point: data quality stops being a feeling and becomes a report — a percentage of checks passed, comparable across sites, versions, and time. A research network can require a quality threshold before a site joins a study. A reviewer can ask for the dashboard output the way they ask for a consort diagram.

None of this works without standards, because identical checks require identically shaped data — which is much of why common data models earn their keep. The registry world reached the same conclusion years earlier: AHRQ's long-running registry guidance treats explicit data standards and quality assurance as the difference between a registry and a pile. And everything downstream depends on it — a computable phenotype is only as good as the fields it reads.

Provenance: every value should know where it came from

When records are aggregated from multiple sources — several providers, a lab network, patient-reported entries — conflicts are not edge cases; they are Tuesday. Two different weights recorded the same day. A medication active in one system and discontinued in another. An allergy present in the specialist's record and absent from primary care.

Without provenance, conflicts get resolved by accident: last write wins, or whichever source loaded second. With provenance per field — source system, timestamp, author, transformation path — a conflict becomes information. The 70-kilogram value came from a clinic scale; the 154 value came from a patient-entered form in a system that displays pounds. Now the discrepancy explains itself. The standards world has the machinery for this: HL7's FHIR includes a Provenance resource designed to carry exactly this lineage. Provenance also keeps corrections honest — you amend forward, leaving the trail intact, rather than silently overwriting history.

Quality is a property you can audit

A quality claim worth anything is reproducible: these checks, at these versions, ran against this dataset version on this date, and here are the results. "Research-grade data" should decompose, on demand, into exactly that — not a vibe, not a brochure adjective, but an artifact someone else can re-run.

That is the standard Mithras Health builds against. The platform keeps FHIR R4 with US Core 6.1.0 as the system of record — pinned versions, so conformance is checked against a fixed target rather than a moving one — and projects OMOP views for research, where the community's standard quality tooling applies directly. Values carry provenance, and every transformation lands on an append-only, cryptographically anchored audit trail, so the history of the data is as inspectable as the data.

Garbage in, garbage out is usually quoted as fatalism, as if input quality were weather. It is not weather. It is a property you can measure, version, and decline to release without. The datasets that move medicine will not be the biggest ones. They will be the ones that can show their work.