Longitudinal Outcome Tracking: Ethics, Consent, and Method

Most program evaluation stops at the moment it's most convenient to stop: at exit, when the beneficiary is still enrolled, still reachable, still inside the relationship that got them to agree to be measured in the first place. What happened to them a year later, or three, is treated as someone else's research question — usually because tracking it well is genuinely hard, not because anyone doubts it matters. We think the difficulty is worth naming precisely, because the three hardest parts of longitudinal outcome tracking are not technical, and treating them as if they were is how well-intentioned tracking systems turn into something closer to surveillance.\n\n## Consent has a shelf life, and most systems ignore it\n\nA beneficiary who consents to follow-up at program exit is consenting to something they can reasonably imagine: a survey call in six months, maybe a year. They are not meaningfully consenting to being in a dataset that a different program team queries four years later for a different purpose, or that gets merged with an unrelated dataset to check for correlation with an outcome nobody mentioned at intake. Yet this is exactly what a permanent "beneficiary ID" as a primary key invites, because the key persists long after the consent that attached to it has expired in every meaningful sense.\n\nThe fix we implement is to treat consent as a scoped, expiring grant attached to specific uses, not a one-time checkbox attached to a person. A beneficiary consents to follow-up at 6, 12, and 24 months, for stated purposes, and the system enforces that scope — it does not merely record that consent was once given and leave enforcement to policy. Re-consent is a deliberate touchpoint, not a formality, and when consent lapses or is withdrawn, the record becomes unreachable for follow-up even though it may remain in aggregate, de-identified analysis. This is more expensive to build than a permanent ID and a "consented: true" flag. It is the difference between a follow-up system and a surveillance system, and the expense is the whole point.\n\n## Re-identification risk grows with every wave, not just the last one\n\nA single wave of outcome data, de-identified, is usually genuinely hard to re-identify. The fifth wave, joined against the first four plus a public dataset — a census tract, a school enrollment list, a news article about the program — often isn't. This compounding is the part most organizations underestimate, because each individual wave looks safe in isolation and nobody re-runs the re-identification analysis after every new join.\n\nWe treat this as a property to monitor continuously, not a one-time review. Every new data source that gets joined to the longitudinal record — even something as ordinary as a district-level economic indicator — triggers a re-identification risk pass against the combined dataset, not just the new field. K-anonymity thresholds are enforced at the query layer, not just at export, so an analyst pulling a small enough slice of a small enough cohort gets blocked before the number of people in the answer set drops low enough to be identifying by elimination.\n\n## Attrition isn't missing data. It's a signal.\n\nThe beneficiaries who are hardest to reach for a 24-month follow-up are rarely a random subset of everyone who enrolled. They've often moved because of instability the program was meant to address, or disengaged because the intervention didn't work for them — which means the follow-up sample that succeeds in reaching people is systematically biased toward outcomes that look better than the true population effect. Reporting the outcome rate among respondents, without accounting for who didn't respond, is not a rounding error. It can flip the sign of a program's real effect.\n\nThe method we use treats non-response as data in its own right: tracking attrition by baseline characteristics, modeling the likely direction of bias rather than assuming it away, and reporting a range that accounts for plausible outcomes among non-respondents rather than a single point estimate that quietly assumes they look like everyone else. It is less satisfying than a clean percentage in a board deck. It is also the difference between a number that's defensible under scrutiny and one that collapses the first time someone asks about the beneficiaries you couldn't reach.\n\n## What this demands of the underlying system\n\nNone of these three problems is solved by better statistics layered on top of an existing dataset. They have to be designed into the data model from the first wave: consent as a scoped and expiring grant, re-identification risk as a property checked at every join, and non-response as a tracked, modeled variable rather than a silent gap. Organizations that build longitudinal tracking without this scaffolding usually get away with it for a wave or two. The risk compounds quietly until an external researcher, a journalist, or a beneficiary themselves finds the seam — and by then it's a governance failure, not a data one.