deid-reid-harnesslisted
Install: claude install-skill Hefrock/agent-skills
# De-id ⟷ Re-id Adversarial Eval Harness
> 📊 **[See RESULTS.md](./RESULTS.md)** for a verified sample run — the
> privacy-utility frontier, the cross-track headline (49/50 records vulnerable
> despite a perfect Safe Harbor scrub), and every figure with a 95% CI — no need
> to run anything.
## The core idea
A de-identifier's real metric is **not its recall** — it is its **survival against the
best available re-identification attacker**. So the attacker *is* the eval. Report a
scrubber's score only as its survival against a defined attack, never as a standalone
"accuracy."
The attacker is really **three different attackers**, corresponding to three threat
models with three different standards. Never collapse them into one number:
1. **Direct-identifier leakage** → HIPAA **Safe Harbor** (a checklist: remove 18
identifier categories). Deterministically scorable because ground truth is known.
2. **Quasi-identifier linkage** → **Expert Determination** (a statistical argument that
re-id risk is very small). Needs a background population as denominator.
3. **Free-text inference** → the pure AI-era threat (a model *derives* identity from
context that never states it). Needs an LLM judge; the only track where ground truth
can't be enumerated.
A Safe Harbor pass can still be an Expert Determination failure. That divergence is the
whole point of the harness — surface it, don't hide it.
## What this slice builds (Tracks 1 and 2)
**Track 1 (Safe Harbor leakage)** is im