← ClaudeAtlas

deid-reid-harnesslisted

Build and run an adversarial de-identification ⟷ re-identification evaluation harness for clinical text: generate synthetic clinical notes with ground-truth identifier spans, run a de-id pipeline (defender) over them, and score leakage against an attacker. Use this whenever the task involves evaluating a PHI scrubber or de-id pipeline, measuring re-identification risk, testing HIPAA Safe Harbor coverage or Expert Determination risk, building a privacy-utility frontier for clinical NLP, or comparing de-identification approaches — even if the user only says "test my de-identifier," "how leaky is this scrubber," or "de-id eval." Prefer this over an ad-hoc script whenever synthetic clinical data, PHI, or de-identification evaluation is mentioned.
Hefrock/agent-skills · ★ 0 · AI & Automation · score 68
Install: claude install-skill Hefrock/agent-skills
# De-id ⟷ Re-id Adversarial Eval Harness > 📊 **[See RESULTS.md](./RESULTS.md)** for a verified sample run — the > privacy-utility frontier, the cross-track headline (49/50 records vulnerable > despite a perfect Safe Harbor scrub), and every figure with a 95% CI — no need > to run anything. ## The core idea A de-identifier's real metric is **not its recall** — it is its **survival against the best available re-identification attacker**. So the attacker *is* the eval. Report a scrubber's score only as its survival against a defined attack, never as a standalone "accuracy." The attacker is really **three different attackers**, corresponding to three threat models with three different standards. Never collapse them into one number: 1. **Direct-identifier leakage** → HIPAA **Safe Harbor** (a checklist: remove 18 identifier categories). Deterministically scorable because ground truth is known. 2. **Quasi-identifier linkage** → **Expert Determination** (a statistical argument that re-id risk is very small). Needs a background population as denominator. 3. **Free-text inference** → the pure AI-era threat (a model *derives* identity from context that never states it). Needs an LLM judge; the only track where ground truth can't be enumerated. A Safe Harbor pass can still be an Expert Determination failure. That divergence is the whole point of the harness — surface it, don't hide it. ## What this slice builds (Tracks 1 and 2) **Track 1 (Safe Harbor leakage)** is im