test-fidelitylisted

Hunt vacuous / low-fidelity tests — the ones that pass whether or not the code is wrong in the way they claim to catch. Probes four shapes: a double that dodges the real thing, an assertion too loose to constrain behavior, inputs that never reach the claim, and coverage that is claimed but not guaranteed. Uses mache/modmap for value-origin and branch-reachability evidence and dispatches an adversarial agent for judgment. Use on a test-bearing PR or diff before review, or when a suite "passes" but you don't trust it.
jamestexas/agents · ★ 2 · AI & Automation · score 61

Install: claude install-skill jamestexas/agents

# test-fidelity — catch tests that pass whether or not the code is right The spine, one question: > **Would this test still fail if the code were wrong in the way it is supposed to catch?** If the answer is no, the test is theatre: it does not merely miss the bug, it **certifies the buggy path as correct**, and the comment asserting fidelity makes the next reader trust it. If your project keeps a test-double doctrine doc — a catalog of tests that lied and the mitigations that closed them — read it once as the local ground truth; the hunting catalog below is the portable version of it. This skill audits whether **tests constrain the code**. It is not a bug hunt on the code, not a coverage-percentage report, and not a style pass — other skills and agents cover those. A line executed is not a line verified. ## When to use - A test-bearing PR before you review or approve it (yours or a peer's). - After an agent (including a sub-agent) wrote tests — self-reports about test quality are not trustworthy; verify. - When a suite is green but you suspect it is green for the wrong reason. ## The hunting catalog — the shapes to investigate Hunt *shapes of behavior*, not specific names. Each entry is **what they were doing** + the probe. ### Class A — the harness dodges the real thing - *Drove a stand-in that claims to mirror the real implementation, and nothing checks the claim.* → Contract-test the double against the real one through one shared operation table; it must fai