← ClaudeAtlas

diagnosing-flakeslisted

Use when a test or CI job fails probabilistically — passes on rerun, fails on one platform only, fails under load, or fails in CI but not locally. Also use when asked to investigate a flaky test, a red run nobody can reproduce, or an intermittent timeout. DO NOT invoke to design a new test for isolation — route that to ledger:designing-concurrent-tests.
qbs784/ledger · ★ 0 · AI & Automation · score 72
Install: claude install-skill qbs784/ledger
# Diagnosing flakes Use this workflow when the task is to investigate an existing probabilistic test or CI failure. Preserve the requested read/write scope: **diagnosis does not authorize a fix, a workflow rerun, or a configuration change.** Ask before crossing into any of those. ## Freeze the evidence Record the repository, workflow, job, commit SHA, runner labels, timestamps, the exact failing test or command, and the first stable failure signature. Keep infrastructure messages separate from test output. Compare multiple failing and passing runs. Prefer runs of the same SHA; when that is impossible, verify that the relevant test and CI configuration are identical across the compared commits. **One passing rerun does not prove an infrastructure fault, and one timeout does not prove a product race.** Use the CI provider's logs and run metadata to establish whether failures overlap on one host or resource namespace. Preserve links to the supporting runs rather than pasting large logs. ## Classify the failure Classify from recorded evidence, not from the eventual fix. The third column is what separates a class from its neighbours; a class assigned without that evidence is a guess. | Class | What it looks like | What discriminates it | |---|---|---| | Host-resource collision | the failure needs something else running at the same time | independent processes or jobs acquire the same port, socket, database, predictable path, cache, or external namespace | | Incomplete life