← ClaudeAtlas

detoxlisted

Hunt down and fix flaky tests. Use when a test suite fails intermittently, CI needs re-runs to go green, or the user wants to know which tests are flaky versus genuinely broken — and why.
tokyubevoxelverse/detox · ★ 1 · AI & Automation · score 74
Install: claude install-skill tokyubevoxelverse/detox
# Detox A flaky test is worse than a missing test: it trains everyone to ignore red. Your job is to separate flaky from broken, diagnose each flake's *root cause class*, and fix the cause — never the symptom. ## Banned fixes Retry wrappers, longer sleeps, and `skip` are not fixes; they are how flakes become permanent residents. The only acceptable retry is around a call to a genuinely external system the test cannot control — and that usually means the test should be mocking it instead. ## Phase 1 — Census Run the full suite repeatedly — default 5 runs, more if runs are cheap — and build a per-test pass/fail matrix. - **Fails every run** → broken, not flaky. Report separately; out of scope unless asked. - **Fails some runs** → flaky. This is the target list, ranked by failure rate. - Record conditions per run: ordering, parallelism, seed, timing — the variance between a passing and failing run often names the culprit by itself. ## Phase 2 — Diagnose each flake Isolate, then classify. Standard experiments: | Experiment | If behavior changes → | |---|---| | Run the test **alone** vs. after the full suite | Shared state / test-order dependence | | Run the suite in **random order** / different parallelism | Order dependence, resource contention (ports, files, DB rows) | | Run with a **fixed seed** / frozen clock | Unseeded randomness, time-of-day or timezone dependence | | Run **repeatedly in a tight loop** | Race conditions, timing assumptions, hardcoded timeouts | | Ru