← ClaudeAtlas

ci-failure-triagelisted

Decides what kind of failure a red CI test is before anyone starts fixing it. Extracts seven failure signals from the runner output, stack trace, run history, and environment metadata, then walks an ordered first-match-wins rule set to exactly one verdict per red test: flaky-known, environment-drift, defect, timeout, flaky-pre-incident, or flake-of-unknown-cause, or undetermined when a lone log with no run history cannot support any verdict. Emits the verdict together with the alternatives that were rejected and the specific condition each one failed, so the triage decision is auditable rather than asserted. Use when a test has just gone red and the next action depends on whether the cause is a product defect, a non-deterministic test, or drifted infrastructure.
testland/qa · ★ 13 · Testing & QA · score 80
Install: claude install-skill testland/qa
# CI failure triage One test is red. Before anyone opens a ticket, reruns the job, or edits a line, someone has to answer a prior question: what kind of failure is this? This is the method for answering it, and for writing the answer down in a form a second person can check. The cost of getting it wrong is measurable. Google reports that about 84% of the pass-to-fail transitions its CI observes involve a flaky test, and that almost 16% of its tests show some level of flakiness (Google 2016). A triage step that treats every red as a defect files mostly noise; one that treats every red as a flake ships regressions. ## What this owns, and what it does not This owns **classification**: turning one failed run into one verdict, with the rejected alternatives recorded. It deliberately does not own: - **Report formatting.** A bug-report schema (summary, environment, steps to reproduce, expected, actual, severity, priority, reproducibility) is a separate job that starts *after* you already know the failure is a defect. This decides which kind of failure you are looking at; that fills in the form once you know. - **Remediation.** No fix hypothesis, no patch, no rerun, no quarantine action. The verdict names the next step; it does not take it. - **Suite-wide prediction.** One failure at a time. Screening a whole green suite for tests likely to flake next is a different technique with different inputs. - **Retrieval.** How you obtain the log, the run history, or the e