testdoc-flakylisted
Install: claude install-skill vivekbhange-hub/test-doctor-skill
# testdoc-flaky: which tests lie, and why
A flaky test is a test that gives different answers for the same code. CI already holds the
evidence: every run uploads a JUnit XML file. This skill collects several of them, lets
`junit_report.py` do the counting, and then you do the reading: open the failing test, find the
cause, and make the smallest change that removes it.
## Inputs
- JUnit XML from at least 3 runs of the same suite, ideally 5-10, on the same commit or on
commits that did not touch the failing code. One run cannot show flakiness.
- Or access to the repo's CI through `gh` so you can download them.
- Commands below say `python3`; on Windows use `python` (or `py -3`).
## Outputs
- `testdoc/flaky-report.md`: the script's report (flaky, failing, slow, clusters).
- A ranked list for the user: test, flake rate, likely cause with the evidence line, the fix.
- Optionally one small change per flaky test, each in its own commit.
## Steps
1. **Confirm permission.** Only pull CI artifacts and run tests in repositories the user owns or
is authorized to work on. Downloaded artifacts can contain secrets in logs; do not paste them.
2. **Get the runs.** If the user has no JUnit files yet, check which tool writes them (pytest
`--junitxml=junit.xml`, jest-junit, Maven Surefire `target/surefire-reports`, Go
`gotestsum --junitfile junit.xml`) and that the workflow uploads them with
`actions/upload-artifact`. Then download recent runs of one workflow:
```
gh