test-flakiness

# Test Flakiness Detection A flaky test is one that sometimes passes and sometimes fails without any code change. Flaky tests are worse than no tests in some ways — they train the team to ignore red CI runs, masking genuine failures. This skill identifies them, explains likely causes, and recommends whether to quarantine or fix each one. **Output:** Updated `tests/regression-suite.md` quarantine section + optional `production/qa/flakiness-report-[date].md` **When to run:** - Polish phase (tests have had many runs; statistical signal is reliable) - When developers start dismissing CI failures as "probably flaky" - After `/regression-suite` identifies quarantined tests that need diagnosis --- ## 1. Parse Arguments **Modes:** - `/test-flakiness [ci-log-path]` — analyse a specific CI run log file - `/test-flakiness scan` — scan all available CI logs in `.github/` or standard log output directories - `/test-flakiness registry` — read existing regression-suite.md quarantine section and provide remediation guidance for already-known flaky tests - No argument — auto-detect: run `scan` if CI logs are accessible, else `registry` --- ## 2. Locate CI Log Data ### Option A — GitHub Actions (preferred) Check for test result artifacts: ```bash ls -t .github/ 2>/dev/null ls -t test-results/ 2>/dev/null ``` For Godot projects: GdUnit4 outputs XML results compatible with JUnit format. Check `test-results/` for `.xml` files. For Unity projects: game-ci test runner outputs NUni...