← ClaudeAtlas

flaky-smoke-testslisted

Diagnose intermittent VS Code Electron smoke-test failures from the Azure DevOps Flaky Smoke Tests pipeline (definition 700). Covers finding failed iterations, downloading task logs and platform artifacts with Azure CLI, correlating cumulative runner logs, tracing the introducing commit, and queueing focused validation runs.
chapmanjw/clawdius · ★ 26 · Code & Development · score 81
Install: claude install-skill chapmanjw/clawdius
# Diagnosing Flaky Smoke Tests Use this skill for failures from the Azure DevOps **Flaky Smoke Tests** pipeline: - Organization: `https://dev.azure.com/monacotools` - Project: `Monaco` - Definition ID: `700` - Pipeline YAML: `build/azure-pipelines/product-smoke-flaky.yml` - Scheduled runs: twice daily on `main` The pipeline builds VS Code from source and runs the complete Electron smoke suite once per entry in the `iterations` parameter. Each platform runs its iterations sequentially in one job. ## Important Pipeline Behavior - Each iteration is a separate timeline task named `Smoke test iteration <N>/<total> (Electron)`. - Iteration tasks use `continueOnError: true`, so a failed iteration normally has result `succeededWithIssues` and later iterations still run. - Do not infer smoke-test health from the overall build or job result. Inspect every iteration task. - Logs are published once per platform job from the shared `.build/logs` directory. The artifact is job-scoped, not iteration-scoped: `smoke-test-runner.log` can contain multiple iterations, and suite directories are not separated by iteration. - The exact iteration task log is authoritative for its failure summary and time range. Correlate that range with the job-level artifact. - Iterations can see persisted smoke-test state from earlier iterations, such as historical session rows. Never assume the newest-looking row or a matching prompt belongs to the active conversation without verifying the