flaky-smoke-tests

Solid

Diagnose intermittent VS Code Electron smoke-test failures from the Azure DevOps Flaky Smoke Tests pipeline (definition 700). Covers finding failed iterations, downloading task logs and platform artifacts with Azure CLI, correlating cumulative runner logs, tracing the introducing commit, and queueing focused validation runs.

Code & Development 8 stars 0 forks Updated today MIT

Install

View on GitHub

Quality Score: 78/100

Stars 20%
32
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Diagnosing Flaky Smoke Tests Use this skill for failures from the Azure DevOps **Flaky Smoke Tests** pipeline: - Organization: `https://dev.azure.com/monacotools` - Project: `Monaco` - Definition ID: `700` - Pipeline YAML: `build/azure-pipelines/product-smoke-flaky.yml` - Scheduled runs: twice daily on `main` The pipeline builds VS Code from source and runs the complete Electron smoke suite once per entry in the `iterations` parameter. Each platform runs its iterations sequentially in one job. ## Important Pipeline Behavior - Each iteration is a separate timeline task named `Smoke test iteration <N>/<total> (Electron)`. - Iteration tasks use `continueOnError: true`, so a failed iteration normally has result `succeededWithIssues` and later iterations still run. - Do not infer smoke-test health from the overall build or job result. Inspect every iteration task. - Logs are published once per platform job from the shared `.build/logs` directory. The artifact is job-scoped, not iteration-scoped: `smoke-test-runner.log` can contain multiple iterations, and suite directories are not separated by iteration. - The exact iteration task log is authoritative for its failure summary and time range. Correlate that range with the job-level artifact. - Iterations can see persisted smoke-test state from earlier iterations, such as historical session rows. Never assume the newest-looking row or a matching prompt belongs to the active conversation without verifying the ...

Details

Author
chapmanjw
Repository
chapmanjw/clawdius
Created
1 months ago
Last Updated
today
Language
TypeScript
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category