← ClaudeAtlas

proving-a-bugfix-pr-red-to-greenlisted

Produce controlled A/B (red to green) proof that a bug-fix PR actually fixes the bug: hold the new regression test constant and swap only the production code between the PR base and the fix; optionally capture before/after UI screenshots. Works on any repo. Triggers on: A/B evidence, red/green proof, prove the fix, get evidence of the fix, before/after for this PR, does the test actually catch the bug, screenshot the bug and the fix.
sblattj/agent-chandlery · ★ 0 · AI & Automation · score 72
Install: claude install-skill sblattj/agent-chandlery
## More triggers pr-ab-evidence (old name) # PR A/B evidence (red → green) ## The core - **B leg:** run the PR's regression test on the fix branch, expect PASS. - **A leg:** same test file, only the production fix removed, expect FAIL on an assertion (a compile error is weaker proof). - Fix edits a file (`git diff --name-status <base> <branch>` says `M`): `git checkout origin/<base> -- <prod file>`. Fix adds a type the test imports (`A`): revert only the hunk that wires it in, so it still compiles. - Restore after the A leg (`git status --short` clean) and never commit the revert. Report each leg's command and the 1-5 line A-leg diff. ## Build gotchas (Maven backend + pnpm/Vitest frontend) - `-am` with a `test` goal can hit **MDEP-98** ("Artifact has not been packaged yet"): install the dep chain first (`mvn -q -pl <module> -am install -DskipTests`, generated API specs before that), then run the test WITHOUT `-am`: `mvn -q -pl <module> -Dtest=<Class> -Dsurefire.failIfNoSpecifiedTests=false test`. - Real counts live in `target/surefire-reports/*.xml` (`tests=/failures=/errors=`); a `@Nested` container shows 0 in the `.txt`. - Local `mvn spotless:apply` is not a trustworthy oracle: a local formatter/config mismatch once reformatted 177 files. - A fresh worktree needs `pnpm install --frozen-lockfile` plus the repo's codegen (`pnpm openapi`) before Vitest resolves generated clients. One file: `pnpm exec vitest run "<path>"`. - `pnpm install` can scribble a stray `allowBuild