← ClaudeAtlas

automation-scorelisted

Score QAIA-generated Playwright test code on two separate tracks - a deterministic static pass plus a mutation pass that inverts each assertion and requires it to go red, then a 6-dimension LLM rubric for what no machine can see. Read-only over code - it judges, it never edits. Use to review generated automation before it is trusted, or to check whether a suite's assertions are load-bearing at all.
QAIA-Project/QAIA · ★ 0 · Testing & QA · score 64
Install: claude install-skill QAIA-Project/QAIA
# automation-score — the judge of generated test code Applies to the Playwright code produced by `automate` (and its siblings `a11y-audit`, `contract-probe`, `security-surface`) for **one** test book. It **scores only** — see the scoring-only guardrails in `../README.md`. **Why this skill exists.** Generated code was, for most of this project's life, the only QAIA output reviewed solely by its own producer — `automate`'s self-review step — in direct exception to rule 3, *a producer plugin never grades its own output*. The two defects a campaign found in generated code (a missing blocking assertion; page-objects-as-fixtures bypassed) are exactly the class of defect no judge was looking for. This skill is that judge. ## The three tracks, and why they are never summed | Track | Answers | Can it be automated? | |---|---|---| | **Static** | Is the code shaped the way the rules require? | Fully — counts and patterns | | **Mutation** | Is each assertion *load-bearing* — can it fail at all? | Fully — needs a runnable suite | | **LLM rubric** | Does it assert **the right thing**? | No — this is the judgment call | **Never add these numbers together.** A perfect static score does not make the code faithful; a high rubric score does not cancel a blocking mutation survivor. Same separation, same reason, as structural score versus rubric on the test book side: one measures form, the other substance, and averaging them hides whichever is worse. ## Prerequisites - The generated test