holdout-validationlisted

Cross-reference agent self-review claims and evidence-bundle entries against actual file state using hidden holdout scenarios, producing P1/P2/P3 findings mapped to visible acceptance criteria only. Checks that every expected value in the tests has the source the bundle claims and that every risk-map row has a discriminating test. Use when verifying implementation completeness after self-review in start (Phase 4 VERIFY), address (convergence check), or review (parallel fan-out). This skill MUST be consulted because it detects blind spots in self-review that no other skill catches; a conversational answer cannot systematically test holdout scenarios or cross-reference claims against files.
synaptiai/synapti-marketplace · ★ 6 · AI & Automation · score 74

Install: claude install-skill synaptiai/synapti-marketplace

# Holdout Validation ## Contract Iron law: the holdout set stays hidden; scenario IDs only, never names or descriptions. Invoked by `/flow:start` Phase 4 VERIFY step 4, `/flow:address` Phase 4 step 3, and `/flow:review` Phase 3 fan-out (twice in Path A, once in Path B) with self-review findings, an evidence bundle draft, and a file list. Returns `Holdout validation: PASS` or `FINDINGS` with P1/P2/P3 rows citing `file:line`, mapped to visible acceptance criteria, consumed by verdict-judge. Permitted skips: none; with a missing input, evaluate what is available and note the gap, halting only when no file list is given. ## Inputs 1. **Self-review findings**: the code-reviewer's P1/P2/P3 findings 2. **Evidence bundle draft**: per-criterion evidence, including `### Test inputs and expected values` and `### Risk map coverage` 3. **File list**: every file modified or created on the branch Schema: `schemas/holdout-validation/input-schema.json`. ## Process 1. **Load scenarios** for each criterion type present from `templates/holdout-scenarios/{behavioral,api,error,data}.md`. Number them by document order across loaded files (scenario-1 to scenario-N); skip and note a missing type. 2. **Parse claims.** Extract each finding's and evidence entry's claim and file references. A claim with no file/line citation is a bare assertion: automatic P2. 3. **Cross-reference claims.** Read each cited location; record CONFIRMED or CONFLICT. "Test added for X" needs assertions that verify X, no