experiment-rigorlisted

Structure an experiment and its write-up so the Methods reconstruct without the conversation and the uncertainty is declared, not hidden — a typed record.yaml across a probe / measurement / decision tier ladder, every load-bearing rule a script that exits non-zero rather than a line of prose. Use when you pre-register an A/B, freeze a plan before running it, write up an experiment or a comparison, add error bars or a confidence interval to a rate, reconcile declared cells against the runs that happened, separate confirmatory from exploratory outcomes, name the threats to validity, design the 2x2, or ask whether a skill, tier, model, or strategy actually helps and how you would show it rigorously. Covers the small-n refusal (no CLT below 30 — Wilson, Clopper-Pearson, or a within-experiment Beta-Binomial), the design-arithmetic reconciliation (declared cells == disposition == denominators), the plan freeze and its drift gate, the rate that needs both a numerator and a denominator, probe self-labeling, and recor
grimaldost/craft-collection · ★ 2 · AI & Automation · score 68

Install: claude install-skill grimaldost/craft-collection

# Experiment Rigor Turn an experiment into a typed `record.yaml` whose `report.md` is derived, not written. This is a **rigid** skill: the scripts under `scripts/` are the mechanism, and each bright line below is a gate that exits non-zero with a stable error code (`validate.py`), not advice to remember. The dividing line the discipline draws: **methods uncertainty is disqualifying, effect uncertainty is declarable** — a reader who cannot reconstruct what was manipulated, where, and how it was measured has found a defect; a wide confidence interval is an honest result. Paths below are relative to `${CLAUDE_PLUGIN_ROOT}/skills/experiment-rigor/`, and PyYAML is the scripts' only non-stdlib dependency — run them as the gates do: ```bash uv run --no-project --with pyyaml -- python \ "${CLAUDE_PLUGIN_ROOT}/skills/experiment-rigor/scripts/validate.py" <record.yaml> ``` The same two gates ship as pre-commit hooks (`experiment-rigor-validate`, `experiment-rigor-render-check`); the README names the install surface. ## The tier ladder `tier` is a field, not a second skill. It selects the required fields and which gates apply. Tier-0 is the exception: it names no artifact, so it never appears as a `tier:` value. - **check** (tier-0) — the structured check. An evaluation act answered inline in the response: no file, no record, no gate. An *evaluation act* asks a question whose correct answer is a valuative claim ("is this effective", "which is better"); an *execution or lo