← ClaudeAtlas

run-ab-experimentslisted

Use this skill when a user wants to plan, review, launch, manage, intervene in, ramp, monitor, analyze, or interpret an online controlled experiment or A/B test; choose hypotheses, units, variants, exposure, metrics/OEC, guardrails, sample size, duration, stopping rules, SRM checks, or ship/iterate/stop decisions. Manage the full lifecycle in three modes—plan, manage, and interpret—with explicit cognitive-load reviews and concise, non-repetitive outputs grounded in Kohavi, Tang, and Xu. Always begin with a mandatory question-first interview and context-and-assumptions confirmation before work. Do not use it for observational-only causal claims unless deciding whether an experiment is feasible.
shahriarfarzadi/run-ab-experiments-skill · ★ 0 · AI & Automation · score 70
Install: claude install-skill shahriarfarzadi/run-ab-experiments-skill
# Trustworthy A/B Experiments Produce a decision-grade experiment plan, audit, or analysis. Keep the causal question, design, execution integrity, statistical evidence, and business decision separate. ## Mandatory question-first protocol Treat context discovery as a hard validity gate. Apply this protocol to every task, including apparently complete requests and requests containing results, queries, dashboards, or datasets. ### Phase A: Interview before work 1. Do not plan, calculate, recommend metrics, inspect outcome data, run queries, interpret results, or give a provisional decision yet. 2. Classify the request as **plan**, **manage/intervene**, **interpret/analyze**, platform/method audit, or ambiguous. 3. Read [references/discovery-question-bank.md](references/discovery-question-bank.md). 4. In the first response: - restate the intended decision in one tentative sentence; - ask 12–20 numbered, high-impact questions from the relevant question bank, grouped under no more than five short headings; - cover hidden causal, product, operational, statistical, data-quality, ethical, and decision assumptions; - ask the user to answer `unknown` when information is unavailable; - do not include analysis, a proposed design, calculations, or conclusions. 5. When the initial prompt is already detailed, do not repeat answered questions. Ask at least five adversarial confirmation or gap questions that could still reverse the design or interpr