experiment-design
FeaturedA discipline for designing experiments (A/B tests, multivariate, holdouts) so the results actually answer the question you asked. Hypothesis writing, sample size, duration, segment analysis, running discipline, matching a result to a pre-committed decision rule, and the common failure modes that produce confidently wrong shipping decisions. Use this skill whenever the user is planning a test that has not run yet: framing a hypothesis, sizing the sample, setting duration, choosing guardrails, or deciding whether something is worth testing at all. Triggers on design an experiment, experiment plan, A/B test, split test, multivariate test, holdout, experiment hypothesis, sample size, minimum detectable effect, MDE, test duration, guardrail metric, no peeking, pre-committed decision rule, is this worth testing. Use `experimentation-analytics` instead when the test has already run and the question is how to read the result panel.
Install
Quality Score: 98/100
Skill Content
Details
- Author
- rampstackco
- Repository
- rampstackco/claude-skills
- Created
- 4 months ago
- Last Updated
- 3 days ago
- Language
- Python
- License
- MIT
Integrates with
Bundled in these plugins
Similar Skills
Semantically similar based on skill content — not just same category
experiment-design
Use when planning A/B tests, fake-door tests, beta rollouts, experiments, metric rules, or interpreting outcomes.
experimentation-analytics
How to read experiment results without fooling yourself. Confidence intervals, p-values, multiple testing, sequential testing, CUPED, heterogeneous treatment effects, ratio metrics, network effects, dashboard reconciliation, and the interpretation failures that produce confidently wrong shipping decisions. Use this skill whenever the user is reading a finished experiment result panel and about to make a ship, kill, or iterate decision, or when an experiment number does not match the dashboard number. Triggers on read experiment results, result panel, ship or kill decision, p-value, confidence interval, statistical significance, multiple testing, peeking, sequential testing, CUPED, variance reduction, heterogeneous treatment effects, ratio metric, network effects, inconclusive test, experiment versus dashboard mismatch. Use `experiment-design` instead when the test has not run yet and the question is hypothesis, sample size, duration, or what to test.
experiment-designer
Turn an assumption or a growth bet into an experiment brief with a mechanism-stated hypothesis, one primary metric, guardrails with floors, an exposure design, a sample size reasoned from the minimum detectable effect, and stop rules written before launch. Use when a growth plan bet needs a test, when an assumption register row is low confidence and high impact, when a price change should be tried on a slice first, or when someone says "let us just A/B it". Takes the assumption, the metric candidate with its baseline, the eligible traffic, and the guardrail candidates; returns the test card, the filled brief, the sizing record with its calculator inputs, and the pre-committed decision rule.