ab-test-planner

Solid

Design statistically rigorous A/B tests for product features, UI changes, onboarding flows, and pricing experiments. Use when asked to set up an experiment, design an A/B test, calculate sample size, or interpret test results. Produces a complete test plan with hypothesis, variant definitions, sample size, duration estimate, guardrail metrics, and a results interpretation guide.

Testing & QA 915 stars 165 forks Updated 3 days ago MIT

Install

View on GitHub

Quality Score: 93/100

Stars 20%
99
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# A/B Test Planner Skill Design experiments that produce trustworthy results — not just directional signals. Every test output includes hypothesis, success metrics, sample size, duration, and a results interpretation guide. ## Required Inputs Ask the user for these if not provided: - **What is being tested** (feature, UI change, copy, pricing, onboarding step) - **Hypothesis** (or ask to help formulate one) - **Primary metric** (conversion rate, click-through, completion rate, etc.) - **Baseline rate** and **minimum detectable effect** (MDE) - **Daily eligible users** (to calculate duration) ## Experiment Design Checklist Before running any test, confirm: - [ ] Clear hypothesis with predicted direction - [ ] Single primary metric (plus up to 2 guardrail metrics) - [ ] Minimum detectable effect (MDE) defined - [ ] Sample size calculated - [ ] Test duration estimated - [ ] Segment isolated (no overlap with other running tests) - [ ] Rollback plan defined ## Hypothesis Template > "We believe that [change] will cause [primary metric] to [increase/decrease] by [X%] for [user segment], because [rationale based on data or insight]." Never run a test without a directional hypothesis. "Let's just see what happens" is not a hypothesis. ## Sample Size Calculator Logic Use this formula (provide the output, not the formula, to the user): - **Baseline conversion rate:** Current rate of primary metric - **MDE:** Smallest change worth detecting (recommend 10–20% relative lift for ...

Details

Author
mohitagw15856
Repository
mohitagw15856/pm-claude-skills
Created
4 months ago
Last Updated
3 days ago
Language
Shell
License
MIT

Similar Skills

Semantically similar based on skill content — not just same category

DevOps & Infrastructure Listed

ab-test-plan

Use when the user needs to design an A/B test, calculate sample size for an experiment, plan a multivariate test, create a testing hypothesis, or set up a CRO experimentation program.

2 Updated today
Faiz07yo
AI & Automation Solid

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.

16,642 Updated yesterday
alirezarezvani
Testing & QA Solid

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," or "hypothesis." For tracking implementation, see analytics-tracking.

27,681 Updated today
davila7
Web & Frontend Listed

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics-tracking. For page-level conversion optimization, see page-cro.

0 Updated 2 days ago
SanctifiedOps
Testing & QA Listed

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," or "how long should I run this test." Use this whenever someone is comparing two approaches and wants to measure which performs better. For tracking implementation, see analytics-tracking. For page-level conversion optimization, see page-cro.

0 Updated 4 days ago
evgeniiakruglova