ab-test-planner

Featured

Design statistically rigorous A/B tests for product features, UI changes, onboarding flows, and pricing experiments. Use when asked to set up an experiment, design an A/B test, calculate sample size, or interpret test results. Produces a complete test plan with hypothesis, variant definitions, sample size, duration estimate, guardrail metrics, and a results interpretation guide.

AI & Automation 1,356 stars 240 forks Updated yesterday MIT

Install

View on GitHub

Quality Score: 96/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# A/B Test Planner Skill Design experiments that produce trustworthy results — not just directional signals. Every test output includes hypothesis, success metrics, sample size, duration, and a results interpretation guide. ## Required Inputs Ask the user for these if not provided: - **What is being tested** (feature, UI change, copy, pricing, onboarding step) - **Hypothesis** (or ask to help formulate one) - **Primary metric** (conversion rate, click-through, completion rate, etc.) - **Baseline rate** and **minimum detectable effect** (MDE) - **Daily eligible users** (to calculate duration) ## Experiment Design Checklist Before running any test, confirm: - [ ] Clear hypothesis with predicted direction - [ ] Single primary metric (plus up to 2 guardrail metrics) - [ ] Minimum detectable effect (MDE) defined - [ ] Sample size calculated - [ ] Test duration estimated - [ ] Segment isolated (no overlap with other running tests) - [ ] Rollback plan defined ## Hypothesis Template > "We believe that [change] will cause [primary metric] to [increase/decrease] by [X%] for [user segment], because [rationale based on data or insight]." Never run a test without a directional hypothesis. "Let's just see what happens" is not a hypothesis. ## Sample Size Calculator Logic Use this formula (provide the output, not the formula, to the user): - **Baseline conversion rate:** Current rate of primary metric - **MDE:** Smallest change worth detecting (recommend 10–20% relative lift for ...

Details

Author
mohitagw15856
Repository
mohitagw15856/pm-claude-skills
Created
7 months ago
Last Updated
yesterday
Language
HTML
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Featured

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.

25,834 Updated 1 weeks ago
alirezarezvani
AI & Automation Listed

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics-tracking. For page-level conversion optimization, see page-cro.

0 Updated 2 weeks ago
planifest
Testing & QA Listed

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," or "hypothesis." For tracking implementation, see analytics-tracking.

4 Updated 1 months ago
kimny1143