ab-testinglisted

# A/B Testing ## When This Skill Applies - Designing A/B or multivariate tests for campaigns - Calculating sample sizes and test duration - Formulating test hypotheses with measurable outcomes - Analyzing test results for statistical significance - Recommending test priorities by expected impact ## Core Guidance ### 1. A/B Test Process ``` Hypothesis -> Variant Design -> Sample Sizing -> Test Setup -> Run Test -> Analyze Results -> Implement Winner -> Document Learnings ``` ### 2. Hypothesis Framework **Template**: "If we [change X], then [metric Y] will [improve/decrease] by [estimated %] because [rationale based on evidence]." | Component | Description | Example | |-----------|-------------|---------| | Change | What is being modified | "change CTA color to green" | | Metric | Primary success metric | "click-through rate" | | Direction | Expected outcome | "increase by 10-15%" | | Rationale | Evidence-based reasoning | "green contrasts better with page design" | ### 3. Test Element Priorities | Element | Impact Potential | Test Complexity | Priority | |---------|-----------------|-----------------|----------| | Value proposition / headline | High | Low | P1 | | CTA text and placement | High | Low | P1 | | Page layout / hero section | High | Medium | P1 | | Form fields (count, order) | High | Medium | P2 | | Social proof placement | Medium | Low | P2 | | Image / visual content | Medium | Medium | P2 | | Color scheme / button color | Low-Medium | Low | P3 | | Microcop