experimentlisted

# Experiment — A/B Testing & Experimentation ## Activate When - User invokes `/godmode:experiment` - User says "A/B test", "split test", "experiment" - User says "statistical significance", "sample size" - User says "Statsig", "Optimizely", "GrowthBook" ## Workflow ### Step 1: Experiment Discovery ```bash # Detect experimentation SDK grep -l "statsig\|optimizely\|growthbook\|launchdarkly" \ package.json pyproject.toml 2>/dev/null # Check for existing experiment configs find . -name "*experiment*" -o -name "*ab_test*" \ | grep -v node_modules | head -10 ``` ``` EXPERIMENT DISCOVERY: Baseline metric: <name> = <current value> Traffic: <daily users hitting the surface> Platform: none | Statsig | Optimizely | GrowthBook Risk: low (revenue) | medium (growth) | high (minor UX) IF no platform: recommend Statsig (best free tier) IF baseline unknown: measure for 7 days first IF traffic < 1000/day: experiment may take months ``` ### Step 2: Experiment Design ``` EXPERIMENT DESIGN: Name: <experiment-slug> Hypothesis: If we <change>, then <metric> will <improve> by <magnitude> because <reasoning>. METRICS: | Type | Metric | Baseline | Target | |-----------|-------------|----------|--------| | Primary | <OEC> | <val> | +<X>% | | Guardrail | latency p95 | <val> | < +10% | | Guardrail | error rate | <val> | < +0.1%| | Guardrail | revenue | <val> | > -2% | | Secondary | <metric> | <val> | +<X>% | ``` ### Step 3: Sample Size & Power