experiment-metricslisted

# Experiment Metrics Selection: STEDII Framework **When to use:** Before launching any experiment, when metrics feel unreliable, or when experiment results are confusing **Framework source:** Aakash Gupta's "How to Choose the Right Metrics to Evaluate Experiments" --- ## The STEDII Framework Choose experiment metrics that are: 1. **S**ensitive 2. **T**imely 3. **E**fficient 4. **D**ebuggable 5. **I**nterpretable 6. **I**solated --- ## 1. Sensitive (Detects Small But Meaningful Changes) **What it means:** The metric moves when your feature actually improves the experience **Bad example:** - Metric: Monthly Active Users (MAU) - Problem: Too coarse. A good onboarding improvement might not move MAU for months. **Good example:** - Metric: Day 7 activation rate - Why: Sensitive enough to detect onboarding improvements within a week **How to check:** Ask: "If this experiment succeeds, will this metric move within the experiment window?" **Common mistake:** Using metrics that are too aggregated (MAU, total revenue) when you need something more granular (daily activation, conversion rate by cohort). --- ## 2. Timely (Results Available Quickly) **What it means:** You get signal fast enough to make decisions **Bad example:** - Metric: 90-day retention - Problem: Takes 90 days to know if your experiment worked **Good example:** - Metric: Day 7 retention + leading indicators - Why: Faster feedback, correlates with long-term retention **Tradeoff alert:** Sometimes you NE