← ClaudeAtlas

drift-monitor-designerlisted

Iterate-stage skill: designs production drift monitoring for a shipped AI feature — every monitored signal carrying a threshold and a named response action. Use when a live feature needs watching — 'design drift monitoring for the summarizer', 'how do we know when it degrades in production', 'what do we watch now that it shipped' — or when /pm routes such a request here. Do NOT use for per-request guardrails (guardrail-designer), for pre-ship change gating (regression-gatekeeper), for building the dashboards themselves, or for drift definitions.
Abhillashjadhav/PM-agent-OS · ★ 1 · Web & Frontend · score 60
Install: claude install-skill Abhillashjadhav/PM-agent-OS
# Drift Monitor Designer Production quality decays quietly. A monitor is a signal with a threshold and a hand that moves when it's crossed — "watch quality and alert if it drops" is anxiety, not monitoring. ## Verification gates (defined first; output is blocked until all pass) - **G1 — Threshold + named action, every signal:** each signal carries a threshold (numeric with a stated basis, or a labeled placeholder with the baseline-collection step that will set it) and a named response action with its route (pull a judge sample, flip the rollback posture, recalibrate). Thresholdless watching or actionless alerting fails. - **G2 — Plumbing-bounded, cost-layered:** signals use only the stated instrumentation, arranged cheap-to-expensive: continuous proxies (edits, deletes, thumbs) triage; scheduled judge runs against the eval's gates are the ground truth; input-distribution signals watch the upstream cause. Proxies never substitute for ground truth, and their biases are stated (12% feedback = self-selected sample). - **G3 — Alert hygiene:** every signal names its expected false-positive source and its damping rule (consecutive windows, not single spikes). No invented baselines — unknown baselines get a collection window, not a made-up number. ## Steps 1. **Bank the surface:** volume, plumbing (what's logged, what users emit, judge-sample capacity), the eval's quality dimensions (gates and rubric — the monitor watches for regressions against exactly these), and whether a rol