model-performance-regression-gatelisted
Install: claude install-skill testland/qa-skills
# model-performance-regression-gate
A CI gate that blocks model promotion when a retrained model regresses on
held-out metrics vs the current production model beyond a configured tolerance.
Covers global metrics (accuracy, F1, AUC, RMSE) and per-segment checks so a
model that improves in aggregate but regresses on a key slice is still blocked.
Differentiation from neighbors:
- `deepchecks-tests` runs the full model-evaluation suite but does not
compare a candidate against a production model or enforce per-metric
tolerances as a promotion gate.
- `evidently-monitoring` detects data and target drift in production traffic;
it does not perform a pairwise metric comparison between two model versions
at deploy time.
- This skill is scoped to the promotion-gating decision: candidate vs.
production, per metric, per segment, with configurable tolerances.
## When to use
Invoke in a CI/CD pipeline step immediately after retraining, before the
model artifact is registered or deployed. The step receives the held-out test
set, the candidate model, and the production model (loaded from the registry).
It exits non-zero when any metric degresses beyond tolerance.
## Step 1 - Install dependencies
```bash
pip install deepchecks scikit-learn joblib
```
Deepchecks is the primary framework for segment-level checks (per
[Deepchecks model evaluation docs]). scikit-learn supplies the scalar
metric functions (per [scikit-learn model evaluation docs]).
## Step 2 - Load models and held