← ClaudeAtlas

evidently-monitoringlisted

Use for any question or task about data drift or quality monitoring for a production ML model, including a fired drift alert and metrics such as precision that degrade slowly unnoticed: Evidently reports with tests for data drift, target drift and performance regression as a CI gate or scheduled job, and a playbook to pick rollback, retrain, quarantine or alert re-tuning.
testland/qa-skills · ★ 0 · AI & Automation · score 72
Install: claude install-skill testland/qa-skills
# evidently-monitoring Evidently is *"an open-source Python library with over 40+ million downloads. It provides 100+ evaluation metrics, a declarative testing API, and a lightweight visual interface"* per [Evidently docs]. ## When to use - Pre-deployment gate: assert no data/target drift between candidate-model evaluation set and the production reference. - Production monitoring: scheduled job comparing yesterday's traffic vs the reference window. - Triage tool: when a model misbehaves in prod, run an Evidently Report comparing the bad period to a known-good window. ## How to use 1. `pip install evidently` into the CI or monitoring environment (Step 1). 2. Load a pinned reference dataset and the current dataset, and wrap both in a `Dataset` with the same `DataDefinition` (Step 2). 3. Run `Report([DataDriftPreset()])` with `run(reference_data=, current_data=)` and `save_html` for review (Step 3). 4. For a CI gate, add `include_tests=True`, read `.dict()["tests"]`, and `raise SystemExit` on any `FAIL`/`ERROR` (Step 4). 5. For performance, map the task in the data definition (`BinaryClassification` with `target`, `prediction_probas` and `pos_label`) and set the business `probas_threshold` before trusting precision or recall (Step 5). 6. Wrap the gated Report in a daily scheduled job comparing yesterday's traffic to the pinned reference, notifying on-call on failure (Step 6). 7. Tune the per-column drift method (`psi`, etc.) and severity tiers so