ai-eval-reviewlisted

Audit the evaluation layer of an AI product against design-completeness questions: offline criteria, ground-truth quality, online signal, cohort/disparate-impact, adversarial + robustness coverage, and drift detection. Eval-side companion to `ai-ux-review`. Produces an editable Markdown artifact plus a self-contained HTML report under `docs/ai-ux/`. Use when the user asks to "review my AI eval setup", "audit my eval design", "is my AI eval rigorous enough", "responsible-AI eval review", "fairness eval check", or "drift detection design", or ships an LLM/ML feature and wants an eval-rigor check. Invoke even if only one block is named — the others stress-test it. Does NOT trigger for: human-AI UX review (`ai-ux-review`); lean-canvas work (`validation-canvas`); adversarial pre-mortem with a verdict (`startup-grill`); SKILL.md audits (`skill-evaluator`); implementing eval pipelines, writing eval code, or labeling datasets (this skill names gaps, it does not build them).
sorawit-w/agent-skills · ★ 1 · AI & Automation · score 77

Install: claude install-skill sorawit-w/agent-skills

# AI Eval Review Audit the eval layer of an AI product or feature against a structured set of design-completeness questions. Each of the seven blocks is a *gap detector* — the value of this skill is whether the elicitation surfaces eval decisions the builder hasn't made yet, not whether the artifact looks complete. The job is **eval-design-completeness**: *have we designed how we'll know if this works?* Covers offline criteria, ground-truth quality, online signal, cohort breakdowns and disparate impact, adversarial / robustness coverage, and drift detection. Regulatory rigor (EU AI Act, FDA SaMD, FTC) is a cross-cutting lens applied across blocks, not a separate block. This skill is the eval-side companion to `ai-ux-review`. Same shape, same elicitation pattern, different subject — `ai-ux-review` audits the human-AI design surface (was the experience intentionally designed?); this skill audits the measurement layer behind it (do we have signal for whether the design works?). This is not an implementation tool. It does not write eval code, label datasets, set up monitoring dashboards, or compute metrics. It names the gaps; you take them to your tools or your team to close. ## What this skill produces Always produced under the resolved review root (default `docs/ai-ux/` — same folder as `ai-ux-review` for clean composition): 1. **`ai-eval-review.md`** — canonical, editable Markdown with one top-level section per block (seven blocks total), plus a `## Gap Summary` sect