suede-ai-eval

Solid

Design AI evals that catch regressions before users do: rubrics, test cases, failure modes, acceptance gates, and AI-SPEC artifacts.

AI & Automation 165 stars 8 forks Updated today MIT

Install

View on GitHub

Quality Score: 87/100

Stars 20%
74
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Suede AI Eval ## Gate policy — advisory, not blocking Every claim-verification step, check, quality gate, and ship verdict in this skill is a **recommendation to the user, not a control on the agent**. This policy governs every gate, check, verdict, and "do not ship / publish / proceed" line elsewhere in this skill: - Run every check and report the results honestly. Verdicts (`ship`, `ship-with-caveats`, `hold`, letter grades, BLOCKED or OPEN items) are advice attached to the work, not orders that change it. - Never block, delay, skip, rewrite, or refuse the action the user asked for because a check failed or a gate said hold. Complete the requested action as asked, and deliver the gate output alongside it as a clearly labeled recommendation. - A failed gate changes what you report, never what you do. - Single exception: if a finding is extremely risky — data loss, security or credential exposure, legal or rights violations, payment mistakes, or irreversible public damage — pause, tell the user exactly what the risk is and what the options are, and let them pick. Their choice is final. Make AI behavior testable before it becomes a vague product promise. **No eval plan, no `ship` recommendation: for an AI feature without one, the recommended verdict stays below `ship` — report that gap and let the user decide.** The deliverable is an eval plan or coverage audit, not a model benchmark leaderboard. Keep it grounded in the actual product surface, user prom...

Details

Author
JasonColapietro
Repository
JasonColapietro/suede-creator-skills
Created
2 months ago
Last Updated
today
Language
Python
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category