← ClaudeAtlas

model-risk-evidence-matrixlisted

Use when an ML model release is up for promotion and someone must decide which fairness and explainability evidence is required, or check the evidence against the model's risk tier. Assigns a low, medium or high tier, lists required artifacts and returns promote, needs-work or block.
testland/qa-skills · ★ 0 · AI & Automation · score 72
Install: claude install-skill testland/qa-skills
# model-risk-evidence-matrix This is a decision matrix for one question: **given what a model decides, what fairness and explainability evidence must exist before anyone can reason about it?** It converts a risk tier into a required-artifact list, and converts a missing or self-contradicting artifact into a named evidence rule. ## What this owns, and what it does not **Owns:** the risk-tiering scheme, the evidence-required-per-tier matrix, the conventional verdict bands on a group fairness metric, the per-vulnerability category blocking table, and the evidence rules that mark a bundle incomplete. **Does not own:** - **Training or mitigation.** Nothing here retrains a model, reweights data, or applies a fairness constraint. When a disparity exceeds a band, this matrix says the bundle needs mitigation provenance. It does not produce the mitigation. - **Certifying a model as fair.** A complete bundle means the evidence exists and is internally consistent. It does not mean the model is fair. Fairness is contested: "different people may have different ideas of what is fair in a particular scenario", and mathematical metrics "fail to capture all relevant aspects" ([Fairlearn, fairness in machine learning](https://fairlearn.org/main/user_guide/fairness_in_machine_learning.html)). - **The release decision.** Whoever owns release accepts or rejects. This matrix supplies the evidence status they act on. ## Step 1 - Assign the risk tier Tier on what the prediction