← ClaudeAtlas

model-upgrade-evaluatorlisted

Iterate-stage skill: turns a new model release into a re-test plan for current prompts and shelved ideas — every verdict cites a run result, no capability assumed from release notes. Use when a new model version lands — 'new model dropped, what should we re-test', 'worth migrating our prompts?', 'build the upgrade evaluation plan', 're-test our shelved ideas' — or when /pm routes such a request here. Do NOT use to gate an already-decided migration's ship (regression-gatekeeper), for per-task tier routing (model-complexity-router), for release-notes summaries, or for academic benchmarking.
Abhillashjadhav/PM-agent-OS · ★ 1 · AI & Automation · score 60
Install: claude install-skill Abhillashjadhav/PM-agent-OS
# Model Upgrade Evaluator Release notes are hypotheses. A new model changes two lists — what might now break (current prompts) and what might now work (shelved ideas) — and both get answered by runs, not by the announcement. ## Verification gates (defined first; output is blocked until all pass) - **G1 — Every verdict cites a run:** MIGRATE/STAY/RESURRECT/STILL-DEAD verdicts exist only with per-case run evidence (counts, deltas, the failing or passing cases named). Before runs execute, every verdict slot reads `PENDING`. Release-note claims appear only as hypotheses-to-test, never as capability facts. - **G2 — Kill reasons become resurrection tests:** each shelved idea is re-tested against the specific reason it was killed (the historical failing case or its closest reconstruction) — not a fresh demo that avoids the old trap. A resurrection test that doesn't encode the kill reason fails. - **G3 — Budget-honest coverage:** the plan ranks runs within the stated budget, names what's cut, and marks unscheduled items `UNTESTED — not a verdict`. Claimed coverage beyond scheduled runs fails. ## Steps 1. **Inventory both lists:** production prompts with their golden sets (these re-run under regression-gatekeeper rules — fail-class assertions must hold), and the shelved-ideas log with each idea's kill reason stated as a testable condition. 2. **Convert release-note claims into targeted hypotheses.** "Better instruction-following" → which of our golden failures were instruction-fo