eval-planlisted

# Eval Plan Turn an AI feature into an evaluation plan with launch gates that are fixed before anyone sees a result. The idea behind this: an eval only protects you if it can fail. Most AI eval plans can't, because the test set only has the easy cases, the bar gets decided after the numbers come in, or "accuracy" hides the one error that actually hurts users. This skill writes a plan designed to catch the failure before launch. ## Usage ``` /eval-plan $ARGUMENTS ``` ## Workflow ### 1. Find the error that actually hurts Read the PRD or description. If it's a file path, read the file. If key facts are missing, ask for them one or two at a time. Before picking any metric, answer: **which mistake costs the most, and who pays for it?** Name the worst error in one sentence. - A support-reply drafter that confidently drafts a wrong billing answer is worse than one that sends the ticket to a human with no draft. So precision on confident predictions matters more than overall accuracy. - A fraud flagger that misses fraud may be worse than one that flags too much. So recall matters more. Say which way the costs lean. Every threshold later in the plan follows from this. ### 2. List what the feature must do, and what it must refuse to do Write 3–6 behaviors as testable statements, in two groups: - **Must do:** "Classifies a password-reset ticket as `password_reset`." - **Must not do / must fall through:** "Sends an off-topic ticket to a human instead of forcing it into the n