← ClaudeAtlas

eval-skillslisted

Eval and improve a skill against golden cases — run the target skill blind in a fresh, context-free subagent on each example input, grade the artifact against a per-case bar, and let the gaps drive the edits. Use when the user wants to test/eval/improve/harden a skill, says "this skill keeps producing X / keeps missing Y", or hands a skill plus cases with a bar for what good looks like. Pairs with write-skills (the authoring principles every fix obeys). Judgment-first and harness-free — prefer skill-creator for harness-based benchmarking, variance analysis, or description-optimization loops.
avivsinai/skills-marketplace · ★ 2 · AI & Automation · score 68
Install: claude install-skill avivsinai/skills-marketplace
# Eval Skills Treat a skill like a function under test. Feed it example inputs in a clean room, check the artifacts against what good looks like, and let the failures drive the edits. The eval is only honest if the run is **blind**: the agent executing the skill must carry none of this conversation's context and must never see the expected output. Leak either and you are teaching to the test. ## Inputs to settle before running Confirm all three before spawning anything. If any is missing, do not run yet: tell the user *exactly which one* is missing and what a good version looks like, help them make it concrete, and echo it back. Do not invent cases, guess intent, or eval against a fuzzy wish. - **Target skill** — must resolve to a real `SKILL.md`. If you can't find it, list the skills you can see and ask which one they mean. - **At least one golden case** — a concrete input the skill will actually receive: a screenshot, a prompt, a file, a scene. "Improve write-spec" with no input attached is not a case. - **The bar per case** — the outcome a good artifact achieves and the smells that would make it bad, *not* an exhaustive parts list. The skill's **judgment** is what's under test, so do not pre-enumerate every requirement — that turns the eval into a conformance check and stops testing whether the skill decides well. "Sliced so each piece is independently buildable and verifiable, at the granularity a competent practitioner would pick — a lazy mega-slice