skill-evaluation

Featured

Evaluate any agent skill against a merged framework — Anthropic's Claude Code best practices plus Matt Pocock's writing-great-skills methodology — across 4 axes (Trigger, Structure, Steering, Pruning). Produces an evidence-cited scorecard (0–100), a weighted overall score, and diagnosed failure modes with prioritized fixes. Use when the user asks to evaluate, rate, or audit a skill ("evaluate this skill", "skill scorecard", "review SKILL.md"), or to compare two skills.

AI & Automation 77 stars 7 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 88/100

Stars 20%
63
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Skill Evaluation If you need the vocabulary and tests behind Axes 1, 3, and 4 (leading words, completion criteria, context pointers, the deletion test, failure-mode definitions), read `references/mechanics.md` before scoring those axes. ## Source - [Lessons from building Claude Code: How we use skills](https://claude.com/blog/lessons-from-building-claude-code-how-we-use-skills) — Anthropic, Jun 2026 - "The Missing Manual: How to Write Great Skills" — Matt Pocock, AI Engineer World's Fair 2026 ([video](https://www.youtube.com/watch?v=UNzCG3lw6O0)), and his `writing-great-skills` skill ## Parameters | Parameter | Description | Default | |-----------|-------------|---------| | `target` | Path to skill directory or SKILL.md to evaluate | Ask user | | `output` | Path to write the scorecard | `<target>/EVALUATION.md` | | `compare` | Optional second skill to compare side-by-side | None | Also runs unattended: in CI, point `target` at skills changed in a PR and gate with `scripts/score.py --fail-below 60 ...` — non-zero exit below the threshold fails the check. ## Criteria 18 criteria: 14 core, scored on every skill, plus 4 conditional criteria scored only when the skill's category makes them apply — otherwise mark **N/A** and exclude the criterion from both the numerator and denominator of the weighted average. Every score is 0–100 with evidence citing file, section, or line. ### Axis 1 — Trigger (invocation) | # | Criterion | Weight | Key question | |---|-----------|---...

Details

Author
fabricioctelles
Repository
fabricioctelles/skills
Created
6 months ago
Last Updated
today
Language
Python
License
Apache-2.0

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

skill-evaluation-graph

Deeply analyze, audit, score, and optimize agent skills conforming to the Agent Skills standard or Antigravity/Codex/Claude formats. Use when reviewing an existing skill, diagnosing why an agent misfires or burns context, pruning skill bloat, refactoring monolithic instructions into progressive disclosure, or benchmarking skill quality. Do not use for creating skills from scratch without an existing procedure (use workflow-skill-creator).

2 Updated 4 days ago
MaxLaurieHutchinson
AI & Automation Listed

evaluate-skill

Use when you want to behaviorally evaluate a Claude Code skill — measure whether it auto-activates on the prompts it should and stays quiet on near-misses (trigger recall and specificity), whether its output actually satisfies its own discipline (correct-usage), and whether it beats the no-skill baseline (with/without) — producing a scorecard with confidence intervals. Triggers on "evaluate this skill", "test my skill", "does my skill fire", "measure or benchmark skill performance", "is my description triggering", "build an eval for this skill", or running "/evaluate-skill". Claude Code only — it spawns many headless `claude -p` runs, so it is cost-gated; show the plan first. Not for a one-off manual spot-check, for judging a skill's design qualitatively (that is a fresh-eyes panel, not a behavioral eval), or for listing which skills are installed (that is toolkit-awareness).

2 Updated 5 days ago
grimaldost
AI & Automation Listed

skill-evaluator

Audit an existing SKILL.md for rule adherence — does the text actually land when Claude runs it? Use when the user wants a behavioral review of a shipped skill. Outputs: failure classification by fix layer (skill text / rubric / brief / fixture) and targeted rule-text diffs. Trigger ON: "audit this skill", "stress-test my skill", "does this skill actually work", "find gaps in this skill", "what's broken in this skill", "validate rule adherence", "review this skill end-to-end", or uploading a SKILL.md for behavior review. Do NOT trigger on: "build a skill", "create a skill from scratch", "benchmark this skill", "evaluate skill quality", "compare versions", "optimize trigger phrases", or "measure variance" — those are all `skill-creator`. If the request mixes both, start with `skill-creator` and chain here. Hard boundary: `skill-creator` builds, benchmarks, measures variance, and optimizes triggering; this skill does NONE of those — it asks "does the text land?".

1 Updated 3 weeks ago
sorawit-w