← ClaudeAtlas

three-layer-evallisted

Three-layer skill quality evaluation (Static → LLM Judge → Monte Carlo). Layer 1: deterministic checks (frontmatter, triggers, size, links) in <2s. Layer 2: semantic eval across 4 dimensions using Haiku+Sonnet. Layer 3: 50-100 simulated runs for statistical reliability.
LucasSantana-Dev/sharekit-profile · ★ 1 · AI & Automation · score 70
Install: claude install-skill LucasSantana-Dev/sharekit-profile
# three-layer-eval Three-layer evaluation framework for skill quality assessment. ## Layer 1: Static Analysis (<2s) Deterministic checks — fast, no LLM required. ### Checks 1. **Frontmatter completeness** - `name` present and matches directory name - `description` present, >20 chars, <200 chars - `triggers` array present, at least 1 trigger, each trigger >3 chars 2. **Trigger-phrase presence** - At least one trigger appears in the skill body (natural usage) - Triggers are not generic words ("help", "run", "do") 3. **Body size** - SKILL.md body <8KB (excluding frontmatter) - SKILL.md <150 lines - Total directory <50KB 4. **Reference link validity** - All relative links resolve to existing files - No broken anchors (#section links) 5. **Structural patterns** - Has clear "When to use" or trigger section - Has output/return format documented - No TODO or placeholder content ### Output ``` Layer 1: STATIC — PASS/FAIL frontmatter: OK triggers: OK (3 triggers, all appear in body) size: OK (4.2KB body, 87 lines) links: FAIL (1 broken: references/old-api.md) structure: OK ``` ## Layer 2: LLM Judge (Haiku + Sonnet) Semantic evaluation across 4 dimensions. Uses Haiku for fast scoring, Sonnet for nuanced cases. ### Dimensions 1. **Completeness** (0-10) - Does the skill cover its stated use cases? - Are edge cases documented? - Is the output format clear? 2. **Trigger-accuracy** (0-10) - Do triggers accurately pr