skill-forge-evallisted

# Skill Evaluation Pipeline Run structured evaluations against Claude Code skills to verify triggering, correctness, and quality using a multi-agent pipeline. ## Process ### Step 1: Define Eval Set Accept eval definitions from: - **Path to eval set JSON**: `evals/evals.json` or user-specified file - **Inline prompts**: User provides eval queries directly - **Auto-generated**: Generate from skill description (see Step 1b) **Eval set JSON schema:** ```json { "skill_name": "my-skill", "skill_path": "./my-skill", "evals": [ { "eval_id": 0, "eval_name": "descriptive-name", "prompt": "The user's task prompt", "input_files": [], "assertions": [ { "name": "output-has-score", "check": "Output contains a numeric score between 0-100", "weight": 1.0 } ], "should_trigger": true } ] } ``` #### Step 1b: Auto-Generate Eval Set If no eval set exists, generate one: 1. Read the skill's SKILL.md description and instructions 2. Run `python scripts/generate_eval_set.py <skill-path>` to produce a starter set 3. Present the generated set to the user for review and editing 4. User approves or modifies before proceeding ### Step 2: Set Up Workspace Create the eval workspace **outside** the skill directory to avoid confusing eval artifacts with skill files. Use a sibling directory or a dedicated location: ``` eval-workspace/ iteration-1/ eval-0/ eval_metadata.json # Asser