skill-evaluatorlisted
Install: claude install-skill HeshamFS/materials-simulation-skills
# Skill Evaluator
Test whether a skill is **correct, discoverable, and valuable** — not just whether
its unit tests pass. The harness is agent-agnostic: it drives whichever coding-agent
CLI the user uses, because Agent Skills are portable across all of them.
## When to use which layer
Three layers, increasing cost and fidelity (full rationale in
`references/methodology.md`):
| Layer | Question | Script | Needs a CLI? |
|------|----------|--------|--------------|
| 1. Deterministic | Do the scripts emit the **documented** numbers? | `run_script_checks.py` | No |
| 2. Trigger | Does the **description** activate on the right prompts? | `run_trigger_eval.py` | Yes |
| 3. Quality | Does following the SKILL.md **beat no skill**? | `run_quality_eval.py` → grade → `aggregate_benchmark.py` | Yes |
Always run Layer 1 (it's free). Add Layers 2–3 when you can run a coding-agent CLI.
## Step 0 — pick the agent CLI
Ask the user which coding agent they use, then map it to an adapter id. Supported:
`claude-code`, `openai-codex`, `antigravity` (the `agy` CLI that replaced Gemini
CLI on 2026-06-18), `cursor-cli`, `github-copilot-cli`, `amp`, `opencode`,
`grok-cli`. See the full matrix and auth in `references/adapters.md`, or run:
```bash
python scripts/agent_adapters.py list
```
Confirm the binary is installed and the auth env var is set (the matrix lists it).
**Before any real run, dry-run it** to see the exact command:
```bash
python scripts/agent_adapters.py build <agent> --prompt