grill-skill

Featured

Build and harden a skill with evals — interview to design its eval tasks, then run, measure, and iterate. Use when the user wants to create or improve a skill's eval, or run the create → test → improve loop for a skill.

AI & Automation 182 stars 17 forks Updated today MIT

Install

View on GitHub

Quality Score: 87/100

Stars 20%
75
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Grill Skill Interview the user to design a skill's eval, then loop run → measure → improve until it ships. Requires `caliper` (`pipx install caliper-eval` if missing). Commands, spec skeleton, and expect/assert guidance: [REFERENCE.md](REFERENCE.md). ## Entry point `/grill-skill [path]` — optional path to a `SKILL.md`. - **Path given** — use it. - **No path** — look for `SKILL.md` in the cwd; if found, confirm before proceeding, else ask where it is. ## Phase 1 — Understand Read the `SKILL.md`. Summarize what it does, when it triggers, and what a successful run looks like. Ask the user to confirm your reading. **Wait for confirmation before continuing.** ## Phase 2 — Detect eval mode Look for `*.eval.yaml` beside the `SKILL.md` (try `<dir-name>.eval.yaml` first). - **None** → New eval. **Found** → Gap-fill. Interview one question at a time and wait for each answer. Never invent the user's answers or write the spec before interviewing. ### New eval — three tasks Elicit three tasks, one question at a time: 1. **Happy path** — the most common successful use. What did the agent do, and what would confirm it worked? 2. **Edge case** — a tricky-but-valid input that might trip the raw agent. 3. **Adversarial** — what the skill should refuse or avoid. Turn each answer into a task: a realistic `prompt`, an observable `expect`, and an `assert` when the outcome is checkable (see [REFERENCE.md](REFERENCE.md)). Show the proposed YAML and confirm before writing. Write the ...

Details

Author
edonadei
Repository
edonadei/caliper
Created
4 months ago
Last Updated
today
Language
Python
License
MIT

Similar Skills

Semantically similar based on skill content — not just same category