benchmark-modelslisted
Install: claude install-skill rjmurillo/ai-agents
# benchmark-models
Run the same prompt across providers and compare them on data, not vibes. The
deterministic work lives in `scripts/model_benchmark.py` (stdlib-only, ported
from gstack's bun benchmark; no SDKs, no gstack runtime). This skill is the
interactive wrapper: pick a prompt, preview auth with a dry-run, confirm
providers, run, interpret, optionally save.
Providers wrap existing CLIs and their own auth (no API keys needed except the
optional judge):
- `claude` via `claude -p --output-format json`
- `gpt` via `codex exec ... -s read-only --json`
- `gemini` via `gemini -p ... --output-format stream-json`
## Triggers
Invoke this skill when the user says any of:
- `benchmark models`
- `compare models`
- `model shootout`
- `which model is best`
- `cross-model comparison`
Do NOT invoke for web page performance (that is a different concern).
## Preconditions
- At least one provider CLI installed and authenticated. The Step 1 dry-run
reports exactly which are ready; if none are, the skill stops.
- The optional quality judge needs `ANTHROPIC_API_KEY` (adds ~$0.05/run).
## Process
Resolve the driver first. In a repository checkout, use `BENCH="$(git rev-parse --show-toplevel)/.claude/skills/benchmark-models/scripts/model_benchmark.py"`. From an installed skill copy, use the local skill path: `BENCH="$PWD/scripts/model_benchmark.py"` when your shell is in the `benchmark-models` skill directory.
### Step 1: Pick a prompt and preview auth (dry-run)
Decide the prom