eval-comparelisted
Install: claude install-skill nacorga/venture-os
# Eval Compare
Arguments: `$ARGUMENTS`
Comparison ID: `$0`
Judge label: `$1`
If either is missing, stop and show:
```text
/eval-compare <compare-id> <judge-label>
```
Set `<compare-dir>` to `evals/compare/$0` and `<judgment-file>` to `<compare-dir>/judgments/$1.md`. If `<judgment-file>` already exists, stop: that label has already judged this comparison.
## Blindness
`<compare-dir>` holds two frozen runs of one case under neutral names, `X/` and `Y/`, the case both started from, and the comparison rubric, `compare.md`. Which run is which is sealed.
- Read only `<compare-dir>/compare.md`, `<compare-dir>/case.yaml`, `<compare-dir>/X/` and `<compare-dir>/Y/`.
- **Never open** `evals/compare/.keys/`, anything under `evals/runs/`, `<compare-dir>/manifest.json`, `<compare-dir>/result.json`, or another judge's file in `<compare-dir>/judgments/`.
- Never run `npm run eval:compare`. Unblinding happens outside judge sessions.
- Do not guess which model, framework version or order produced a run, and do not let a guess weigh on a preference.
Run this skill in a fresh Claude Code session that did not author either run and has not judged this comparison under another label.
## Judging
Follow `compare.md`: for each behavior, prefer X, Y or neither, and quote the passage from each run that decides it. A preference is not a score — two strong runs still have a better one when a difference can be quoted, and `tie` means no quotable difference matters.
Write `<judgment-file>` with