run-sefi-benchmarklisted
Install: claude install-skill xsefirosus/sefi-agents
# Run Sefi Benchmark
Operate the blinded paired A/B harness that compares the full sefi-agents chain against
one strong model on a frozen set of cases. The design of record and the trial-record
schema live in `benchmarks/README.md`; this skill is the run-and-report workflow.
User instructions always override this skill.
All factual output follows the anti-hallucination skill: cite or mark UNKNOWN, never
guess. Missing telemetry is `null`, never `0`.
## Cadence -- never inside a loop
Run this per release, or monthly, or when a human explicitly asks. It is manual and
out-of-loop by construction. **Never more than one benchmark run per day, AND only with
explicit per-invocation authorization** -- there is no standing pre-authorization, a day
passing does not auto-approve a run, and a prior approval never carries to the next run.
Same statement in `docs/BUDGET.md` and `docs/METRICS-PROVENANCE.md`. A `loops/*.loop.md`
spec MUST NOT call this skill: a benchmark run spends real tokens and time well past
`per_run_usd_cap`, and an unattended loop has no authority to approve that. If a loop
needs benchmark evidence, it parks a request in `inbox/` for a human.
Nothing mechanically prevents a loop from invoking this skill -- there is no runtime
guard that blocks a `loops/*.loop.md` from naming it. The "no loop may call this" rule is
model-compliance prose, backed by the explicit-per-invocation-authorization gate below
(step 2), which a compliant operator will not clear for an unatte