llm-output-gatelisted
Install: claude install-skill bakw00ds/yakos
# LLM Output Gate
## Purpose
Pair with `prompt-eval` and `prompt-injection-test` to enforce a CI-
side gate: the build fails if the prompt change regresses the golden
set beyond `<threshold>` percent on any rubric, OR if any injection
payload at or above `<injection-severity>` succeeded.
This is the "don't ship a worse prompt" lever. The gate is wired by
the `eval-engineer` into the project's CI workflow (GitHub Actions,
GitLab pipelines, etc.) and runs on PRs that touch `prompts/**`,
`.claude/agents/**`, or any rubric/dataset/corpus path.
## Scope
- Wraps `prompt-eval` + `prompt-injection-test` invocations and
composes their results into a single CI verdict.
- Reads project config from `<project>/eval/.gate.yaml`:
- `regression_threshold`: max acceptable per-rubric drop (default 0%).
- `aggregate_threshold`: max acceptable aggregate drop (default 2%).
- `injection_severity_floor`: severity at which a single hit fails
the gate (default `high`).
- `paths`: which file changes trigger the gate.
- `agents`: which agents to test (defaults to all production agents).
- Emits a comment-friendly markdown summary for the PR ("X rubrics
regressed; Y injection payloads succeeded; gate: FAIL").
- Exit code: 0 = green, 1 = regressed/jailbroken, 2 = config error.
- Designed for `eval-engineer` to wire into CI. The lead does not
invoke this skill in normal sessions.
## When to use
- As a required CI status check on PRs touching prompt / agent /
rubric / dataset p