llm-regression-suite-authorlisted
Install: claude install-skill testland/qa-skills
# llm-regression-suite-author
[pf-config]: https://promptfoo.dev/docs/configuration/guide/
[pf-testcases]: https://promptfoo.dev/docs/configuration/test-cases/
[pf-asserts]: https://promptfoo.dev/docs/configuration/expected-outputs/
[pf-ref]: https://promptfoo.dev/docs/configuration/reference/
[pf-cli]: https://promptfoo.dev/docs/usage/command-line/
[pf-openai]: https://promptfoo.dev/docs/providers/openai/
[pf-gha]: https://promptfoo.dev/docs/integrations/github-action/
[de-gh]: https://github.com/confident-ai/deepeval
[de-start]: https://deepeval.com/docs/getting-started
[de-datasets]: https://deepeval.com/docs/evaluation-datasets
A regression suite for LLM outputs pairs a **versioned golden dataset** (stable
inputs + expected outputs) with **deterministic eval settings** and a
**pass-rate gate** so that promoting a model from v1 to v2 surfaces quality
drops before they reach production. The per-tool skills
(`promptfoo-evaluation`, `deepeval-evaluation`) handle tool mechanics;
this skill handles the versioning workflow that neither covers.
## When to use
- A vendor announces a model update (e.g., `gpt-4.1` to `gpt-5`) and you
need a before/after quality comparison on your own inputs.
- A prompt regression suite exists but has no stable dataset version tied to
each model snapshot, so regressions are discovered in production.
- CI must gate a model-version bump the same way it gates a code change.
## Step 1 - Structure the golden dataset
A golden dataset is a version