promptfoo-evaluationlisted
Install: claude install-skill testland/qa-skills
# promptfoo-evaluation
[pf-intro]: https://promptfoo.dev/docs/intro/
[pf-config]: https://promptfoo.dev/docs/configuration/guide/
A `promptfooconfig.yaml` declares **providers** (LLMs under test),
**prompts** (templates with `{{var}}` placeholders), **tests**
(input variables + assertions); `promptfoo eval` runs the
cross-product (per [pf-config][pf-config]).
## When to use
- The repo has a `promptfooconfig.yaml` or the user wants to
author one.
- The user needs prompt regression suites - same inputs, multiple
providers, fail-on-diff.
- A CI workflow needs an eval gate on prompt or model changes.
- The team prefers vendor-neutral eval (vs OpenAI Evals' OpenAI-first
posture).
## Step 1 - Install
Per [github.com/promptfoo/promptfoo][pf-gh]:
[pf-gh]: https://github.com/promptfoo/promptfoo
```bash
npm install -g promptfoo
# or
brew install promptfoo
# or one-off
npx promptfoo@latest
```
Per [pf-gh][pf-gh]: "Node.js 20.20+ or 22.22+ is required for npm
and npx usage."
## Step 2 - First eval
Initialize from a built-in example, then run:
```bash
promptfoo init --example getting-started
cd getting-started
promptfoo eval
promptfoo view # opens HTML report
```
(Commands per [pf-gh][pf-gh].)
A minimal `promptfooconfig.yaml` per [pf-config][pf-config]:
```yaml
prompts:
- file://prompt1.txt
providers:
- openai:gpt-5-mini
- anthropic:claude-haiku-4-5
tests:
- vars:
language: French
input: Hello world
assert:
- type: contains-json
```
P