grpo-verifierlisted
Install: claude install-skill whilehq/whileai-sdk
# GRPO with a verifier reward
The reward is a function of the trajectory. It costs nothing per call and
can be wrong in one way: rejecting right answers or accepting wrong ones. So
the order is fixed: test the reward, freeze the held-out test, then generate,
select, export, train, score, report.
`check.py` runs every block below offline. Names it defines: `TOOLS`, `POLICY`,
`LIMIT`, `TRAIN_SEEDS`, `HELD_SEEDS`, `K` (rollouts per ask), `before_agent`
(the policy before training), `after_agent` (stands in for the trained
policy), `placeholder` (an agent whose replies are never scored), `points`
(pass@1 as score, 95% half-width, task count), `fake` (a recording transport),
`fake_grpo_loop` (a stand-in for the trainer), `OUT`.
## 1. Write the reward and unit-test it first
**The verifier reads the trajectory**: which tools ran, with what arguments,
in what order, and the final text. `@wai.verifier` wraps
`fn(candidate, reference, row) -> score` or `(score, reason)`; `candidate` is
the final text, `row["steps"]` the tool calls. Compose rules with
`verify.All([...])`.
**Three hand-written rows before anything else.** A reward that rejects a
right answer teaches the policy to avoid right answers, and no later step can
see it. `wai.audit_grades` does this at scale with a model judge, and
`select_for_rl(audit=...)` carries its warning.
```python
@wai.verifier
def refund_verifier(candidate: str, reference: Any, row: dict) -> tuple[float, str]:
"""Reads the trajectory: tool cal