← ClaudeAtlas

grpo-verifierlisted

Train an agent with GRPO when the reward is a program, and prove the gain. Use when a coding agent is asked to do RL on a rule code can check (a tool called before another, an argument in range, a phrase in the reply), to write or test a verifier reward, to export a verifiers environment for a GRPO trainer, or to report a trained version against a frozen held-out test with its training curve. Not for rewards that need a model judge.
whilehq/whileai-sdk · ★ 3 · AI & Automation · score 77
Install: claude install-skill whilehq/whileai-sdk
# GRPO with a verifier reward The reward is a function of the trajectory. It costs nothing per call and can be wrong in one way: rejecting right answers or accepting wrong ones. So the order is fixed: test the reward, freeze the held-out test, then generate, select, export, train, score, report. `check.py` runs every block below offline. Names it defines: `TOOLS`, `POLICY`, `LIMIT`, `TRAIN_SEEDS`, `HELD_SEEDS`, `K` (rollouts per ask), `before_agent` (the policy before training), `after_agent` (stands in for the trained policy), `placeholder` (an agent whose replies are never scored), `points` (pass@1 as score, 95% half-width, task count), `fake` (a recording transport), `fake_grpo_loop` (a stand-in for the trainer), `OUT`. ## 1. Write the reward and unit-test it first **The verifier reads the trajectory**: which tools ran, with what arguments, in what order, and the final text. `@wai.verifier` wraps `fn(candidate, reference, row) -> score` or `(score, reason)`; `candidate` is the final text, `row["steps"]` the tool calls. Compose rules with `verify.All([...])`. **Three hand-written rows before anything else.** A reward that rejects a right answer teaches the policy to avoid right answers, and no later step can see it. `wai.audit_grades` does this at scale with a model judge, and `select_for_rl(audit=...)` carries its warning. ```python @wai.verifier def refund_verifier(candidate: str, reference: Any, row: dict) -> tuple[float, str]: """Reads the trajectory: tool cal