design-a-rewardlisted
Install: claude install-skill ubik-dsh/deepseek-harness-skills
# Designing a reward
Everything a learner becomes, it becomes from the number you give it. The algorithm
is the easy half; five separate defects in one project were all in the reward and in
what the reward was attached to, and each one produced a confident wrong answer.
This skill is the procedure and the rules. It applies to reinforcement learning, to a
bandit, to a hill climb, to a fitness function, to any grader an agent is optimised
against.
---
## Step 1 — Write the claim down as a sentence, then try to beat it
A reward is a claim: *"more of this number means more of what I want."* Write that
sentence in the skill or the code before writing the function.
> Aiming at the bow of the crossing scores more than aiming at the straight midpoint.
Then **try to falsify it**, by hand, before the learner ever runs:
- what is the **cheapest** way to get a high number? Not the intended way — the
cheapest way.
- what does **doing nothing** score?
- what does **doing the maximum of everything** score?
- what scores well **without the task being done at all**?
`scripts/probe-reward.py` runs exactly this: your reward against a set of degenerate
and adversarial policies. Run it first. A reward that a lazy policy wins will train a
lazy agent, and it will do so smoothly and convincingly.
## Step 2 — Check the reward can fail
Same doctrine as any success test, and here it is not a formality: **the reward is
the success test**, and a weak reward does not merely fail to help — it