harness-searchlisted
Install: claude install-skill whilehq/whileai-sdk
# Harness search
Lee et al. 2026 (Meta-Harness, arXiv:2603.28052) search over harness code
with an agentic proposer that reads the source, scores and traces of every
prior candidate through a filesystem, and check the pick on held-out tasks
and held-out models. The recipe `recipes/papers/meta-harness` is the inner
half: it scores every file in `candidates/`, writes the ledger, the
proposal and the gate's answer. You are the outer half. Every step below is
in `check.py`, which runs the recipe's dry run on a temporary copy in under
a minute; its setup defines `RECIPE` (that copy), `MODEL` (the search
model), `ROUNDS` (3), `TRACES` (three days of the agent's traffic as a
JSONL, the shape `wai.load_traces` reads, or an OTLP JSON batch), `run(*flags)`
(the recipe with `--dry-run`), `propose_candidate(proposal)` (your job,
scripted there), `load_harness` (a candidate file as `wai.Harness`),
`judge` and `TOOLS` (the recipe's program judge and tool list), `NEXT_DAY`
and `DAY_AFTER` (the traffic the pick served the day after, and its date)
and `fake` (a platform that answers like the API).
## 1. Run the loop once on the traffic and read what it wrote
The frozen set is the traffic: one task per distinct prompt. The holdout
is its latest days, whole days, so no row the proposer reads comes from the
days that decide, and train prompts that overlap a holdout prompt leave the
proposer's window (Wang et al. 2026, arXiv:2607.12227: a harness tuned on
the tasks it is scored on gained 0.6 p