← ClaudeAtlas

harness-searchlisted

How a coding agent runs the Meta-Harness loop as the proposer on an agent's own production traffic: point the recipe at yesterday's traces, read out/proposal.md, write the next candidates/<n>_<name>.py, run the recipe, read the ledger, stop when the gate passes or after N rounds, report Changed / Moved / Why / Learned / Reproduce, post every candidate as a harness version, and score the next day's traffic on the pick. Use when the thing to improve is the harness (the prompt, the tools, the turn cap, the retry) and not the weights, on a closed model or an open one. No GPU, no key for the dry run.
whilehq/whileai-sdk · ★ 3 · AI & Automation · score 77
Install: claude install-skill whilehq/whileai-sdk
# Harness search Lee et al. 2026 (Meta-Harness, arXiv:2603.28052) search over harness code with an agentic proposer that reads the source, scores and traces of every prior candidate through a filesystem, and check the pick on held-out tasks and held-out models. The recipe `recipes/papers/meta-harness` is the inner half: it scores every file in `candidates/`, writes the ledger, the proposal and the gate's answer. You are the outer half. Every step below is in `check.py`, which runs the recipe's dry run on a temporary copy in under a minute; its setup defines `RECIPE` (that copy), `MODEL` (the search model), `ROUNDS` (3), `TRACES` (three days of the agent's traffic as a JSONL, the shape `wai.load_traces` reads, or an OTLP JSON batch), `run(*flags)` (the recipe with `--dry-run`), `propose_candidate(proposal)` (your job, scripted there), `load_harness` (a candidate file as `wai.Harness`), `judge` and `TOOLS` (the recipe's program judge and tool list), `NEXT_DAY` and `DAY_AFTER` (the traffic the pick served the day after, and its date) and `fake` (a platform that answers like the API). ## 1. Run the loop once on the traffic and read what it wrote The frozen set is the traffic: one task per distinct prompt. The holdout is its latest days, whole days, so no row the proposer reads comes from the days that decide, and train prompts that overlap a holdout prompt leave the proposer's window (Wang et al. 2026, arXiv:2607.12227: a harness tuned on the tasks it is scored on gained 0.6 p