← ClaudeAtlas

a-combination-is-not-the-candidate-setlisted

Use once more than one trained artifact exists on disk -- two checkpoints, two seeds, two architectures, a continuation run -- and something is deciding which of them, or which combination of them, writes the predictions file. Covers the ballot that lists every artifact as a submission on its own before any blend, re-running it whenever a training job finishes, the known-bad canary that tests the objective, and persisting a rejected candidate's predictions.
tangxiangru/AutoR · ★ 805 · AI & Automation · score 79
Install: claude install-skill tangxiangru/AutoR
# Every artifact you built is a candidate submission; the combinations are extras A run that trains several models grows a write gate: an incumbent, a stream of challengers, a bar, a decision. The gate is the right instinct. It fails in a way that looks like discipline from the inside. `a-scoreable-file-in-the-first-hour` owns the incumbent-versus-challenger loop and `select-on-the-file-you-would-ship` owns the number it compares. This owns *which candidates enter it*. One run's gate ran at every new checkpoint and held every time. Its final cycle priced 3,184 weighted combinations of six arms, picked a four-way blend, and refused it. Every decision was correct on its own terms. The single best artifact in the workspace -- one checkpoint, alone, with the task's own post-processing on top -- was never entered as a candidate in its own right, and it scores 2.2 points above what shipped and above the scaffold-free control run of the same task on the same hardware. The gate was choosing carefully among things that all shared a defect. ## 1. Keep a ballot, and put the singletons on it first N artifacts means the candidate set *starts* as N singletons. Combinations are additions to it, never a replacement for it. ```python cands = {tag: single(tag) for tag in artifacts} # first, unconditionally cands |= {name: combo(w) for name, w in weight_grid} # then the search ``` Write the result as a file, one row per candidate -- tag, path to its persisted per-row scores, score