a-model-you-can-audit-is-not-a-model-that-scores

# The auditable method wins the argument and loses the score Given a fixed clock and a scored predictions file, there is a recurring choice between two methods: - one you can build in twenty minutes, cross-validate cleanly, explain fully, and defend against every question a reviewer asks; - one the field actually uses to get the published number, which needs an hour of setup you have not done, has failure modes you cannot fully enumerate, and might not finish. Every incentive inside a rigour-checking pipeline points at the first. The gates reward an auditable choice. The reviewer is easier to satisfy. The write-up is cleaner. And on a benchmark that grades predictions, none of that is measured. Measured on a scored arm: the pipeline shipped a gradient-boosted tree on hand-built features, with a set of scripts beside it named for what they audited — the train/test alignment, the units, the row alignment, the submission state. The control arm, on the same task with the same clock, wrote a graph network. The control won by a fifth of the normalized range. ## Price it; do not judge it The decision is not "is the expensive method better" — you already know it is. It is "does it fit". That is a measurement, and it takes ten minutes: 1. **Run one unit of it.** One epoch on 1% of the data, or one forward pass on one batch, timed. Not an estimate from experience; the actual wall clock on this machine, which is the only one whose speed matters. 2. **Extrapolate to th...