← ClaudeAtlas

a-null-test-bounds-the-instrument-not-the-answerlisted

Use at analysis and again at writing whenever you run a permutation, shuffle, placebo, unforced-control or power test against your own headline result, especially when it comes back saying the result is not distinguishable from noise. Covers giving every condition the task names its own value line and stating the relation across them as a result, keeping the estimate and the bound as two results with two different subjects, and the sentence order that stops a bound replacing the answer.
tangxiangru/AutoR · ★ 805 · AI & Automation · score 79
Install: claude install-skill tangxiangru/AutoR
# Give every condition its value line, then bound it ## The spine For every condition the task names — every scenario, arm, dose, cohort, model, regime — the report carries one line of this shape: condition: value [interval] vs reference value, gap Including conditions you could only reach by derivation, interpolation or borrowing. A derived condition gets its line with the derivation named in the same clause. It does not get a footnote in place of a number. Then, in its own sentence, **the relation across conditions, stated as a result**: the ordering, the monotonicity, the crossing — whichever relation the task's comparison is about. Over all the conditions the task named, not the subset your claim structure happened to scope. This is where runs holding correct numbers lose. The results file has the values in the right relation; the preregistration scoped the ordering claim to a subset because one condition arrived late or by derivation; and the report never states the full relation anywhere. The relation is a separate result from the values, and having the values on disk is not stating it. A condition your hypothesis excluded is still a condition the task asked about. `reproduce-then-extend` owns the shape of the published-versus-ours table these lines live in. Do not restate it; fill it. ## Two results with two different subjects Building a no-signal null against your own criterion — permuting the labels that carry the contrast, an unforced control run, a pla